Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7643381b9b | ||
|
|
1430f2a040 | ||
|
|
fc099d9b3c | ||
|
|
d2587e1d29 | ||
|
|
7c4dbebe24 | ||
|
|
860386d283 |
No files matched your search
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=262144` pool** (the card's recommendation: the ledger's 7.2 GiB where the old 524,288 pool measured 14.6 — per-request ceiling unchanged at 262,144, two maximum-length conversations no longer coexist, and the freed RAM is page cache for the n-gram table), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=200704` pool at `HALOGEN_CTX=200704`** (lowered from the card's 262,144 the same evening — 5.5 GiB reserved where 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; pool == ctx so ONE maximum-length request fills it, and the freed RAM is page cache for the n-gram table), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
||||||
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
@@ -380,25 +380,29 @@ endpoint).
|
|||||||
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
||||||
policy disables the MTP head for whole requests after low acceptance and
|
policy disables the MTP head for whole requests after low acceptance and
|
||||||
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
||||||
for it (accepted). `HALOGEN_CTX=262144` is the PER-SLOT hard ceiling (400
|
for it (accepted). `HALOGEN_CTX=200704` is the PER-REQUEST hard ceiling (400
|
||||||
naming the limit, never truncation) — RAISED from 200192 on 2026-10-07
|
naming the limit, never truncation) — the checkpoint's native window is
|
||||||
under the 400-zone law: the harness clamps max_tokens to
|
262,144 and was served until 2026-10-17, when Frank lowered it for RAM. The
|
||||||
|
**400-zone law is invariant to the value**: the harness clamps max_tokens to
|
||||||
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
|
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
|
||||||
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
|
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
|
||||||
request 43,158 vs room 42,804), so a harness window == engine ceiling
|
request 43,158 vs room 42,804), so a harness window == engine ceiling
|
||||||
made every request over ~146k est a guaranteed 400. `models.yml`
|
makes every long request a guaranteed 400. `models.yml`
|
||||||
`contextWindow: 217088` leaves the 45k gap that absorbs the estimate
|
`contextWindow: 155648` (= ceiling − 45056) keeps that gap (worst-case
|
||||||
error (worst-case request ≈ 219k ≤ 262144); compaction fires at 85 % of
|
request ≈ 156.2k ≤ 200,704); compaction fires at 85 % of the harness window
|
||||||
the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
|
= 132,301 est ≈ 136k measured (the 180k target died with the 262,144 window).
|
||||||
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
||||||
law does NOT carry over; batching and speculation coexist (the card's
|
law does NOT carry over; batching and speculation coexist (the card's
|
||||||
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
||||||
`HALOGEN_KV_POOL_POSITIONS=262144` (the card's recommendation and the memory
|
`HALOGEN_KV_POOL_POSITIONS=200704` — **lowered from the card's 262,144 the same
|
||||||
saving of the 2026-10-17 swap; ~28 KiB/position ⇒ 7.2 GiB) is **ONE pool
|
evening** (Frank's call: every position costs KV RAM and nothing in the fleet
|
||||||
behind both slots** — ledger `2 slots over ONE 262144-position KV pool, each
|
asks for the biggest window) — is **ONE pool behind both slots**, ledger form
|
||||||
request up to 262144` — so the per-request ceiling is untouched while two
|
`2 slots over ONE <pool>-position KV pool, each request up to HALOGEN_CTX`, and
|
||||||
MAXIMUM-length conversations no longer coexist (the old 524,288 = 2 × ctx
|
at pool == ctx a single MAXIMUM-length request fills it: two maximum-length
|
||||||
spent the measured 14.6 GiB and allowed it).
|
conversations never coexist (two of ≤100k do). ~28 KiB/position ⇒ the ledger
|
||||||
|
reserves "about 5.5 GiB", where 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6; the
|
||||||
|
engine's preflight FORMULA bills slightly more than the reservation (block
|
||||||
|
scratch + ~115 MiB/slot of O(1) state) — quote the reservation.
|
||||||
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||||
@@ -415,16 +419,17 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
|||||||
`image_url` content parts only; ≥256 px probe images or the model answers
|
`image_url` content parts only; ≥256 px probe images or the model answers
|
||||||
"video".
|
"video".
|
||||||
|
|
||||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger — the
|
||||||
weights ~62.9 (trunk 62.11 + vision 0.84) + KV 7.2 + arena ~8
|
200,704 boot printed exactly): weights 62.9 (trunk 62.11 + vision 0.84) + KV
|
||||||
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ 78 GiB engine-side +
|
5.5 + working 5.7 (`3.8 + 8.4 × MAX_TOK/16384`) = **74.1 GiB in all**, the
|
||||||
`HALOGEN_HOST_RESERVE_GIB=20` → ~25 GiB margin over baseline co-tenancy.
|
~1.0 GiB prompt cache booked separately by the ledger, +
|
||||||
The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint
|
`HALOGEN_HOST_RESERVE_GIB=20` → ~30 GiB
|
||||||
(ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why the pool
|
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd page
|
||||||
cut is a double saving: **RAM the pool takes is RAM that table loses**.
|
cache beside the checkpoint (ledger: `mapped 51.2 GB, NOT pinned
|
||||||
`HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the v2 loader, so disk-backing
|
(Pin::None)`) — which is why every pool cut is a double saving: **RAM the pool
|
||||||
the trunk is not a lever on this format. READ the boot ledger and re-size
|
takes is RAM that table loses**. `HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the
|
||||||
if the engine prints a clamp.
|
v2 loader, so disk-backing the trunk is not a lever on this format. READ the
|
||||||
|
boot ledger and re-size if the engine prints a clamp.
|
||||||
|
|
||||||
Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
|
Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
|
||||||
`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
|
`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
|
||||||
@@ -1103,9 +1108,15 @@ linked runbooks.
|
|||||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
||||||
ports off the gluetun service.
|
ports off the gluetun service.
|
||||||
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
|
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
|
||||||
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
|
2026-09-11) retires the XDNA2 NPU: the driver's SVA/PASID bind needs an
|
||||||
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
|
enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). That trade is on an
|
||||||
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
expiring path: **AMD PerfOpt** (iommu tree 2026-09-24, Linux 7.4) stops
|
||||||
|
iGPU GPA→SPA translation while the IOMMU stays enabled for every other
|
||||||
|
device, so the next boot-line window is `iommu=pt` with the NPU usable —
|
||||||
|
measured only 2–4 % residual on a Strix Halo desktop (18–23 % on Strix
|
||||||
|
Point laptops). Not installable here yet: Ubuntu 26.04 caps at
|
||||||
|
`7.0.0-38`. Full research in
|
||||||
|
`serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
||||||
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
|
- Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
|
||||||
only the CURRENT instance; history across recreations lives in
|
only the CURRENT instance; history across recreations lives in
|
||||||
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
|
`journalctl --user -u <name>.service`. Read rootless storage WITHOUT sudo
|
||||||
@@ -1156,7 +1167,7 @@ Active:
|
|||||||
|
|
||||||
| Service | Runbook |
|
| Service | Runbook |
|
||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 262,144 pool, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
|
| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 200,704 pool at ctx 200704, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
|
||||||
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
||||||
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
@@ -1169,7 +1180,7 @@ Retired (rollback targets and history):
|
|||||||
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, 2 slots × 200,192 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
|
| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, TWO slots at ctx 262,144 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||||
|
|||||||
@@ -1097,9 +1097,9 @@ deploy_anubis() {
|
|||||||
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next —
|
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next —
|
||||||
# qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident)
|
# qwen38-flash-next-v2-abliterated.hgn (66,687,678,432 B, ~62 GiB resident)
|
||||||
# + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode
|
# + the REQUIRED n-gram lookup table + the vision tower — MTP spec decode
|
||||||
# (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 262,144-position KV pool, image
|
# (HALOGEN_SPEC_ADAPT=0), 2 slots over ONE 200,704-position KV pool at the
|
||||||
# input — on the Strix Halo iGPU. It stays a ROOT unit because the serve
|
# matching ctx 200704 ceiling, image input — on the Strix Halo iGPU. It stays a
|
||||||
# script drives `podman run` directly (the frank-archon pattern):
|
# ROOT unit because the serve script drives `podman run` directly (the frank-archon pattern):
|
||||||
# relink_unit points /etc/systemd/system/frank-dragoon.service at the repo
|
# relink_unit points /etc/systemd/system/frank-dragoon.service at the repo
|
||||||
# file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity
|
# file, boot persistence is that unit's [Install], and LimitMEMLOCK=infinity
|
||||||
# is fleet parity. 2026-10-17 cutover from the Swift1.5-finetune shape —
|
# is fleet parity. 2026-10-17 cutover from the Swift1.5-finetune shape —
|
||||||
|
|||||||
+50
-42
@@ -20,6 +20,10 @@
|
|||||||
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
|
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
|
||||||
# base thinks ~1.9x longer per answer.
|
# base thinks ~1.9x longer per answer.
|
||||||
#
|
#
|
||||||
|
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered both
|
||||||
|
# to 200,704 (RAM + the harness's compaction budget); the rates above are the
|
||||||
|
# ceiling-262,144 record and the production ledger is the 200,704 one.
|
||||||
|
#
|
||||||
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
|
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
|
||||||
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
|
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
|
||||||
# .hgn loads ONLY in the flash-server line (the old halogen engine is for
|
# .hgn loads ONLY in the flash-server line (the old halogen engine is for
|
||||||
@@ -29,50 +33,54 @@
|
|||||||
# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
|
# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
|
||||||
# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
|
# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
|
||||||
# red-PNG probe verified on the card; the 27B engine ignored its encoder),
|
# red-PNG probe verified on the card; the 27B engine ignored its encoder),
|
||||||
# tool calls verified, and the native 262,144 context.
|
# tool calls verified, and the native 262,144 context — served at 200,704 by
|
||||||
|
# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
|
||||||
|
# capability limit).
|
||||||
#
|
#
|
||||||
# SHAPE — the knobs that are NOT free, with the arithmetic:
|
# SHAPE — the knobs that are NOT free, with the arithmetic:
|
||||||
# HALOGEN_CTX=262144 is the PER-SLOT window and the per-conversation hard
|
# HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
|
||||||
# ceiling — a prompt over it is a 400 naming the limit, never a silent
|
# hard limit — a prompt over it is a 400 naming the limit, never a silent
|
||||||
# truncation. RAISED from 200192 on 2026-10-07: the harness clamps
|
# truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
|
||||||
# max_tokens to contextWindow − est − 4096, its chars/4 est UNDER-counts
|
# checkpoint's native window is 262,144 and the engine serves anything
|
||||||
# the rendered prompt by ~3 % on code/agent traffic, and compaction only
|
# below it): every position costs KV pool RAM and shrinks the prompt-cache
|
||||||
# fires at 85 % — so with window == engine ceiling, every request over
|
# entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
|
||||||
# ~146k est (65-85 % of the window) was a guaranteed 400. Measured
|
# to the value — it says the harness window must sit ~45k UNDER the engine
|
||||||
# 2026-10-07: est 152,938, server 157,388, request 43,158 vs room 42,804
|
# ceiling, because the harness clamps max_tokens to contextWindow − est −
|
||||||
# — over by 354, margin 4,096 < the 4,450 under-count.
|
# 4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
|
||||||
# Harness contract (models.yml): contextWindow 217088, maxTokens 45056.
|
# code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
|
||||||
# Compaction fires at 85 % = 184,524 est ≈ 190k measured (Frank's 180k
|
# at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
|
||||||
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
# Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
|
||||||
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
# the gap that absorbs the estimate error), maxTokens 45056.
|
||||||
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
# Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
|
||||||
# 10 % under-count). Compaction rides on frank-zealot: input ≈
|
# of the 262,144 era is unreachable at this ceiling: the window itself is
|
||||||
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
# 155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
|
||||||
# HALOGEN_KV_POOL_POSITIONS=262144 is the card's recommended pool and the
|
# est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
|
||||||
# memory saving of this swap. One budget behind HALOGEN_KV_SLOTS=2 (boot
|
# ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
|
||||||
# ledger: "2 slots over ONE 262144-position KV pool, each request up to
|
# keepRecent(20k) ≈ 112k, inside its 200,704 slot.
|
||||||
# 262144"), so the per-request ceiling stays HALOGEN_CTX while two
|
# HALOGEN_KV_POOL_POSITIONS=200704 — ONE pool behind HALOGEN_KV_SLOTS=2
|
||||||
# MAXIMUM-length conversations can no longer coexist — two moderate ones
|
# (ledger form: "2 slots over ONE <pool>-position KV pool, each request up to
|
||||||
# can. ~28 KiB/position (FLAGS.md) => 7.2 GiB, and the ledger said exactly
|
# HALOGEN_CTX"). With pool == ctx a single MAXIMUM-length request fills the
|
||||||
# that: "reserving the KV pool, 262144 positions (about 7.2 GiB)". The
|
# whole pool. ~28 KiB/position (FLAGS.md) ⇒ ~5.4 GiB, where 262144 cost 7.2
|
||||||
# 2026-10-06 shape's 524,288 cost 14.6 GiB; FLAGS.md law — "RAM the pool
|
# and the 2026-10-06 shape's 524,288 measured 14.6. The engine's own preflight
|
||||||
# takes is RAM that table loses" — makes the 7.4 GiB handed back page
|
# formula bills a little more than the reservation because it adds block
|
||||||
# cache for the 47.7 GiB n-gram table, which is what long prompts read.
|
# scratch and ~115 MiB/slot of O(1) state — quote the RESERVATION for the RAM
|
||||||
|
# ledger. FLAGS.md law: "RAM the pool takes is RAM that table loses", so every
|
||||||
|
# position cut is page cache for the 47.68 GiB n-gram table long prompts read.
|
||||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||||
# quickstart ships 2 slots WITH the head), so the second slot is
|
# quickstart ships 2 slots WITH the head), so the second slot is
|
||||||
# concurrent capacity, not a speculation tax.
|
# concurrent capacity, not a speculation tax.
|
||||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
|
||||||
# card's own quickstart arithmetic does not bill it, and the boot ledger
|
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
||||||
# agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~5.4 +
|
||||||
# weights ~62 GiB resident + KV 7.2 + arena (HALOGEN_MAX_TOK=8192 =>
|
# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 74 GiB engine-side, with the
|
||||||
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
# prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked
|
||||||
# = ~79 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
# separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||||
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
|
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
|
||||||
# ~10-15. ~25 GiB margin. The boot ledger is the referee — read it
|
# ~10-15. ~30 GiB margin. The boot ledger is the referee — read it after
|
||||||
# after deploy and re-size if the engine prints a clamp.
|
# deploy and re-size if the engine prints a clamp.
|
||||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||||
# turns the MTP head off for whole requests after low acceptance and
|
# turns the MTP head off for whole requests after low acceptance and
|
||||||
# HALVES decode as a result. Frank accepted the head always-on trade
|
# HALVES decode as a result. Frank accepted the head always-on trade
|
||||||
@@ -99,11 +107,11 @@
|
|||||||
# so the models.yml fold (high->xhigh) still applies.
|
# so the models.yml fold (high->xhigh) still applies.
|
||||||
# * Vision: OpenAI image_url content parts only (the fleet's standard
|
# * Vision: OpenAI image_url content parts only (the fleet's standard
|
||||||
# contract). >=256 px probe images or the model answers "video".
|
# contract). >=256 px probe images or the model answers "video".
|
||||||
# * TWO maximum-length conversations no longer share the endpoint: the KV
|
# * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
|
||||||
# pool is one 262,144-position budget behind 2 slots. The PER-REQUEST
|
# budget behind 2 slots — exactly one maximum-length request fills it. Two
|
||||||
# ceiling is unchanged (HALOGEN_CTX), so the harness contract below
|
# conversations coexist only while they stay under ~100k each. The
|
||||||
# (contextWindow 217088) does not move; only concurrency at full depth
|
# PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
|
||||||
# does, and this endpoint carries the slow + security-reviewer roles.
|
# contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
|
||||||
# * response_format json_object/json_schema unsupported in this line
|
# * response_format json_object/json_schema unsupported in this line
|
||||||
# (no constrained decoding) — unchanged from the 27B engine.
|
# (no constrained decoding) — unchanged from the 27B engine.
|
||||||
# * The engine's token protocol has no auth and binds loopback INSIDE
|
# * The engine's token protocol has no auth and binds loopback INSIDE
|
||||||
@@ -165,8 +173,8 @@ FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_conf
|
|||||||
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
|
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
|
||||||
# model_ids for the :7770 backend MUST name this value.
|
# model_ids for the :7770 backend MUST name this value.
|
||||||
HALOGEN_MODEL_ID=frank-dragoon
|
HALOGEN_MODEL_ID=frank-dragoon
|
||||||
HALOGEN_CTX=262144
|
HALOGEN_CTX=200704
|
||||||
HALOGEN_KV_POOL_POSITIONS=262144
|
HALOGEN_KV_POOL_POSITIONS=200704
|
||||||
HALOGEN_KV_SLOTS=2
|
HALOGEN_KV_SLOTS=2
|
||||||
HALOGEN_HOST_RESERVE_GIB=20
|
HALOGEN_HOST_RESERVE_GIB=20
|
||||||
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 262144-position pool, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
Description=halogen-flash-server container (Quat3rnion NATIVE v2 abliterated Qwen3.8-Flash-Next, MTP spec decode, 2 slots over ONE 200704-position pool at ctx 200704, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
|
|||||||
@@ -82,3 +82,14 @@ real IdP credentials.
|
|||||||
5. **Solve-once cookie.** Since Anubis v1.27.0 + `COOKIE_DOMAIN` (see
|
5. **Solve-once cookie.** Since Anubis v1.27.0 + `COOKIE_DOMAIN` (see
|
||||||
[[Anubis]]), one PoW per fresh context covers every guarded vhost — crawl
|
[[Anubis]]), one PoW per fresh context covers every guarded vhost — crawl
|
||||||
and SSO specs complete in ~0.5 s per vhost instead of a PoW each.
|
and SSO specs complete in ~0.5 s per vhost instead of a PoW each.
|
||||||
|
6. **Apps navigate too — and may leave.** `12-crawl` returns to the homepage
|
||||||
|
between links, and Plex Web (signed out in the `qa-e2e` profile) runs its own
|
||||||
|
sign-in flow: it redirects the top frame, so Chromium cancelled our
|
||||||
|
`goto("/")` with `net::ERR_ABORTED` (once as a 420 s timeout, once at 5 s on
|
||||||
|
the same link), and the same redirect then lands the browser on
|
||||||
|
`app.plex.tv` — Plex's hosted surface for a session-less client — which broke
|
||||||
|
the `hostname == target` law. Two narrow answers: `backHome()` re-issues the
|
||||||
|
navigation on that one error class, and the landing check takes an explicit
|
||||||
|
hand-off list (`HANDOFFS`) instead of a single host. The guard contract is
|
||||||
|
untouched: challenge cleared, no authentik `uidField`, status < 500, and every
|
||||||
|
host NOT in the list still has to land on its own vhost.
|
||||||
@@ -89,6 +89,70 @@ with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
|
|||||||
endpoint is wanted back. The IOMMU change is independent of the halogen
|
endpoint is wanted back. The IOMMU change is independent of the halogen
|
||||||
deployment — reverting one does not require reverting the other.
|
deployment — reverting one does not require reverting the other.
|
||||||
|
|
||||||
|
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
|
||||||
|
|
||||||
|
Asked and answered — **the decision changes, the host does not.** Nothing was
|
||||||
|
changed on Dankclaw today; this section records why the 13–16 % figure is on an
|
||||||
|
expiring path.
|
||||||
|
|
||||||
|
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
|
||||||
|
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
|
||||||
|
extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation
|
||||||
|
for the *privileged integrated GPU* — only the IR/IW permission bits are
|
||||||
|
enforced — while the IOMMU stays fully enabled for every other device, the NPU
|
||||||
|
included. Driver: `drivers/iommu/amd/{iommu.c,amd_iommu_types.h}`. The only
|
||||||
|
consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and
|
||||||
|
the identity domain**, and the shared per-IOMMU bit is refcounted so one
|
||||||
|
device's teardown cannot clear it under a peer.
|
||||||
|
|
||||||
|
**Status.** v2 merged through the iommu tree 2026-09-24, present in
|
||||||
|
`linux-next-20261016`, queued for **Linux 7.4** (merge window late October 2026,
|
||||||
|
stable ~end of year, likely LTS candidate). NOT available here: `/boot/
|
||||||
|
System.map-7.0.0-38-generic` contains zero `perfopt` symbols, and Ubuntu 26.04's
|
||||||
|
archives cap at `7.0.0-38.38` for both `linux-image-generic` and
|
||||||
|
`linux-generic-hwe-26.04-edge` — reaching 7.4 means a mainline PPA or a local
|
||||||
|
build. A 2026-09-28 follow-up removes the `amdgpu.iommu_perfopt` knob and makes
|
||||||
|
PerfOpt compulsory.
|
||||||
|
|
||||||
|
**Measured.** Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel,
|
||||||
|
`amdgpu.iommu_perfopt=0` vs default): **18–23 %** on Strix Point / Krackan
|
||||||
|
laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our
|
||||||
|
silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02).
|
||||||
|
|
||||||
|
**The September number was never strict-flush cost.** Ubuntu ships
|
||||||
|
`CONFIG_IOMMU_DEFAULT_DMA_LAZY=y` (STRICT not set), so IOTLB invalidation was
|
||||||
|
already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce.
|
||||||
|
What we measured was **translated-domain** cost — and `iommu=pt` avoids that
|
||||||
|
today, with or without PerfOpt. The March attempt to make passthrough the AMD
|
||||||
|
default was rejected outright, so `iommu=pt` / `iommu.passthrough=1` stays the
|
||||||
|
sanctioned route.
|
||||||
|
|
||||||
|
**The NPU is compatible with identity mode.** `amdxdna` binds through
|
||||||
|
`iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
|
||||||
|
**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`),
|
||||||
|
and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that
|
||||||
|
is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA
|
||||||
|
failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7
|
||||||
|
disables SVA outright**; skip that range.
|
||||||
|
|
||||||
|
**Container access needs no reboot to settle:** `/dev/accel/accel0` already
|
||||||
|
exists as `crw-rw---- root:render 261,0` and `dank` is in `render`, so a quadlet
|
||||||
|
wants `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same
|
||||||
|
pattern ComfyUI uses for `/dev/kfd`.
|
||||||
|
|
||||||
|
**Revised procedure for the next boot-line window** (still a maintenance window,
|
||||||
|
still an explicit OK per run):
|
||||||
|
|
||||||
|
1. `iommu=pt` in place of `amd_iommu=off` — NPU usable, translated cost gone,
|
||||||
|
residual 2–4 % on this silicon class.
|
||||||
|
2. Re-measure prefill against the **current** payload. The 13–16 % belongs to the
|
||||||
|
2026-09-11 shape; the 2026-10-17 native-v2 boot runs 1,513–1,604 t/s cold
|
||||||
|
prefill, so the old baseline is not comparable.
|
||||||
|
3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
|
||||||
|
4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity
|
||||||
|
domain. Confirm with `dmesg | grep -i perfopt`; force off with
|
||||||
|
`amdgpu.iommu_perfopt=0` while the knob still exists.
|
||||||
|
|
||||||
## Related
|
## Related
|
||||||
|
|
||||||
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
||||||
@@ -98,3 +162,9 @@ deployment — reverting one does not require reverting the other.
|
|||||||
## Citations
|
## Citations
|
||||||
|
|
||||||
- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
|
- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
|
||||||
|
- [iommu/amd: Add PerfOpt IOMMU performance optimization support (v2)](https://lists.freedesktop.org/archives/amd-gfx/2026-September/152501.html) — series 0/2 at [/153378.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153378.html), merged via the iommu tree 2026-09-24 ([/153844.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153844.html))
|
||||||
|
- [Compulsory-PerfOpt follow-up (removes `amdgpu.iommu_perfopt`)](https://patchew.org/linux/20260928045050.955165-1-superm1@kernel.org/)
|
||||||
|
- [Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4](https://www.phoronix.com/review/amd-perfopt) — Strix Halo desktop arm (2–4 %), `amdgpu.iommu_perfopt=0` A/B; news write-up [here](https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4)
|
||||||
|
- [Robin Murphy rejecting the AMD passthrough-default patch](https://lkml.org/lkml/2026/3/26/924) — `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned path
|
||||||
|
- [Debian wiki: XDNA](https://wiki.debian.org/XDNA) — driver + firmware upstreamed in 6.14; **amdxdna broken on 6.18–6.18.7** by an IOMMU regression that disables SVA ([report](https://lore.kernel.org/regressions/870872aa-28e9-412a-bac6-8020bf560e4f@amd.com/))
|
||||||
|
- [amd/xdna-driver#704](https://github.com/amd/xdna-driver/issues/704) — the `err=-19` bind failure when the IOMMU is disabled
|
||||||
@@ -66,9 +66,11 @@ ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b8680670
|
|||||||
|
|
||||||
## Shape — the card's settings, and the three deliberate deviations
|
## Shape — the card's settings, and the three deliberate deviations
|
||||||
|
|
||||||
`HALOGEN_CTX=262144` (per-request ceiling), `HALOGEN_KV_SLOTS=2`,
|
`HALOGEN_CTX=200704` (per-request ceiling — the card's 262,144 lowered the
|
||||||
**`HALOGEN_KV_POOL_POSITIONS=262144`** (the card's recommendation, and the
|
same evening, see below), `HALOGEN_KV_SLOTS=2`,
|
||||||
memory saving), `HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
**`HALOGEN_KV_POOL_POSITIONS=200704`** (one pool behind both slots; the card
|
||||||
|
recommends it equal to ctx, and that equality is where the memory is),
|
||||||
|
`HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
||||||
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
|
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
|
||||||
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
|
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
|
||||||
`HALOGEN_HOST_RESERVE_GIB=20`.
|
`HALOGEN_HOST_RESERVE_GIB=20`.
|
||||||
@@ -81,24 +83,30 @@ Deviations from the card, each on purpose:
|
|||||||
(the adaptive default disables the drafter per request and halves decode).
|
(the adaptive default disables the drafter per request and halves decode).
|
||||||
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
|
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
|
||||||
|
|
||||||
**The pool is one budget, not a partition.** Boot ledger: `listening on
|
**The pool is one budget, not a partition.** Boot ledger form: `listening on
|
||||||
127.0.0.1:8730, 2 slots over ONE 262144-position KV pool, each request up to
|
127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to
|
||||||
262144`. The per-request ceiling is unchanged, so the harness contract
|
<HALOGEN_CTX>`. Both numbers were lowered to **200,704** the same evening
|
||||||
(`contextWindow: 217088`) does not move — what changed is that **two
|
(the card ships 262,144; Frank took the RAM — nothing else in the fleet asks
|
||||||
maximum-length conversations can no longer share the endpoint**. That is
|
for the bigger window), so `models.yml` `contextWindow` moved to **155,648**
|
||||||
acceptable here because frank-dragoon carries the `slow` (medium-thinking)
|
= ceiling − 45k with it. With pool == ctx, one maximum-length request fills
|
||||||
role and the `security-reviewer` subagent, not the interactive workhorse
|
the pool: **two long conversations can no longer share the endpoint** (two of
|
||||||
(that is frank-archon).
|
≤100k can). That is acceptable here because frank-dragoon carries the `slow`
|
||||||
|
(medium-thinking) role and the `security-reviewer` subagent, not the
|
||||||
|
interactive workhorse (that is frank-archon).
|
||||||
|
|
||||||
**Memory ledger** (121.2 GiB unified), from the boot ledger of the live
|
**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
|
||||||
candidate: `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||||
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
|
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
|
||||||
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
|
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
|
||||||
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …`. The n-gram
|
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in
|
||||||
table rides as page cache — `mapped 51.2 GB, NOT pinned (Pin::None)` — which
|
all`. The cut boot printed the same shape at the new size: `reserving the KV
|
||||||
is exactly why cutting the pool is a *double* saving. FLAGS.md law: **RAM the
|
pool, 200704 positions (about 5.5 GiB)` and `… 5.5 GiB of KV pool, 5.7 GiB of
|
||||||
pool takes is RAM that table loses**; the 7.4 GiB the smaller pool hands back
|
working memory, 74.1 GiB in all`.
|
||||||
is page cache for the 47.68 GiB table that long prompts read.
|
The n-gram table rides as page cache — `mapped 51.2 GB,
|
||||||
|
NOT pinned (Pin::None)` — which is exactly why every pool cut is a *double*
|
||||||
|
saving. FLAGS.md law: **RAM the pool takes is RAM that table loses**; the ~9
|
||||||
|
GiB the two cuts together handed back is page cache for the 47.68 GiB table
|
||||||
|
that long prompts read.
|
||||||
|
|
||||||
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||||
stays pinned. Disk-backing the weights is not a lever on this format.
|
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||||
@@ -185,3 +193,79 @@ mandatory); unknown sampling params are a 400; the prompt cache is on by
|
|||||||
default, so a warm 100k follow-up costs seconds. See
|
default, so a warm 100k follow-up costs seconds. See
|
||||||
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
||||||
carries verbatim, and [[deployment]] for the deploy/health gate.
|
carries verbatim, and [[deployment]] for the deploy/health gate.
|
||||||
|
|
||||||
|
## Production boot — the deploy ledger (2026-10-17 22:20, `fcd7630`)
|
||||||
|
|
||||||
|
`./scripts/deploy.sh dankclaw --service frank-dragoon`: healthy after ~12 s,
|
||||||
|
deploy marker `fcd7630`, e2e gate 33/33. The trial drop-in went to
|
||||||
|
`~/.AGENTTRASH/20261017-v2abl-trial/` FIRST, so this boot is governed by the
|
||||||
|
repo env — `systemctl show -p DropInPaths` is empty.
|
||||||
|
|
||||||
|
Ledger lines that settle the shape (engine stdout, not the journal):
|
||||||
|
|
||||||
|
|Line|Figure|
|
||||||
|
|---|---|
|
||||||
|
|preflight|`MemAvailable now 105.4 GiB` against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache|
|
||||||
|
|weights|`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)`|
|
||||||
|
|KV pool|`reserving the KV pool, 262144 positions (about 7.2 GiB)`; `the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions`; slots `1 -> 2 POOLED, 111.0 MiB each of O(1) state`|
|
||||||
|
|working memory|5.7 GiB at `HALOGEN_MAX_TOK 8192`|
|
||||||
|
|total|`75.8 GiB in all` — ~25 GiB of the 121.2 GiB box left for co-tenants|
|
||||||
|
|n-gram|`47.68 GiB from /models/qwen38-flash-next-ngram.hgn`, `checkpoint: mapped 51.2 GB, NOT pinned (Pin::None)` — disk-backed exactly as designed|
|
||||||
|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, `HALOGEN_CACHE_INPLACE=1` (the KV rows are never copied)|
|
||||||
|
|spec decode|`MTP depth-2 selectable (drafter 1, speculates while alone)`|
|
||||||
|
|clamps|none — the requested pool is the delivered pool|
|
||||||
|
|
||||||
|
Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB
|
||||||
|
(~28 KiB/position **including** block scratch and ~115 MiB/slot of O(1) state)
|
||||||
|
while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the
|
||||||
|
RAM ledger and the formula when comparing pool sizes.
|
||||||
|
|
||||||
|
Live probes on the deployed endpoint: `27*59` → `1593` with a warm thinking
|
||||||
|
channel, tool call `{"city": "Seattle"}`, 512 px solid-red PNG → **Red** (320
|
||||||
|
prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s
|
||||||
|
on the FIRST request after boot, i.e. the warmup artifact; the trial's settled
|
||||||
|
figure is 50.0. OpenWebUI's `frank-dragoon` row description re-keyed to the
|
||||||
|
native payload (backup `webui.db.bak-20261017b`; no container restart forced —
|
||||||
|
forcing it logs every family session out for one cosmetic string).
|
||||||
|
|
||||||
|
## Ceiling lowered to 200,704 — the second production boot (`7c4dbeb`)
|
||||||
|
|
||||||
|
The same evening, after the ledger above was read: `HALOGEN_CTX` and
|
||||||
|
`HALOGEN_KV_POOL_POSITIONS` 262144 → **200704** (49 × 4096, the per-slot rung
|
||||||
|
the llama.cpp endpoint frank-zealot already runs). The checkpoint's native
|
||||||
|
window is 262,144 and the engine serves anything below it, so this is a RAM
|
||||||
|
decision, not a capability limit. Nothing in the fleet asks for the bigger
|
||||||
|
window; the harness moved WITH the ceiling (400-zone law), `models.yml`
|
||||||
|
`contextWindow: 155648`.
|
||||||
|
|
||||||
|
|Line|Figure|
|
||||||
|
|---|---|
|
||||||
|
|marker|`revision=7c4dbeb…`, `status=deployed`; unit `active`, `/health` 200|
|
||||||
|
|KV pool|`reserving the KV pool, 200704 positions (about 5.5 GiB)`|
|
||||||
|
|shape|`2 slots over ONE 200704-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), max_tok 8192 … MTP depth-2 selectable (drafter 1…)`|
|
||||||
|
|working|5.7 GiB (unchanged — it scales with `HALOGEN_MAX_TOK`, not the window)|
|
||||||
|
|total|`62.9 GiB of weights locked … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all`|
|
||||||
|
|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, **111 MiB each at ctx 200704**|
|
||||||
|
|clamps|none — requested pool == delivered pool|
|
||||||
|
|
||||||
|
The refusal contract at the new ceiling, asked for explicitly (the 1.29 MB
|
||||||
|
wikitext corpus through `/v1/chat/completions`):
|
||||||
|
|
||||||
|
```
|
||||||
|
HTTP 400 {"error":{"message":"the prompt does not fit: it is 297109 tokens and
|
||||||
|
the context is 200704, 96405 tokens over. Shorten the prompt, or raise the
|
||||||
|
context (HALOGEN_CTX, and HALOGEN_ROPE_YARN for more than the native 262144).",
|
||||||
|
"type":"invalid_request_error"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
Refused in 0.86 s — the preflight rejects before any prefill, so probing the
|
||||||
|
ceiling costs the endpoint nothing. OpenWebUI's `frank-dragoon` row description
|
||||||
|
re-keyed to "ONE 200,704-position KV pool at ctx 200,704" (still no container
|
||||||
|
restart; `webui.db.bak-20261017b` holds the pre-cutover row).
|
||||||
|
|
||||||
|
**What the cut cost:** with pool == ctx one maximum-length request fills the
|
||||||
|
pool, so two 200k conversations never coexist (two of ≤100 k do), and the
|
||||||
|
harness window is now 155,648 — the 180 k compaction target of the 262,144 era
|
||||||
|
is unreachable because the window itself is smaller. **What it bought:** 1.7
|
||||||
|
GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
|
||||||
|
2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
|
||||||
@@ -19,7 +19,7 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
|||||||
digest serving the NATIVE v2 conversion of the abliterated
|
digest serving the NATIVE v2 conversion of the abliterated
|
||||||
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||||
262,144-position KV pool, vision ON)** and
|
200,704-position KV pool at ctx 200704, vision ON)** and
|
||||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||||
@@ -54,7 +54,7 @@ gfx1151 iGPU 114 GiB GTT]
|
|||||||
E[flash-server container gfx1151, digest-pinned image
|
E[flash-server container gfx1151, digest-pinned image
|
||||||
frank-dragoon, port 7770]
|
frank-dragoon, port 7770]
|
||||||
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||||
ONE 262144-position KV pool, 2 slots, MTP always-on, image input]
|
ONE 200704-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
|
||||||
G[llama.cpp PrismML fork CUDA build
|
G[llama.cpp PrismML fork CUDA build
|
||||||
frank-zealot, port 7771]
|
frank-zealot, port 7771]
|
||||||
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
||||||
@@ -87,9 +87,10 @@ frank-zealot, port 7771]
|
|||||||
**hardlinked** beside it — the card publishes only the main file, and the
|
**hardlinked** beside it — the card publishes only the main file, and the
|
||||||
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
|
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
|
||||||
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
|
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
|
||||||
Card-recommended `HALOGEN_KV_POOL_POSITIONS=262144`: **one pool behind two
|
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=200704`: **one pool
|
||||||
slots** — 7.2 GiB where the old shape spent 14.6, the per-request ceiling
|
behind two slots** (the card ships 262,144/262,144 = 7.2 GiB; Frank took the
|
||||||
unchanged at 262,144, and the freed RAM becomes page cache for the n-gram
|
RAM the same evening — 5.5 GiB reserved, one maximum-length request filling
|
||||||
|
the pool), and the freed RAM becomes page cache for the n-gram
|
||||||
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||||
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||||
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
|
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
|
||||||
@@ -98,7 +99,7 @@ frank-zealot, port 7771]
|
|||||||
stay on disk
|
stay on disk
|
||||||
- [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17
|
- [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17
|
||||||
(rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned
|
(rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned
|
||||||
v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots × 200,192 ctx over
|
v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots at ctx 262,144 over
|
||||||
a 524,288-position pool; owns the 400-zone law (the harness chars/4
|
a 524,288-position pool; owns the 400-zone law (the harness chars/4
|
||||||
under-count against the engine ceiling), the MTP-always-on law, and the
|
under-count against the engine ceiling), the MTP-always-on law, and the
|
||||||
sampler / prompt-cache / vision contract the native era carries verbatim
|
sampler / prompt-cache / vision contract the native era carries verbatim
|
||||||
|
|||||||
@@ -313,9 +313,10 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook
|
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook
|
||||||
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
|
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
|
||||||
abliterated** conversion of orcarouter's pinned BF16
|
abliterated** conversion of orcarouter's pinned BF16
|
||||||
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) with the card's
|
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx/pool **200,704**
|
||||||
memory-saving `HALOGEN_KV_POOL_POSITIONS=262144` (7.2 GiB, one pool behind
|
(`HALOGEN_KV_POOL_POSITIONS=200704`, ~5.4 GiB — one pool behind both slots,
|
||||||
both slots) and the stock peonist n-gram/vision companions **hardlinked**
|
lowered the same evening from the card's 262,144) and the stock peonist
|
||||||
|
n-gram/vision companions **hardlinked**
|
||||||
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
|
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
|
||||||
always-on, vision ON, no server-side sampler change
|
always-on, vision ON, no server-side sampler change
|
||||||
- [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17
|
- [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17
|
||||||
|
|||||||
@@ -1896,3 +1896,114 @@ every restart since the cutover.
|
|||||||
hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook
|
hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook
|
||||||
header corrected (was still "LIVE", never updated at retirement) and its
|
header corrected (was still "LIVE", never updated at retirement) and its
|
||||||
Rollback table rebuilt.
|
Rollback table rebuilt.
|
||||||
|
|
||||||
|
## 2026-10-17 — frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool
|
||||||
|
|
||||||
|
- Ask: move frank-dragoon to `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
|
||||||
|
with the card's recommended settings, save memory, keep the n-gram table
|
||||||
|
disk-backed. Census first: the endpoint carries only the `slow` +
|
||||||
|
`security-reviewer` roles (the interactive workhorse is frank-archon), the
|
||||||
|
table was ALREADY disk-backed (`mapped 51.2 GB, NOT pinned (Pin::None)`), and
|
||||||
|
`HALOGEN_FLASH_PIN_TRUNK=0` is rejected by the v2 loader — so the only real
|
||||||
|
lever was the KV pool: 524,288 → the card's 262,144.
|
||||||
|
- **Filename trap caught before the fetch:** this repo name carried two
|
||||||
|
different artifacts. The writer-transplant era's file measured
|
||||||
|
66,687,678,464 B; the native 2026-10-11 conversion is 66,687,678,432 B —
|
||||||
|
32 bytes apart. Byte-exact gate + LFS sha256 `b5bd925f…` in
|
||||||
|
`scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the peonist
|
||||||
|
n-gram/vision/tokenizer trio (the release publishes only the main file; a
|
||||||
|
hardlink is a regular file, so the no-symlink serve gate passes without a
|
||||||
|
47.7 GiB copy).
|
||||||
|
- **Companion risk adjudicated by the engine, not by argument.** The card does
|
||||||
|
NOT establish the stock peonist companions as matched to a native conversion
|
||||||
|
and publishes no source-matched pair, so the A/B ran the checkpoint tools on a
|
||||||
|
stopped service under one knob set (`scripts/census-frank-dragoon.sh`):
|
||||||
|
`verify` OK, **NIAH 18/18 vs 17/18**, PPL 3.968 vs 3.877 (+2.3 % — the
|
||||||
|
finetune's edge is gone), KL vs the swift15 logits mean 0.155 / median 0.057
|
||||||
|
with top-1 85.1 % = two sound models disagreeing on content, not a corrupt
|
||||||
|
forward pass. Behavior battery (`scripts/battery-frank-dragoon.py`) on both
|
||||||
|
boots: refusals 0/5 both, tools/vision identical, needles hit at both depths,
|
||||||
|
decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 (+12.6 %), thinking
|
||||||
|
tokens 1,476 vs 784 (the finetune compressed reasoning ×1.9).
|
||||||
|
- Boot window run through a transient `FRANK_DRAGOON_ENV_FILE` drop-in — never
|
||||||
|
a scratch port (one iGPU, one server) — with the stop+mask law and a
|
||||||
|
trap-guarded restore (`scripts/trial-frank-dragoon.sh`). Masking fails on a
|
||||||
|
relinked unit file, as documented; stop alone was enough.
|
||||||
|
- Deployed at 22:20 (`fcd7630`, `--service frank-dragoon`): healthy in ~12 s,
|
||||||
|
e2e 33/33. Production ledger: 62.9 GiB weights locked (0.8 vision) + 7.2 GiB
|
||||||
|
KV (ONE 262,144-position pool, 6,656 MiB, two slots) + 5.7 GiB working =
|
||||||
|
**75.8 GiB in all**, no clamp line, n-gram 47.68 GiB read from disk, prompt
|
||||||
|
cache ON with 12 entries, MTP depth-2. The trial drop-in was moved to
|
||||||
|
`~/.AGENTTRASH/20261017-v2abl-trial/` before the deploy so the repo env
|
||||||
|
governs (`DropInPaths=` empty). Live probes: `27*59` → 1593, tool call
|
||||||
|
`{"city": "Seattle"}`, 512 px red PNG → Red (320 prompt tokens).
|
||||||
|
- Files moved: env (payload dir/checkpoint/bytes/sha256/pool + the measured
|
||||||
|
A/B in the header), serve script + deployer (paths, fetch hints), unit
|
||||||
|
`Description`, seam (`frank_dragoon_unit.bats`: pool law relaxed from
|
||||||
|
`pool == slots × ctx` to `ctx <= pool <= slots × ctx`, fetch-script name),
|
||||||
|
new runbook `frank-dragoon-flash-server-v2-native`, swift15 page re-banned
|
||||||
|
RETIRED, both indexes + AGENTS.md re-keyed, homepage card, `models.yml`
|
||||||
|
comment (numbers unchanged: window 217,088 / cap 45,056 — the per-REQUEST
|
||||||
|
ceiling did not move). OpenWebUI `frank-dragoon` row description re-keyed
|
||||||
|
(backup `webui.db.bak-20261017b`, no restart forced).
|
||||||
|
- Two stale claims fixed on the way: the retired-era rows said `200,192` ctx
|
||||||
|
(that ceiling was raised to 262,144 on 2026-10-07, inside the same era), and
|
||||||
|
the root index still called the halogen-27B page ACTIVE.
|
||||||
|
- Cost of the swap, stated plainly: two MAXIMUM-length conversations no longer
|
||||||
|
share the endpoint (one 262,144-position pool behind both slots), PPL +2.3 %,
|
||||||
|
thinking ×1.9. Bought: 7.4 GiB of RAM back as page cache for the table long
|
||||||
|
prompts read, +12.6 % cold prefill, retrieval 18/18, the base checkpoint
|
||||||
|
itself instead of the finetune.
|
||||||
|
|
||||||
|
## 2026-10-17 — frank-dragoon: ceiling and pool cut to 200,704; crawl reset fix
|
||||||
|
|
||||||
|
- Ask: lower frank-dragoon's context "to 196k". Settled on **200,704** for BOTH
|
||||||
|
`HALOGEN_CTX` and `HALOGEN_KV_POOL_POSITIONS` (49 × 4096 — the per-slot rung
|
||||||
|
frank-zealot runs; the card recommends pool == ctx, and that equality is where
|
||||||
|
the memory is). Nothing in the fleet needs the bigger window, and the v2
|
||||||
|
checkpoint serves anything up to its native 262,144.
|
||||||
|
- Boot ledger (`7c4dbeb`, `status=deployed`): `reserving the KV pool, 200704
|
||||||
|
positions (about 5.5 GiB)`, `62.9 GiB of weights … 5.5 of KV pool, 5.7 of
|
||||||
|
working memory, 74.1 GiB in all`, prompt cache entries 111 MiB at the new ctx,
|
||||||
|
no clamp line. My arithmetic predicted 5.4/74 — the ledger is quoted instead.
|
||||||
|
- Harness moved WITH the ceiling (400-zone law): `models.yml`
|
||||||
|
`contextWindow: 155648` = ceiling − 45,056; compaction now fires at 85 % =
|
||||||
|
132,301 est ≈ 136k measured, and the 180k compaction target of the 262,144 era
|
||||||
|
is unreachable because the window itself is smaller. `maxTokens` 45,056 and the
|
||||||
|
coupled cap/timeout 65536/7200 untouched. Worst-case harness request =
|
||||||
|
1.03 × est + min(45056, 151552 − est) ≤ 156.2k tokens vs the 200,704 ceiling.
|
||||||
|
- Probes on the live endpoint: over the ceiling → `400 … it is 297109 tokens and
|
||||||
|
the context is 200704, 96405 tokens over` in 0.86 s (preflight rejects before
|
||||||
|
any prefill, so probing costs the endpoint nothing); harness-shaped admission →
|
||||||
|
122,679-token prompt with `max_tokens: 45056` (reservation 167,735) accepted,
|
||||||
|
81.1 s wall (≈1,513 t/s cold prefill), `27*59` answered **1593** from inside a
|
||||||
|
122k-token haystack, `finish_reason: stop`.
|
||||||
|
- Seam re-keyed (`frank_dragoon_unit.bats`): the fit-law test now proves the
|
||||||
|
invariant at the new pair — `ctx ≤ pool ≤ slots × ctx` and
|
||||||
|
`harness window ≤ ctx − 45056` — so a future raise of one without the other
|
||||||
|
fails the gate. bats 371/371.
|
||||||
|
- Deploy verdict: the endpoint dispatched healthy (`status=deployed`, `/health`
|
||||||
|
200), but the `[3/3]` Playwright gate failed on `12-crawl` and the deployer
|
||||||
|
correctly said **DEPLOY INCOMPLETE**. Root cause was NOT the endpoint — it was
|
||||||
|
Plex Web, signed out in the `qa-e2e` profile, running its own sign-in flow,
|
||||||
|
which broke the spec two different ways:
|
||||||
|
1. it redirects the top frame, so Chromium cancelled the crawl's in-loop
|
||||||
|
`page.goto("/")` with `net::ERR_ABORTED` — and that reset sat outside the
|
||||||
|
`try`, so one host aborted the whole crawl (once as a 420 s timeout, once at
|
||||||
|
5.4 s); fixed with `backHome()`, which re-issues on that one error class.
|
||||||
|
2. the same hand-off leaves the browser on `app.plex.tv` (Plex's hosted surface
|
||||||
|
for a session-less client), so the `hostname == target` law could never
|
||||||
|
settle — this is what the first full-suite re-run surfaced as `1 flaky`;
|
||||||
|
fixed with an explicit `HANDOFFS` landing list.
|
||||||
|
Plex itself was healthy throughout (container up 2 days; 32400 answers HTTPS
|
||||||
|
401 / plain-HTTP empty reply, which is what Plex Web authenticates against),
|
||||||
|
and the guard contract is still asserted for every vhost: challenge cleared, no
|
||||||
|
authentik `uidField`, status < 500. Both symptoms recorded as gotcha 6.
|
||||||
|
- Gate after both fixes: **33 passed (1.2 m)**, crawl green on the first attempt
|
||||||
|
(6.5 s) — the earlier `DEPLOY INCOMPLETE` is cleared, and the deploy marker
|
||||||
|
stays at `7c4dbeb` because every later commit is docs/tests only, which the
|
||||||
|
planner treats as a no-op for the service.
|
||||||
|
- Cost of the cut: two maximum-length conversations never coexist (one
|
||||||
|
200,704-position pool behind both slots; two of ≤100k still do). Bought: 1.7
|
||||||
|
GiB back as page cache for the 47.68 GiB n-gram table — 9.1 GiB against the
|
||||||
|
2026-10-06 shape's 524,288 pool — and a smaller prompt-cache entry.
|
||||||
@@ -8,18 +8,53 @@ import { expect, test, type Page } from "@playwright/test";
|
|||||||
|
|
||||||
const CHALLENGE = "Making sure you're not a bot";
|
const CHALLENGE = "Making sure you're not a bot";
|
||||||
|
|
||||||
async function settleThroughGuard(page: Page, targetHost: string): Promise<void> {
|
// Vendor hand-offs. An app may legitimately carry a session-less browser to its
|
||||||
|
// OWN hosted surface: Plex Web (plex.franksdank.cloud, served by our stack)
|
||||||
|
// bounces a signed-out visitor to app.plex.tv after it has already answered.
|
||||||
|
// That is the app working, not a guard-chain failure — the fleet contract this
|
||||||
|
// spec owns is still judged: the Anubis challenge cleared, no authentik
|
||||||
|
// uidField prompt appeared, and the app answered below 500. Any host NOT listed
|
||||||
|
// here still has to land on its own vhost.
|
||||||
|
const HANDOFFS: Record<string, string[]> = {
|
||||||
|
"plex.franksdank.cloud": ["app.plex.tv", "accounts.plex.tv", "clients.plex.tv"],
|
||||||
|
};
|
||||||
|
|
||||||
|
async function settleThroughGuard(
|
||||||
|
page: Page,
|
||||||
|
targetHost: string,
|
||||||
|
foreignHosts: string[] = [],
|
||||||
|
): Promise<void> {
|
||||||
// Each guarded vhost runs its own Anubis proof-of-work on first visit;
|
// Each guarded vhost runs its own Anubis proof-of-work on first visit;
|
||||||
// wait for the challenge to clear before judging the destination.
|
// wait for the challenge to clear before judging the destination.
|
||||||
await expect
|
await expect
|
||||||
.poll(async () => !(await page.locator("body").innerText()).includes(CHALLENGE), { timeout: 90_000 })
|
.poll(async () => !(await page.locator("body").innerText()).includes(CHALLENGE), { timeout: 90_000 })
|
||||||
.toBe(true);
|
.toBe(true);
|
||||||
|
const landing = [targetHost, ...foreignHosts];
|
||||||
await expect
|
await expect
|
||||||
.poll(() => new URL(page.url()).hostname, { timeout: 60_000 })
|
.poll(() => landing.includes(new URL(page.url()).hostname), { timeout: 60_000 })
|
||||||
.toBe(targetHost);
|
.toBe(true);
|
||||||
expect(await page.locator('input[name="uidField"]').count()).toBe(0);
|
expect(await page.locator('input[name="uidField"]').count()).toBe(0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function backHome(page: Page): Promise<void> {
|
||||||
|
// The homepage is the loop's only anchor, and some apps keep issuing their
|
||||||
|
// OWN navigation after load — Plex Web's "would like to sign in to your
|
||||||
|
// Plex account" iframe redirects the top frame, which cancels our goto with
|
||||||
|
// net::ERR_ABORTED. That abort is the app's behaviour, not a guard-chain
|
||||||
|
// failure (the app itself was already judged above), so tolerate it and
|
||||||
|
// re-issue rather than letting one host abort the whole crawl.
|
||||||
|
for (let attempt = 0; attempt < 3; attempt++) {
|
||||||
|
try {
|
||||||
|
await page.goto("/", { waitUntil: "domcontentloaded" });
|
||||||
|
return;
|
||||||
|
} catch (err) {
|
||||||
|
if (!String(err).includes("ERR_ABORTED")) throw err;
|
||||||
|
await page.waitForTimeout(1_000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await page.goto("/");
|
||||||
|
}
|
||||||
|
|
||||||
test("every user open link reaches its app", async ({ page }) => {
|
test("every user open link reaches its app", async ({ page }) => {
|
||||||
test.setTimeout(420_000);
|
test.setTimeout(420_000);
|
||||||
// Expected link count comes from the live registry, mirroring the
|
// Expected link count comes from the live registry, mirroring the
|
||||||
@@ -39,7 +74,7 @@ test("every user open link reaches its app", async ({ page }) => {
|
|||||||
const target = new URL(link.href);
|
const target = new URL(link.href);
|
||||||
try {
|
try {
|
||||||
await page.goto(link.href);
|
await page.goto(link.href);
|
||||||
await settleThroughGuard(page, target.hostname);
|
await settleThroughGuard(page, target.hostname, HANDOFFS[target.hostname] ?? []);
|
||||||
const resp = await page.request.get(page.url());
|
const resp = await page.request.get(page.url());
|
||||||
if (resp.status() >= 500) {
|
if (resp.status() >= 500) {
|
||||||
failures.push(`${link.label}: ${page.url()} answered ${resp.status()}`);
|
failures.push(`${link.label}: ${page.url()} answered ${resp.status()}`);
|
||||||
@@ -47,7 +82,7 @@ test("every user open link reaches its app", async ({ page }) => {
|
|||||||
} catch (err) {
|
} catch (err) {
|
||||||
failures.push(`${link.label}: ${err instanceof Error ? err.message : String(err)}`);
|
failures.push(`${link.label}: ${err instanceof Error ? err.message : String(err)}`);
|
||||||
}
|
}
|
||||||
await page.goto("/");
|
await backHome(page);
|
||||||
await expect(page.locator("#grid a.open")).toHaveCount(expectedUserLinks);
|
await expect(page.locator("#grid a.open")).toHaveCount(expectedUserLinks);
|
||||||
}
|
}
|
||||||
expect(failures, failures.join("\n")).toEqual([]);
|
expect(failures, failures.join("\n")).toEqual([]);
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
|
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
|
||||||
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
|
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
|
||||||
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE
|
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE
|
||||||
# 262,144-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
|
# 200,704-position KV pool with the MTP head always on (HALOGEN_SPEC_ADAPT=0)
|
||||||
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
|
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
|
||||||
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
|
# shape, which itself cut over the halogen 27B-dense shape on 2026-10-06.
|
||||||
#
|
#
|
||||||
@@ -81,8 +81,8 @@ good_env() {
|
|||||||
"FRANK_DRAGOON_VISION_BYTES=0"
|
"FRANK_DRAGOON_VISION_BYTES=0"
|
||||||
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
|
"FRANK_DRAGOON_TOKENIZER_FILES=tok.json"
|
||||||
"HALOGEN_MODEL_ID=frank-dragoon"
|
"HALOGEN_MODEL_ID=frank-dragoon"
|
||||||
"HALOGEN_CTX=262144"
|
"HALOGEN_CTX=200704"
|
||||||
"HALOGEN_KV_POOL_POSITIONS=262144"
|
"HALOGEN_KV_POOL_POSITIONS=200704"
|
||||||
"HALOGEN_KV_SLOTS=2"
|
"HALOGEN_KV_SLOTS=2"
|
||||||
"HALOGEN_SPEC_ADAPT=0"
|
"HALOGEN_SPEC_ADAPT=0"
|
||||||
"HALOGEN_MAX_TOK=8192"
|
"HALOGEN_MAX_TOK=8192"
|
||||||
@@ -153,13 +153,13 @@ good_env() {
|
|||||||
pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")"
|
pool="$(sed -nE 's/^HALOGEN_KV_POOL_POSITIONS=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||||
mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")"
|
mt="$(sed -nE 's/^HALOGEN_MAX_TOK=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||||
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
|
reserve="$(sed -nE 's/^HALOGEN_HOST_RESERVE_GIB=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||||
# The pool is ONE budget behind the slots (boot ledger: "2 slots over ONE
|
# The pool is ONE budget behind the slots (boot ledger form: "2 slots over
|
||||||
# 262144-position KV pool, each request up to 262144"), NOT slots x ctx:
|
# ONE <pool>-position KV pool, each request up to HALOGEN_CTX"), NOT
|
||||||
# the card ships it at one window and that IS the memory saving of this
|
# slots x ctx — one window behind both slots IS the memory saving. What
|
||||||
# payload. What still binds is that the pool back ONE full-length
|
# still binds: the pool must back ONE full-length request (or the
|
||||||
# request, or the advertised per-request ceiling would be unreachable —
|
# advertised per-request ceiling would be unreachable, hence pool >= ctx),
|
||||||
# and that it stay inside unified memory, which the ledger sum below
|
# it must not pretend to be slots x ctx (pool <= slots * ctx), and the
|
||||||
# checks.
|
# ledger sum below must stay inside unified memory.
|
||||||
[ "$pool" -ge "$ctx" ]
|
[ "$pool" -ge "$ctx" ]
|
||||||
[ "$pool" -le $(( slots * ctx )) ]
|
[ "$pool" -le $(( slots * ctx )) ]
|
||||||
kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
|
kv_gib=$(( pool * 28 * 1024 / 1073741824 ))
|
||||||
@@ -273,8 +273,8 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
[[ "$output" == *"--name frank-dragoon-halogen"* ]]
|
[[ "$output" == *"--name frank-dragoon-halogen"* ]]
|
||||||
[[ "$output" == *"--publish 7770:8731"* ]]
|
[[ "$output" == *"--publish 7770:8731"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
|
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-dragoon"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]]
|
[[ "$output" == *"-e HALOGEN_CTX=200704"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=262144"* ]]
|
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=200704"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
|
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
|
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
|
||||||
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
|
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
|
||||||
@@ -291,23 +291,24 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
||||||
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
# HALOGEN_CTX is the hard per-request ceiling (a longer prompt is a 400
|
||||||
# (a longer prompt is a 400 naming the limit, never a silent
|
# naming the limit, never a silent truncation). The law is INVARIANT to
|
||||||
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
# the value: the harness window must sit ~45k UNDER it, because the
|
||||||
# − 4096), compacts at 85 % of its declared 217088 window, and its
|
# harness clamps max_tokens = min(45056, window − est − 4096), compacts
|
||||||
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
|
# at 85 % of its declared window, and its chars/4 est UNDER-counts the
|
||||||
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with
|
# rendered prompt by ~3 % on code/agent traffic (measured 4,450 tokens at
|
||||||
# window == ceiling, every request over ~146k est was a guaranteed
|
# est 152,938 on 2026-10-07 — with window == ceiling, every request over
|
||||||
# 400 before compaction could fire at 85 %). The 45k gap between the
|
# ~65 % of the window was a guaranteed 400 before compaction could fire).
|
||||||
# harness window and the engine ceiling must absorb that error at
|
# The shipped pair is ctx 200704 / window 155648 (lowered from
|
||||||
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
|
# 262144/217088 on 2026-10-17); the gap must absorb the error at BOTH
|
||||||
# full budget at the compaction threshold: 1.03*184524 + 45056
|
# monotone maxima of est + budget (est ≤ window − 4096):
|
||||||
# clamped budget at the clamp's zero point: 1.03*212992 + 0
|
# full budget at the compaction threshold: 1.03*132300 + 45056
|
||||||
|
# clamped budget at the clamp's zero point: 1.03*151552 + 0
|
||||||
# and the compaction request that rides on frank-zealot (its
|
# and the compaction request that rides on frank-zealot (its
|
||||||
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
|
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
|
||||||
grep -qE '^HALOGEN_CTX=262144$' "$ENV_FILE"
|
grep -qE '^HALOGEN_CTX=200704$' "$ENV_FILE"
|
||||||
ctx=262144
|
ctx=200704
|
||||||
hw=217088; mt=45056
|
hw=155648; mt=45056
|
||||||
thr=$(( hw * 85 / 100 ))
|
thr=$(( hw * 85 / 100 ))
|
||||||
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
|
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
|
||||||
zero=$(( hw - 4096 ))
|
zero=$(( hw - 4096 ))
|
||||||
|
|||||||
Reference in new issue
Block a user