docs(frank-dragoon): native-era runbook; retire the swift15 page
New LLM/frank-dragoon-flash-server-v2-native.md carries the shape with the three deliberate deviations from the card (served id, SPEC_ADAPT=0, engine pin 0.16.2), the memory ledger with the PIN_TRUNK rejection, the full measured A/B table, and the companion law (hardlinked stock peonist files, engine-tool certified). The swift15 page is re-banned RETIRED 2026-10-06 → 10-17 with its rollback terms; both indexes and AGENTS.md (topology table, service section, runbook index) are re-keyed, and the root index also stops describing the halogen-27B page as ACTIVE.
This commit is contained in:
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=262144` pool** (the card's recommendation: the ledger's 7.2 GiB where the old 524,288 pool measured 14.6 — per-request ceiling unchanged at 262,144, two maximum-length conversations no longer coexist, and the freed RAM is page cache for the n-gram table), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
||||||
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
@@ -334,7 +334,7 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
|||||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||||
|
|
||||||
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
||||||
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
||||||
@@ -346,19 +346,23 @@ run` FOREGROUND — the container's exit code is the service's exit so
|
|||||||
7770 from the Strix Halo iGPU. Engine stdout goes to
|
7770 from the Strix Halo iGPU. Engine stdout goes to
|
||||||
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
|
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
|
||||||
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
||||||
Cutover 2026-10-06 from the halogen 27B-dense era
|
Cutover 2026-10-17 from the Swift1.5-finetune shape
|
||||||
([[frank-dragoon-halogen27b-dflash2]], retired the same window).
|
([[frank-dragoon-flash-server-v2-swift15]], retired the same window; it
|
||||||
|
itself cut over the halogen 27B-dense era on 2026-10-06).
|
||||||
|
|
||||||
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
Serving `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`:
|
||||||
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
`qwen38-flash-next-v2-abliterated.hgn` (66,687,678,432 B ≈ 62.1 GiB, sha256
|
||||||
GiB, sha256 `9c880b8b…` — a compacted r2 of the card's 89,775,868,032 B
|
`b5bd925f…` — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned
|
||||||
append-only original; the abliteration workflow appends and the repo's
|
BF16 checkpoint, abliterated; NOT the writer transplant that carried this
|
||||||
tooling drops dead padding, live tensors byte-identical) + the **REQUIRED**
|
filename, which measured 66,687,678,464 B — 32 bytes apart, same filename
|
||||||
n-gram lookup table `qwen38-flash-next-ngram.hgn` (51,200,246,144 B,
|
family, different model) + the **REQUIRED** n-gram lookup table
|
||||||
peonist @5cc17cea) + the vision tower `qwen38-flash-next-vision.hgn`
|
`qwen38-flash-next-ngram.hgn` (51,200,246,144 B, peonist @5cc17cea) + the
|
||||||
(897,916,416 B, @e053f488) + the flat `tokenizer/` directory (6 files,
|
vision tower `qwen38-flash-next-vision.hgn` (897,916,416 B, @e053f488) + the
|
||||||
fetched fresh from peonist). Cutover 2026-10-06 from the halogen
|
flat `tokenizer/` directory (6 files) — the three companions **hardlinked**
|
||||||
27B-dense shape, chosen by Frank for the ABLITERATED base.
|
out of the swift15 payload dir, because the release publishes only the main
|
||||||
|
file and no source-matched pair exists. Cutover 2026-10-17 from the Swift1.5
|
||||||
|
finetune, chosen by Frank for the base checkpoint itself plus the card's
|
||||||
|
memory-saving pool.
|
||||||
|
|
||||||
**The format is the lock.** The v2 `.hgn` loads ONLY in the
|
**The format is the lock.** The v2 `.hgn` loads ONLY in the
|
||||||
flash-server line — the 27B-era halogen 0.1.4 container refuses it, so the
|
flash-server line — the 27B-era halogen 0.1.4 container refuses it, so the
|
||||||
@@ -389,8 +393,13 @@ the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
|
|||||||
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
||||||
law does NOT carry over; batching and speculation coexist (the card's
|
law does NOT carry over; batching and speculation coexist (the card's
|
||||||
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
||||||
`HALOGEN_KV_POOL_POSITIONS=524288` (explicit 2 × ctx; ~28 KiB/position ⇒
|
`HALOGEN_KV_POOL_POSITIONS=262144` (the card's recommendation and the memory
|
||||||
14.0 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
saving of the 2026-10-17 swap; ~28 KiB/position ⇒ 7.2 GiB) is **ONE pool
|
||||||
|
behind both slots** — ledger `2 slots over ONE 262144-position KV pool, each
|
||||||
|
request up to 262144` — so the per-request ceiling is untouched while two
|
||||||
|
MAXIMUM-length conversations no longer coexist (the old 524,288 = 2 × ctx
|
||||||
|
spent the measured 14.6 GiB and allowed it).
|
||||||
|
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||||
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
|
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
|
||||||
@@ -407,27 +416,34 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
|||||||
"video".
|
"video".
|
||||||
|
|
||||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
||||||
weights ~62 + KV 14.0 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
weights ~62.9 (trunk 62.11 + vision 0.84) + KV 7.2 + arena ~8
|
||||||
~2 ≈ 86 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~15 GiB
|
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ 78 GiB engine-side +
|
||||||
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
|
`HALOGEN_HOST_RESERVE_GIB=20` → ~25 GiB margin over baseline co-tenancy.
|
||||||
page cache beside the checkpoint — the card's quickstart arithmetic does
|
The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint
|
||||||
not bill it as resident, but READ the boot ledger and re-size if the
|
(ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why the pool
|
||||||
engine prints a clamp.
|
cut is a double saving: **RAM the pool takes is RAM that table loses**.
|
||||||
|
`HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the v2 loader, so disk-backing
|
||||||
|
the trunk is not a lever on this format. READ the boot ledger and re-size
|
||||||
|
if the engine prints a clamp.
|
||||||
|
|
||||||
Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
|
Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
|
||||||
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
|
||||||
sha256 + `--verify-only`, no token on the command line; resume goes
|
sha256 + `--verify-only`, no token on the command line; resume goes against
|
||||||
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
the RESOLVED CDN URL because curl drops `Range` across HF's 302) — the main
|
||||||
The serve gates mirror the deployer EXACTLY (byte-exact checkpoint +
|
file is fetched, the three companions are HARDLINKED from the verified
|
||||||
n-gram + vision tower, flat tokenizer with all six files as regular
|
swift15 dir (a hardlink is a regular file, so the no-symlink serve gate
|
||||||
files) — the 2026-10-01 crash-loop lesson.
|
passes without a 47.7 GiB copy). The serve gates mirror the deployer EXACTLY
|
||||||
|
(byte-exact checkpoint + n-gram + vision tower, flat tokenizer with all six
|
||||||
|
files as regular files) — the 2026-10-01 crash-loop lesson.
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
|
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md`.
|
||||||
Rollback: revert the swap commit and redeploy — the 27B payload
|
Rollback: revert the swap commit and redeploy — the Swift1.5 payload
|
||||||
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
(`/home/dank/models/halogen-flashnext-v2/`, re-verified by
|
||||||
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
`scripts/download-frank-dragoon-flashnext.sh`), the 27B payload
|
||||||
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
(`/home/dank/models/halogen-qwen3.8-27b/`) and the halogen 0.1.4 image
|
||||||
|
digest all remain on disk; the ROCmFPX door behind that
|
||||||
|
(`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||||
|
|
||||||
### frank-zealot — llama.cpp (PrismML fork) on the RTX 3090 (Dankclaw, ACTIVE)
|
### frank-zealot — llama.cpp (PrismML fork) on the RTX 3090 (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
@@ -1140,7 +1156,7 @@ Active:
|
|||||||
|
|
||||||
| Service | Runbook |
|
| Service | Runbook |
|
||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 262,144 pool, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
|
||||||
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
||||||
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
@@ -1153,6 +1169,7 @@ Retired (rollback targets and history):
|
|||||||
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
|
| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, 2 slots × 200,192 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||||
|
|||||||
@@ -0,0 +1,187 @@
|
|||||||
|
# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, Strix Halo iGPU)
|
||||||
|
|
||||||
|
**ACTIVE since 2026-10-17.** Rollback = revert the swap commit and redeploy
|
||||||
|
(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
|
||||||
|
`scripts/frank-dragoon.service`); the Swift1.5 payload stays on disk at
|
||||||
|
`/home/dank/models/halogen-flashnext-v2/` (its fetch script
|
||||||
|
`scripts/download-frank-dragoon-flashnext.sh` stays the door back), and the
|
||||||
|
27B-era payload beside that at `/home/dank/models/halogen-qwen3.8-27b/`.
|
||||||
|
Era write-up: [[frank-dragoon-flash-server-v2-swift15]].
|
||||||
|
|
||||||
|
## Why this payload
|
||||||
|
|
||||||
|
Frank's ask: serve
|
||||||
|
`https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
|
||||||
|
with the card's recommended settings, and **save memory**.
|
||||||
|
|
||||||
|
This is the **native 2026-10-11 v2 conversion** of orcarouter's pinned BF16
|
||||||
|
Qwen3.8-Flash-Next, abliterated — the base itself, converted straight into
|
||||||
|
the v2 `.hgn` format. It is **not** the writer transplant that carried this
|
||||||
|
filename before: the artifact measures **66,687,678,432 B** where the
|
||||||
|
Swift1.5 r2 measured 66,687,678,464 B — 32 bytes apart, same filename
|
||||||
|
family, different model. Byte counts and the Hub LFS sha256 are the only way
|
||||||
|
to tell them apart, which is why the fetch script and both gates check them.
|
||||||
|
|
||||||
|
What the swap buys, measured (table below): retrieval at depth, decode,
|
||||||
|
prefill, and **7.4 GiB of RAM handed back**. What it costs: the Swift1.5
|
||||||
|
finetune's compressed reasoning and its small perplexity edge.
|
||||||
|
|
||||||
|
## The companion law (the reason this swap needed a referee)
|
||||||
|
|
||||||
|
The release publishes **only the main file**. The n-gram lookup table
|
||||||
|
(REQUIRED — the engine reads it beside the checkpoint), the vision tower,
|
||||||
|
and the tokenizer are peonist's, and the card does **not** establish them as
|
||||||
|
interchangeable with this native conversion. No source-matched pair is
|
||||||
|
published anywhere, so the only available companions are the stock ones.
|
||||||
|
|
||||||
|
They are **hardlinked** out of the verified Swift1.5 payload directory
|
||||||
|
(`scripts/download-frank-dragoon-v2abl.sh`): a hardlink is a regular file, so
|
||||||
|
it satisfies the serve gate's no-symlink law, and it costs no disk and no
|
||||||
|
copy next to a 62 GB payload. The engine's own checkpoint tools are the
|
||||||
|
referee, and they answered: **NIAH 18/18** (vs 17/18 for the payload those
|
||||||
|
files were published with), needle retrieval exact at both depths, the red-PNG
|
||||||
|
vision probe exact, perplexity in the normal band. A wrong PLE table does not
|
||||||
|
whisper — see Verification.
|
||||||
|
|
||||||
|
## The payload — one new file, three inherited ones
|
||||||
|
|
||||||
|
`/home/dank/models/halogen-v2-abliterated/`, fetched by
|
||||||
|
`scripts/download-frank-dragoon-v2abl.sh` (`-f` + exact byte counts + LFS
|
||||||
|
sha256 + `--verify-only`; no token on the command line):
|
||||||
|
|
||||||
|
|File|Bytes|sha256|Source|
|
||||||
|
|---|---|---|---|
|
||||||
|
|`qwen38-flash-next-v2-abliterated.hgn`|66,687,678,432|`b5bd925f…`|Quat3rnion `@main`, fetched|
|
||||||
|
|`qwen38-flash-next-ngram.hgn`|51,200,246,144|`3450acada…`|peonist `@5cc17cea`, **hardlinked**, REQUIRED|
|
||||||
|
|`qwen38-flash-next-vision.hgn`|897,916,416|`d62e0ae5…`|peonist `@e053f488`, hardlinked, vision ON|
|
||||||
|
|`tokenizer/` (6 files)| — |`tokenizer.json` LFS|peonist `@e053f488`, hardlinked|
|
||||||
|
|
||||||
|
The engine stays **halogen-flash-server 0.16.2**, digest-pinned (the card
|
||||||
|
tested 0.15.0; 0.16.2 is newer in the same line and is what the v2 format was
|
||||||
|
qualified on in this fleet):
|
||||||
|
|
||||||
|
```
|
||||||
|
ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||||
|
```
|
||||||
|
|
||||||
|
## Shape — the card's settings, and the three deliberate deviations
|
||||||
|
|
||||||
|
`HALOGEN_CTX=262144` (per-request ceiling), `HALOGEN_KV_SLOTS=2`,
|
||||||
|
**`HALOGEN_KV_POOL_POSITIONS=262144`** (the card's recommendation, and the
|
||||||
|
memory saving), `HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
|
||||||
|
`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
|
||||||
|
sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
|
||||||
|
`HALOGEN_HOST_RESERVE_GIB=20`.
|
||||||
|
|
||||||
|
Deviations from the card, each on purpose:
|
||||||
|
|
||||||
|
- `HALOGEN_MODEL_ID=frank-dragoon`, not the card's `halogen-v2-abliterated` —
|
||||||
|
the fleet naming law (service = unit = served id).
|
||||||
|
- `HALOGEN_SPEC_ADAPT=0` — the card is silent; this is a measured 0.16.2 law
|
||||||
|
(the adaptive default disables the drafter per request and halves decode).
|
||||||
|
- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
|
||||||
|
|
||||||
|
**The pool is one budget, not a partition.** Boot ledger: `listening on
|
||||||
|
127.0.0.1:8730, 2 slots over ONE 262144-position KV pool, each request up to
|
||||||
|
262144`. The per-request ceiling is unchanged, so the harness contract
|
||||||
|
(`contextWindow: 217088`) does not move — what changed is that **two
|
||||||
|
maximum-length conversations can no longer share the endpoint**. That is
|
||||||
|
acceptable here because frank-dragoon carries the `slow` (medium-thinking)
|
||||||
|
role and the `security-reviewer` subagent, not the interactive workhorse
|
||||||
|
(that is frank-archon).
|
||||||
|
|
||||||
|
**Memory ledger** (121.2 GiB unified), from the boot ledger of the live
|
||||||
|
candidate: `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
|
||||||
|
`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
|
||||||
|
7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
|
||||||
|
RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …`. The n-gram
|
||||||
|
table rides as page cache — `mapped 51.2 GB, NOT pinned (Pin::None)` — which
|
||||||
|
is exactly why cutting the pool is a *double* saving. FLAGS.md law: **RAM the
|
||||||
|
pool takes is RAM that table loses**; the 7.4 GiB the smaller pool hands back
|
||||||
|
is page cache for the 47.68 GiB table that long prompts read.
|
||||||
|
|
||||||
|
`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
|
||||||
|
stays pinned. Disk-backing the weights is not a lever on this format.
|
||||||
|
|
||||||
|
## Measured A/B — same battery, both boots (2026-10-17)
|
||||||
|
|
||||||
|
Behavior probes ran against the live service (tagged `swift15` / `v2abl`);
|
||||||
|
the engine tools ran with the service **stopped**, per the fleet benchmarking
|
||||||
|
law, under one identical knob set (`scripts/census-frank-dragoon.sh`).
|
||||||
|
|
||||||
|
|Probe|Swift1.5 r2 (out)|native v2 (in)|
|
||||||
|
|---|---|---|
|
||||||
|
|`verify` on the file|—|OK|
|
||||||
|
|NIAH 18 cases (4k/32k × 3 depths × 3 positions)|17/18 (94.4 %)|**18/18 (100 %)**|
|
||||||
|
|wikitext-2 PPL (297,053 ids, 2 × 262,144 seqs)|**3.8772**|3.9679 (+2.3 %)|
|
||||||
|
|KL of the new model against the old model's logits|—|mean 0.155 / median 0.057, top-1 85.1 %|
|
||||||
|
|refusal census (5 hostile prompts)|0/5 refusing|0/5 refusing|
|
||||||
|
|thinking tokens, mean of 6 questions (temp 1.0)|**784**|1,476 (×1.9)|
|
||||||
|
|tool call, 1-turn / 2-turn|`{"city": "Seattle"}` / answer|`{"city": "Seattle"}` / answer|
|
||||||
|
|vision (512 px solid red, `image_url` part)|Red|Red|
|
||||||
|
|needle @21,568 tokens, planted 0.85 / 0.10|hit / hit|hit / hit|
|
||||||
|
|decode, greedy, 512 tokens|46.8 t/s (spread 25.6–46.8)|50.0 t/s (steady 50.0/50.2)|
|
||||||
|
|prefill, cold, 59,024 tokens|1,424.8 t/s|**1,604.3 t/s (+12.6 %)**|
|
||||||
|
|prompt-cache ratio (same prompt, second pass)|136.7×|131.6×|
|
||||||
|
|KV pool|14.6 GiB (524,288 positions)|**7.2 GiB (262,144)**|
|
||||||
|
|
||||||
|
Read the KL honestly: it is a **model-vs-model** divergence, not the
|
||||||
|
quant-vs-source comparison the tool was built for. Two different models on
|
||||||
|
the same text *should* differ; median 0.057 with 85 % top-1 agreement is the
|
||||||
|
signature of two sound models that disagree on content, and the worst rows
|
||||||
|
are date/name completions. A n-gram table mismatched to the checkpoint
|
||||||
|
corrupts the forward pass and shows up as elevated KL **everywhere** plus
|
||||||
|
degraded retrieval — which NIAH (18/18) and the needle probes rule out.
|
||||||
|
|
||||||
|
The decode line needs care: the baseline's own spread was 25.6–46.8 t/s
|
||||||
|
because its samples were taken while the 62 GB fetch was still streaming into
|
||||||
|
page cache, and the candidate's **first** sample after the battery's long
|
||||||
|
prompts read 32.8 before settling at 50.0/50.2. Honest claim: **no decode
|
||||||
|
regression, plausibly faster**; never quote either single number.
|
||||||
|
|
||||||
|
## Verification — how to reproduce
|
||||||
|
|
||||||
|
```
|
||||||
|
# 1. payload: bytes + LFS sha256 + the hardlinked trio
|
||||||
|
bash scripts/download-frank-dragoon-v2abl.sh --verify-only
|
||||||
|
# 2. engine tools on a STOPPED service (fleet law: no GPU experiment beside
|
||||||
|
# a live endpoint; stop the unit first — mask fails on a relinked unit)
|
||||||
|
bash scripts/census-frank-dragoon.sh verify # no GPU, safe while serving
|
||||||
|
sudo systemctl stop frank-dragoon
|
||||||
|
bash scripts/census-frank-dragoon.sh ref # reference dump, old payload
|
||||||
|
bash scripts/census-frank-dragoon.sh cand # PPL + KL/top-1 vs that dump
|
||||||
|
bash scripts/census-frank-dragoon.sh niah-cur # baseline retrieval
|
||||||
|
bash scripts/census-frank-dragoon.sh niah-new # candidate retrieval
|
||||||
|
# 3. boot the candidate through a transient drop-in, then the battery
|
||||||
|
bash scripts/trial-frank-dragoon.sh candidate
|
||||||
|
python3 scripts/battery-frank-dragoon.py v2abl all
|
||||||
|
bash scripts/trial-frank-dragoon.sh restore # back onto the repo env
|
||||||
|
```
|
||||||
|
|
||||||
|
The battery needs two staged assets in `/tmp/halogen-eval/` (a 512 px solid
|
||||||
|
red PNG as base64, `red512.b64`, and the wikitext-2 raw test split,
|
||||||
|
`wikitext-2-raw-test.txt`). Laws it encodes:
|
||||||
|
|
||||||
|
- **the engine bills wikitext at ~4.33 characters/token** — the harness
|
||||||
|
chars/4 estimate under-bills by ~8 %, and a prompt over `HALOGEN_CTX` is
|
||||||
|
refused with a 400 that names the exact overage (never truncated). Size
|
||||||
|
haystacks in characters against a **token** budget.
|
||||||
|
- the wikitext-2 raw test split as staged joins rows with a **single**
|
||||||
|
newline (2,891 lines, zero blank lines), so a blank-line split returns one
|
||||||
|
1.29 MB "paragraph" and every budget silently becomes "everything".
|
||||||
|
- the first request after a boot is a warmup artifact; the battery discards
|
||||||
|
it (`warmup` row) and never measures it.
|
||||||
|
- `ppl`/`niah` **load the model** → stopped service. `verify` reads the file
|
||||||
|
back with no GPU and can run while serving.
|
||||||
|
|
||||||
|
## Contract (unchanged from the Swift1.5 era)
|
||||||
|
|
||||||
|
`reasoning_content` carries the thinking and counts against `max_tokens`
|
||||||
|
(≥512-token budget on any vision probe or the endpoint looks blind while it
|
||||||
|
is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
|
||||||
|
probes; `response_format` unsupported; the chat template rejects
|
||||||
|
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
|
||||||
|
mandatory); unknown sampling params are a 400; the prompt cache is on by
|
||||||
|
default, so a warm 100k follow-up costs seconds. See
|
||||||
|
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
|
||||||
|
carries verbatim, and [[deployment]] for the deploy/health gate.
|
||||||
@@ -1,10 +1,13 @@
|
|||||||
# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)
|
# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)
|
||||||
|
|
||||||
**ACTIVE since 2026-10-06.** Rollback = revert the swap commit's trio
|
**RETIRED 2026-10-06 → 2026-10-17**, superseded by the native v2 conversion:
|
||||||
(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
|
[[frank-dragoon-flash-server-v2-native]]. Rollback target — the payload
|
||||||
`scripts/frank-dragoon.service`); the retired 27B payload stays on disk at
|
stays on disk at `/home/dank/models/halogen-flashnext-v2/`,
|
||||||
`/home/dank/models/halogen-qwen3.8-27b/` and its fetch script
|
`scripts/download-frank-dragoon-flashnext.sh` re-verifies it, and restoring
|
||||||
(`scripts/download-frank-dragoon-halogen.sh`) stays in the repo.
|
the era is a revert of the 2026-10-17 swap commit. The 27B payload this era
|
||||||
|
replaced stays at `/home/dank/models/halogen-qwen3.8-27b/` with
|
||||||
|
`scripts/download-frank-dragoon-halogen.sh`. Everything below is the era as
|
||||||
|
it shipped; the engine-flag detail still governs the current page.
|
||||||
|
|
||||||
## Why this shape exists
|
## Why this shape exists
|
||||||
|
|
||||||
|
|||||||
@@ -16,10 +16,10 @@ disk); tags frank-archon-freetoken-pr587-20261005 and
|
|||||||
frank-archon-vllm-shapeG-20261003),
|
frank-archon-vllm-shapeG-20261003),
|
||||||
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||||
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
||||||
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
digest serving the NATIVE v2 conversion of the abliterated
|
||||||
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
|
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
|
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||||
ctx, vision ON)** and
|
262,144-position KV pool, vision ON)** and
|
||||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||||
@@ -53,12 +53,12 @@ PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
|||||||
gfx1151 iGPU 114 GiB GTT]
|
gfx1151 iGPU 114 GiB GTT]
|
||||||
E[flash-server container gfx1151, digest-pinned image
|
E[flash-server container gfx1151, digest-pinned image
|
||||||
frank-dragoon, port 7770]
|
frank-dragoon, port 7770]
|
||||||
F[flash-server v2 abliterated Swift1.5 .hgn + ngram table + vision tower
|
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||||
2 x 200192 ctx, MTP always-on, image input]
|
ONE 262144-position KV pool, 2 slots, MTP always-on, image input]
|
||||||
G[llama.cpp master CUDA sm_86
|
G[llama.cpp PrismML fork CUDA build
|
||||||
frank-zealot, port 7771]
|
frank-zealot, port 7771]
|
||||||
H[Gemma4-26B-A4B QAT Uncensored Q4_K_M GGUF
|
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
||||||
1 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
2 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
||||||
E2 --> E --> F
|
E2 --> E --> F
|
||||||
E2 --> G --> H
|
E2 --> G --> H
|
||||||
end
|
end
|
||||||
@@ -77,23 +77,31 @@ frank-zealot, port 7771]
|
|||||||
|
|
||||||
## Active Deployment
|
## Active Deployment
|
||||||
|
|
||||||
- [[frank-dragoon-flash-server-v2-swift15]] — **ACTIVE since 2026-10-06**
|
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE since 2026-10-17**
|
||||||
(root systemd unit + podman, port 7770): the digest-pinned
|
(root systemd unit + podman, port 7770): the digest-pinned
|
||||||
`ghcr.io/peonist-ai/halogen-flash-server` 0.16.2 engine serving the
|
`ghcr.io/peonist-ai/halogen-flash-server` 0.16.2 engine serving the
|
||||||
Quat3rnion ABLITERATED Swift1.5 finetune of peonist's
|
**native v2 conversion** of the abliterated Qwen3.8-Flash-Next
|
||||||
Qwen3.8-Flash-Next v2 checkpoint (`qwen38-flash-next-v2-swift15-abliterated.hgn`,
|
(`qwen38-flash-next-v2-abliterated.hgn`, 66,687,678,432 B, Hub LFS
|
||||||
66.7 GB compacted r2 of the card's 89.8 GB append-only original,
|
sha256-verified at fetch) with the REQUIRED n-gram lookup table (51.2 GB,
|
||||||
sha256-verified) with the REQUIRED n-gram lookup table (51.2 GB,
|
@peonist 5cc17cea) and the vision tower (897.9 MB, @e053f488)
|
||||||
@peonist 5cc17cea) and the vision tower (897.9 MB, @e053f488) beside
|
**hardlinked** beside it — the card publishes only the main file, and the
|
||||||
it. TWO slots of 200,192 ctx, MTP head ALWAYS ON (`HALOGEN_SPEC_ADAPT=0`
|
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
|
||||||
— the adaptive default turns the head off per request and halves
|
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
|
||||||
decode), server-side sampler at the card's 1.0/0.95/20, coupled
|
Card-recommended `HALOGEN_KV_POOL_POSITIONS=262144`: **one pool behind two
|
||||||
cap/timeout 65536/7200, prompt cache ON by default (the 27B engine's
|
slots** — 7.2 GiB where the old shape spent 14.6, the per-request ceiling
|
||||||
manual `HALOGEN_CACHE_MB` pin era is over — warm 100k turn ≈ 2 s vs
|
unchanged at 262,144, and the freed RAM becomes page cache for the n-gram
|
||||||
≈ 88 s cold). Vision ON (image_url parts only). Card: 53-57 t/s
|
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||||
greedy, ~1,640 t/s prefill at 34k, ~72 % fewer thinking tokens.
|
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||||
Rollback: revert the swap commit; the 27B payload + fetch script stay
|
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
|
||||||
on disk
|
PPL +2.3 %, thinking tokens ×1.9 (the finetune's compression is gone).
|
||||||
|
Rollback: revert the swap commit; the swift15 payload and its fetch script
|
||||||
|
stay on disk
|
||||||
|
- [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17
|
||||||
|
(rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned
|
||||||
|
v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots × 200,192 ctx over
|
||||||
|
a 524,288-position pool; owns the 400-zone law (the harness chars/4
|
||||||
|
under-count against the engine ceiling), the MTP-always-on law, and the
|
||||||
|
sampler / prompt-cache / vision contract the native era carries verbatim
|
||||||
- [[frank-dragoon-halogen27b-dflash2]] — **RETIRED 2026-10-05 → 10-06
|
- [[frank-dragoon-halogen27b-dflash2]] — **RETIRED 2026-10-05 → 10-06
|
||||||
(rollback target; payload + fetch script + image on disk)**: the
|
(rollback target; payload + fetch script + image on disk)**: the
|
||||||
digest-pinned `ghcr.io/peonist-ai/halogen` 0.1.4 engine serving
|
digest-pinned `ghcr.io/peonist-ai/halogen` 0.1.4 engine serving
|
||||||
|
|||||||
@@ -302,17 +302,27 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||||||
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
||||||
rollback record)
|
rollback record)
|
||||||
- [[frank-dragoon-halogen27b-dflash2]] — the ACTIVE frank-dragoon runbook since
|
- [[frank-dragoon-halogen27b-dflash2]] — RETIRED 2026-10-05 → 10-06 frank-dragoon
|
||||||
2026-10-05 (root systemd unit + podman, port 7770): the digest-pinned
|
runbook (root systemd unit + podman, port 7770): the digest-pinned
|
||||||
ghcr.io/peonist-ai/halogen image serving peonist-ai/halogen-qwen3.8-27b
|
ghcr.io/peonist-ai/halogen image serving peonist-ai/halogen-qwen3.8-27b
|
||||||
(qwen3.8-27b-p1w4d-d2.hgn, 33.4 GiB — a .hgn that loads in no other
|
(qwen3.8-27b-p1w4d-d2.hgn, 33.4 GiB — a .hgn that loads in no other
|
||||||
engine) with the DFlash2 drafter on ONE slot of 200,700 ctx (slots>1
|
engine) with the DFlash2 drafter on ONE slot of 200,700 ctx (slots>1
|
||||||
disables speculation — engine law); TEXT-ONLY and censored base, both
|
disables speculation — engine law); TEXT-ONLY and censored base, both
|
||||||
accepted tradeoffs; no server-side sampler, so the thinking recipe rides
|
accepted tradeoffs; no server-side sampler, so the thinking recipe rides
|
||||||
the wire
|
the wire
|
||||||
- [[frank-dragoon-flash-server-v2-swift15]] — the ACTIVE frank-dragoon runbook
|
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook
|
||||||
since 2026-10-06 (supersedes halogen 0.1.4): halogen-flash-server 0.16.2
|
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
|
||||||
serving the abliterated Swift1.5 Qwen3.8-Flash-Next v2 (.hgn)
|
abliterated** conversion of orcarouter's pinned BF16
|
||||||
|
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) with the card's
|
||||||
|
memory-saving `HALOGEN_KV_POOL_POSITIONS=262144` (7.2 GiB, one pool behind
|
||||||
|
both slots) and the stock peonist n-gram/vision companions **hardlinked**
|
||||||
|
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
|
||||||
|
always-on, vision ON, no server-side sampler change
|
||||||
|
- [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17
|
||||||
|
frank-dragoon runbook (rollback target — payload + fetch script on disk):
|
||||||
|
halogen-flash-server 0.16.2 serving the abliterated **Swift1.5 finetune**
|
||||||
|
of the Qwen3.8-Flash-Next v2 `.hgn`, two slots over a 524,288-position
|
||||||
|
pool; owns the 400-zone law and the sampler/prompt-cache contract
|
||||||
- [[frank-dragoon-rocmfpx-q6agent-uncensored-mtp]] — RETIRED 2026-10-03 → 10-05
|
- [[frank-dragoon-rocmfpx-q6agent-uncensored-mtp]] — RETIRED 2026-10-03 → 10-05
|
||||||
(rollback target; payload, fork build and build script on disk): the pinned
|
(rollback target; payload, fork build and build script on disk): the pinned
|
||||||
kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
|
kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
|
||||||
|
|||||||
Reference in New Issue
Block a user