docs(frank-dragoon): native-era runbook; retire the swift15 page
New LLM/frank-dragoon-flash-server-v2-native.md carries the shape with the three deliberate deviations from the card (served id, SPEC_ADAPT=0, engine pin 0.16.2), the memory ledger with the PIN_TRUNK rejection, the full measured A/B table, and the companion law (hardlinked stock peonist files, engine-tool certified). The swift15 page is re-banned RETIRED 2026-10-06 → 10-17 with its rollback terms; both indexes and AGENTS.md (topology table, service section, runbook index) are re-keyed, and the root index also stops describing the halogen-27B page as ACTIVE.
This commit is contained in:
1 parent
e36f6a732c
commit
13162e6615
5 files changed
+294
-69
No files matched your search
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
| Target | Service | Runtime | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=262144` pool** (the card's recommendation: the ledger's 7.2 GiB where the old 524,288 pool measured 14.6 — per-request ceiling unchanged at 262,144, two maximum-length conversations no longer coexist, and the freed RAM is page cache for the n-gram table), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
|
||||
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
||||
|
||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||
@@ -334,7 +334,7 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||
|
||||
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
||||
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, ACTIVE)
|
||||
|
||||
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
||||
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
||||
@@ -346,19 +346,23 @@ run` FOREGROUND — the container's exit code is the service's exit so
|
||||
7770 from the Strix Halo iGPU. Engine stdout goes to
|
||||
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
|
||||
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
||||
Cutover 2026-10-06 from the halogen 27B-dense era
|
||||
([[frank-dragoon-halogen27b-dflash2]], retired the same window).
|
||||
Cutover 2026-10-17 from the Swift1.5-finetune shape
|
||||
([[frank-dragoon-flash-server-v2-swift15]], retired the same window; it
|
||||
itself cut over the halogen 27B-dense era on 2026-10-06).
|
||||
|
||||
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
||||
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
||||
GiB, sha256 `9c880b8b…` — a compacted r2 of the card's 89,775,868,032 B
|
||||
append-only original; the abliteration workflow appends and the repo's
|
||||
tooling drops dead padding, live tensors byte-identical) + the **REQUIRED**
|
||||
n-gram lookup table `qwen38-flash-next-ngram.hgn` (51,200,246,144 B,
|
||||
peonist @5cc17cea) + the vision tower `qwen38-flash-next-vision.hgn`
|
||||
(897,916,416 B, @e053f488) + the flat `tokenizer/` directory (6 files,
|
||||
fetched fresh from peonist). Cutover 2026-10-06 from the halogen
|
||||
27B-dense shape, chosen by Frank for the ABLITERATED base.
|
||||
Serving `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`:
|
||||
`qwen38-flash-next-v2-abliterated.hgn` (66,687,678,432 B ≈ 62.1 GiB, sha256
|
||||
`b5bd925f…` — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned
|
||||
BF16 checkpoint, abliterated; NOT the writer transplant that carried this
|
||||
filename, which measured 66,687,678,464 B — 32 bytes apart, same filename
|
||||
family, different model) + the **REQUIRED** n-gram lookup table
|
||||
`qwen38-flash-next-ngram.hgn` (51,200,246,144 B, peonist @5cc17cea) + the
|
||||
vision tower `qwen38-flash-next-vision.hgn` (897,916,416 B, @e053f488) + the
|
||||
flat `tokenizer/` directory (6 files) — the three companions **hardlinked**
|
||||
out of the swift15 payload dir, because the release publishes only the main
|
||||
file and no source-matched pair exists. Cutover 2026-10-17 from the Swift1.5
|
||||
finetune, chosen by Frank for the base checkpoint itself plus the card's
|
||||
memory-saving pool.
|
||||
|
||||
**The format is the lock.** The v2 `.hgn` loads ONLY in the
|
||||
flash-server line — the 27B-era halogen 0.1.4 container refuses it, so the
|
||||
@@ -389,8 +393,13 @@ the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
|
||||
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
||||
law does NOT carry over; batching and speculation coexist (the card's
|
||||
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
||||
`HALOGEN_KV_POOL_POSITIONS=524288` (explicit 2 × ctx; ~28 KiB/position ⇒
|
||||
14.0 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
`HALOGEN_KV_POOL_POSITIONS=262144` (the card's recommendation and the memory
|
||||
saving of the 2026-10-17 swap; ~28 KiB/position ⇒ 7.2 GiB) is **ONE pool
|
||||
behind both slots** — ledger `2 slots over ONE 262144-position KV pool, each
|
||||
request up to 262144` — so the per-request ceiling is untouched while two
|
||||
MAXIMUM-length conversations no longer coexist (the old 524,288 = 2 × ctx
|
||||
spent the measured 14.6 GiB and allowed it).
|
||||
`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
|
||||
@@ -407,27 +416,34 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
||||
"video".
|
||||
|
||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
||||
weights ~62 + KV 14.0 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
||||
~2 ≈ 86 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~15 GiB
|
||||
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
|
||||
page cache beside the checkpoint — the card's quickstart arithmetic does
|
||||
not bill it as resident, but READ the boot ledger and re-size if the
|
||||
engine prints a clamp.
|
||||
weights ~62.9 (trunk 62.11 + vision 0.84) + KV 7.2 + arena ~8
|
||||
(`3.8 + 8.4 × MAX_TOK/16384`) ≈ 78 GiB engine-side +
|
||||
`HALOGEN_HOST_RESERVE_GIB=20` → ~25 GiB margin over baseline co-tenancy.
|
||||
The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint
|
||||
(ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why the pool
|
||||
cut is a double saving: **RAM the pool takes is RAM that table loses**.
|
||||
`HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the v2 loader, so disk-backing
|
||||
the trunk is not a lever on this format. READ the boot ledger and re-size
|
||||
if the engine prints a clamp.
|
||||
|
||||
Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
|
||||
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
||||
sha256 + `--verify-only`, no token on the command line; resume goes
|
||||
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
||||
The serve gates mirror the deployer EXACTLY (byte-exact checkpoint +
|
||||
n-gram + vision tower, flat tokenizer with all six files as regular
|
||||
files) — the 2026-10-01 crash-loop lesson.
|
||||
Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
|
||||
`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
|
||||
sha256 + `--verify-only`, no token on the command line; resume goes against
|
||||
the RESOLVED CDN URL because curl drops `Range` across HF's 302) — the main
|
||||
file is fetched, the three companions are HARDLINKED from the verified
|
||||
swift15 dir (a hardlink is a regular file, so the no-symlink serve gate
|
||||
passes without a 47.7 GiB copy). The serve gates mirror the deployer EXACTLY
|
||||
(byte-exact checkpoint + n-gram + vision tower, flat tokenizer with all six
|
||||
files as regular files) — the 2026-10-01 crash-loop lesson.
|
||||
|
||||
Runbook:
|
||||
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
|
||||
Rollback: revert the swap commit and redeploy — the 27B payload
|
||||
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
||||
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
||||
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md`.
|
||||
Rollback: revert the swap commit and redeploy — the Swift1.5 payload
|
||||
(`/home/dank/models/halogen-flashnext-v2/`, re-verified by
|
||||
`scripts/download-frank-dragoon-flashnext.sh`), the 27B payload
|
||||
(`/home/dank/models/halogen-qwen3.8-27b/`) and the halogen 0.1.4 image
|
||||
digest all remain on disk; the ROCmFPX door behind that
|
||||
(`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||
|
||||
### frank-zealot — llama.cpp (PrismML fork) on the RTX 3090 (Dankclaw, ACTIVE)
|
||||
|
||||
@@ -1140,7 +1156,7 @@ Active:
|
||||
|
||||
| Service | Runbook |
|
||||
|---------|---------|
|
||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||
| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 262,144 pool, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
|
||||
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
||||
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
@@ -1153,6 +1169,7 @@ Retired (rollback targets and history):
|
||||
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||
| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, 2 slots × 200,192 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||
|
||||
Reference in new issue
Block a user