docs: frank-large halogen runbook, AGENTS.md + indices for the swap

New runbook frank-large-halogen27b-dflash2.md; LLM + vault indices
re-pointed, ROCmFPX q6agent runbook marked RETIRED (rollback target);
AGENTS.md topology row, detail section, image-capability paragraph
(two endpoints accept images now), bench-law prose and runbook index
updated.
This commit is contained in:
Frank committed 2026-10-05 16:29:11 -07:00
1 parent fece3ffdd0
commit 0900551317
5 files changed
+219 -102

No files matched your search

+27 -18
View File
@@ -14,11 +14,11 @@ mounted cache dir** (the image ships no `/root/.cache` at all; unmounted, every
`torch.compile took 181.20 s`). `--max-num-seqs` must stay 3:
SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
and Dankclaw
runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
session at 262,144, f16 KV, vision via the BF16 projector)** and
runs two endpoints — **halogen container (frank-large, port 7770:
ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
lives on frank-small)** and
**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q6_K
GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.
@@ -45,10 +45,10 @@ expert-precise-memory arena 7,522 slots, /root/.cache mounted]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 114 GiB GTT]
E[ROCmFPX llama.cpp gfx1151, pinned fork build
E[halogen container gfx1151, digest-pinned image
frank-large, port 7770]
F[Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP-Q4_0 head
1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
1 x 200700 ctx, d2 KV, text-only]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
H[MiMo-V2.6 Q5_K_M GGUF
@@ -71,16 +71,17 @@ frank-small, port 7771]
## Active Deployment
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **ACTIVE since 2026-10-03**
(root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
the uncensored protected-heads rung `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`
(23.15 GiB, ftype 114 = 159 type-103 + 339 GGML_TYPE_Q6_0_ROCMFPX tensors,
64 protected output heads) with the repo's `mtp-Qwen3.8-27B-Q4_0.gguf` head
drafting 4 tokens, one 262,144 window, f16 KV, the BF16 projector for
vision. Measured 27.6-28.1 t/s prose decode at 86.8 % draft acceptance; the
ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
disk
- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
(root systemd unit + podman, port 7770): the digest-pinned
`ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
(`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
(the engine ignores the vision encoder; frank-small owns images) and the
CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
No server-side sampler — the thinking recipe rides the wire in models.yml.
Rollback: revert the swap commit; the ROCmFPX payload + fork build +
`build-rocmfpx.sh` pin all remain on disk
- [[frank-small-llama-mimo-q6k-2x200704]] — **ACTIVE since 2026-10-03**
(root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090, TWO
@@ -202,6 +203,14 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **RETIRED 2026-10-03 →
10-05 (rollback target; payload, fork build and build script on disk)**:
the pinned ROCmFPX fork serving the uncensored protected-heads
Q6_0_ROCMFPX_AGENT rung with the repo's MTP-Q4_0 head drafting 4 tokens,
one 262,144 window, f16 KV; measured 27.6-28.1 t/s prose decode at 86.8 %
acceptance. Replaced by the halogen DFlash2 endpoint; the ngram-mod-onto-MTP
rejection (-15 %, recurrent-state rollback) stays a live lesson for this
architecture.
- [[frank-large-rocmfpx-q8agent-mtp]] — **RETIRED 2026-10-02 → 10-03 (the
rollback target; payload still on disk)**: the same pinned ROCmFPX fork
serving the ALIGNED `Q8_0_ROCMFPX_AGENT` (ftype 115, 17.0 t/s, MTP