docs: frank-large halogen runbook, AGENTS.md + indices for the swap
New runbook frank-large-halogen27b-dflash2.md; LLM + vault indices re-pointed, ROCmFPX q6agent runbook marked RETIRED (rollback target); AGENTS.md topology row, detail section, image-capability paragraph (two endpoints accept images now), bench-law prose and runbook index updated.
This commit is contained in:
1 parent
fece3ffdd0
commit
0900551317
5 files changed
+219
-102
No files matched your search
@@ -14,11 +14,11 @@ mounted cache dir** (the image ships no `/root/.cache` at all; unmounted, every
|
||||
`torch.compile took 181.20 s`). `--max-num-seqs` must stay 3:
|
||||
SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
|
||||
and Dankclaw
|
||||
runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
|
||||
kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
|
||||
Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
|
||||
read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
|
||||
session at 262,144, f16 KV, vision via the BF16 projector)** and
|
||||
runs two endpoints — **halogen container (frank-large, port 7770:
|
||||
ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
|
||||
/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
|
||||
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
|
||||
lives on frank-small)** and
|
||||
**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q6_K
|
||||
GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.
|
||||
|
||||
@@ -45,10 +45,10 @@ expert-precise-memory arena 7,522 slots, /root/.cache mounted]
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
E2[RTX 3090 (PCIe x4 dock) +
|
||||
gfx1151 iGPU 114 GiB GTT]
|
||||
E[ROCmFPX llama.cpp gfx1151, pinned fork build
|
||||
E[halogen container gfx1151, digest-pinned image
|
||||
frank-large, port 7770]
|
||||
F[Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP-Q4_0 head
|
||||
1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
|
||||
F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
|
||||
1 x 200700 ctx, d2 KV, text-only]
|
||||
G[llama.cpp master CUDA sm_86
|
||||
frank-small, port 7771]
|
||||
H[MiMo-V2.6 Q5_K_M GGUF
|
||||
@@ -71,16 +71,17 @@ frank-small, port 7771]
|
||||
|
||||
## Active Deployment
|
||||
|
||||
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **ACTIVE since 2026-10-03**
|
||||
(root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
|
||||
the uncensored protected-heads rung `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`
|
||||
(23.15 GiB, ftype 114 = 159 type-103 + 339 GGML_TYPE_Q6_0_ROCMFPX tensors,
|
||||
64 protected output heads) with the repo's `mtp-Qwen3.8-27B-Q4_0.gguf` head
|
||||
drafting 4 tokens, one 262,144 window, f16 KV, the BF16 projector for
|
||||
vision. Measured 27.6-28.1 t/s prose decode at 86.8 % draft acceptance; the
|
||||
ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
|
||||
hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
|
||||
disk
|
||||
- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
|
||||
(root systemd unit + podman, port 7770): the digest-pinned
|
||||
`ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
|
||||
(`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
|
||||
of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
|
||||
31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
|
||||
(the engine ignores the vision encoder; frank-small owns images) and the
|
||||
CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
|
||||
No server-side sampler — the thinking recipe rides the wire in models.yml.
|
||||
Rollback: revert the swap commit; the ROCmFPX payload + fork build +
|
||||
`build-rocmfpx.sh` pin all remain on disk
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — **ACTIVE since 2026-10-03**
|
||||
(root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090, TWO
|
||||
@@ -202,6 +203,14 @@ frank-small, port 7771]
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **RETIRED 2026-10-03 →
|
||||
10-05 (rollback target; payload, fork build and build script on disk)**:
|
||||
the pinned ROCmFPX fork serving the uncensored protected-heads
|
||||
Q6_0_ROCMFPX_AGENT rung with the repo's MTP-Q4_0 head drafting 4 tokens,
|
||||
one 262,144 window, f16 KV; measured 27.6-28.1 t/s prose decode at 86.8 %
|
||||
acceptance. Replaced by the halogen DFlash2 endpoint; the ngram-mod-onto-MTP
|
||||
rejection (-15 %, recurrent-state rollback) stays a live lesson for this
|
||||
architecture.
|
||||
- [[frank-large-rocmfpx-q8agent-mtp]] — **RETIRED 2026-10-02 → 10-03 (the
|
||||
rollback target; payload still on disk)**: the same pinned ROCmFPX fork
|
||||
serving the ALIGNED `Q8_0_ROCMFPX_AGENT` (ftype 115, 17.0 t/s, MTP
|
||||
|
||||
Reference in new issue
Block a user