docs: frank-large Q6 uncensored cutover + frank-small 2x200704 f16-KV reslice
- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors), 27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain rejection (-15% prose: recurrent-state rollback disabled), the bench-window hardening laws - frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected (gufo payloads removed from disk 2026-10-03 - that rollback is now a re-download; the aligned Q8 file is the direct one, still on disk) - frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully unquantized f16 KV: boot-pair table gains the measured row (20,530 used / 4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to the 3x180224 + q8_0-V shape documented - AGENTS.md: fleet table rows, both endpoint sections, the chain law extended with the recurrent-model measurement, the --parallel law with the reslice, runbook index (active + retired) - LLM/index.md + root index entries updated
This commit is contained in:
1 parent
47a2c1287b
commit
b80886e318
6 files changed
+438
-216
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP"
|
||||
description: "The active frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, and the bundled BF16 projector for vision. Measured MTP-vs-none A/B, the client contract, and rollback to gufo."
|
||||
title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (retired)"
|
||||
description: "The retired frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head, f16 KV, and the BF16 projector. Measured MTP-vs-none A/B, the client contract, and the rollback state as of 2026-10-03."
|
||||
tags:
|
||||
- llm
|
||||
- frank-large
|
||||
@@ -15,14 +15,20 @@ tags:
|
||||
timestamp: 2026-10-02
|
||||
---
|
||||
|
||||
# frank-large — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151)
|
||||
# frank-large — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151, retired)
|
||||
|
||||
> **ACTIVE since 2026-10-02.** Cut over from
|
||||
> [[frank-large-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s). Both requests came from
|
||||
> Frank: *highest quality that runs*, and *one full context*. This endpoint
|
||||
> delivers 8.39 bpw weights — two quant rungs above what gufo could load — and
|
||||
> measures **17.0 t/s** decode instead of 15, so quality was not bought with
|
||||
> speed.
|
||||
> **RETIRED 2026-10-03** — superseded by
|
||||
> [[frank-large-rocmfpx-q6agent-uncensored-mtp]]: Frank swapped to the
|
||||
> uncensored Q6_0_ROCMFPX_AGENT file (protected heads, one rung down), which
|
||||
> measures **27.6–28.1 t/s** at a deeper prompt than this file's 17.0. This
|
||||
> runbook stays live as the rollback reference and the trap catalogue; the
|
||||
> payload it documents remains on disk at
|
||||
> `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`.
|
||||
>
|
||||
> Cut over 2026-10-02 from [[frank-large-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s)
|
||||
> on two requests from Frank: *highest quality that runs*, and *one full
|
||||
> context*. It delivered 8.39 bpw weights — two quant rungs above what gufo
|
||||
> could load — at 17.0 t/s, so quality was not bought with speed.
|
||||
>
|
||||
> The capability that forced this whole shape is a GGUF tensor type:
|
||||
> **`GGML_TYPE_Q8_0_ROCMFPX` (103)**. It exists in exactly one loader on this
|
||||
@@ -184,17 +190,24 @@ broken payload never stops the running unit. Caller environment **overrides**
|
||||
`frank-large.env`, and defaults use `:=` so an explicitly empty value surfaces
|
||||
as the error it reads as.
|
||||
|
||||
## Rollback
|
||||
## Rollback (as of retirement, 2026-10-03)
|
||||
|
||||
git tag **`frank-large-gufo-q4kxl-20261001`**: gufo v0.1.1
|
||||
(`scripts/build-gufo.sh` still on disk), the Q4_K_XL Qwen3.8-27B payload and the
|
||||
DFlash2 drafter all remain on the host. Restore that tag's unit/env/serve trio
|
||||
and redeploy — nothing here deletes anything.
|
||||
**Primary rollback: the ALIGNED Q8 file itself is the previous shape's
|
||||
payload and remains on disk** at
|
||||
`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` — revert the
|
||||
2026-10-03 swap commit (unit/env/serve trio) or repoint
|
||||
`FRANK_LARGE_MODEL_FILE`/`LLM_LARGE_MODEL_FILE`, verify with
|
||||
`scripts/download-frank-large-kingjones.sh --verify-only`, redeploy.
|
||||
|
||||
The rollback payloads stay exactly where the tag's env file expects them:
|
||||
`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and the DFlash2
|
||||
drafter `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf`
|
||||
(1.07 GiB). The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
|
||||
git tag **`frank-large-gufo-q4kxl-20261001`** (gufo v0.1.1, Q4_K_XL +
|
||||
DFlash2) is now a **re-download** rollback: the gufo payloads
|
||||
`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and
|
||||
`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/` (incl. the DFlash2 drafter) were
|
||||
removed from disk on 2026-10-03 with Frank's explicit approval to free space.
|
||||
`scripts/build-gufo.sh` remains on disk; re-fetch per
|
||||
[[frank-large-gufo-qwen38-27b]].
|
||||
|
||||
The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
|
||||
Qwen3.5-9B MTP source 19 GiB, both DFlash draft dirs) were staged to
|
||||
`/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 —
|
||||
recoverable until the trash is purged, and never part of a rollback plan.
|
||||
|
||||
Reference in new issue
Block a user