dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

Browse Source 
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank committed 2026-10-10 17:03:14 -07:00
1 parent 5ced474eb9
commit 55f87fcdce
195 files changed
+2085 -2064
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/LLM/frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md
+177
View File
@@ -0,0 +1,177 @@
---
type: reference
title: "frank-dragoon on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP"
description: "The active frank-dragoon runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Uncensored Q6_0_ROCMFPX_AGENT GGUF (ftype 114, protected output heads), with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, the BF16 projector for vision, and the measured rejection of the ngram-mod chain."
tags:
- llm
- frank-dragoon
- dankclaw
- llama-cpp
- rocmfpx
- gfx1151
- gguf
- spec-decode
- mtp
timestamp: 2026-10-03
---
# frank-dragoon — ROCmFPX Uncensored Q6_0_ROCMFPX_AGENT + MTP (gfx1151)
> **RETIRED 2026-10-03 → 2026-10-05 (rollback target; payload, fork build and
> build script all on disk).** Replaced by [[frank-dragoon-halogen27b-dflash2]]
> (DFlash2, publisher 31.7 t/s mean vs the 27.6–28.1 measured here). This page
> remains the contract record for the rollback. Original banner follows.
> ACTIVE since 2026-10-03. Cut over from
> [[frank-dragoon-rocmfpx-q8agent-mtp]] (ALIGNED Q8_0_ROCMFPX_AGENT, 17.0 t/s)
> on Frank's request: the *uncensored* variant of the same protected-heads
> family, one quant rung down. The swap turned out to be a strict upgrade on
> the axis that matters here: **decode 27.6–28.1 t/s at a 4,813-token prompt
> against 17.0 at a 2,202-token prompt** — faster with a deeper context, not a
> same-depth comparison, but the sign is not in question.
## Identity
| | |
|---|---|
| Host / device | Dankclaw (192.168.0.143), Radeon 8060S iGPU `gfx1151`, ~114 GiB GTT aperture |
| Endpoint | `http://192.168.0.143:7770` — alias `frank-dragoon`, `/v1` OpenAI surface |
| Runtime | **`kingjones30/ROCmFPX`** llama.cpp fork @ `36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built by [`scripts/build-rocmfpx.sh`](../../scripts/build-rocmfpx.sh) — unchanged from the Q8 era; the fork already knew type 102 |
| Unit | root systemd `frank-dragoon.service` ([`scripts/frank-dragoon.service`](../../scripts/frank-dragoon.service)), `User=dank`, `Restart=always` |
| Serve script | [`scripts/serve-frank-dragoon.sh`](../../scripts/serve-frank-dragoon.sh) (owns the argv), tunables [`scripts/frank-dragoon.env`](../../scripts/frank-dragoon.env) |
| Weights | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` — 24,856,607,008 B, sha256 `e966deb3…` |
| Draft | `…/mtp-Qwen3.8-27B-Q4_0.gguf` — 1,680,271,648 B, sha256 `051a1764…` (byte-identical to the aligned repo's head; hardlinked, not re-downloaded) |
| Projector | `…/mmproj-Qwen3.8-27B-BF16.gguf` — 931,145,888 B (hardlinked from the aligned payload dir; the same file the Q8 era served) |
| Fetch | [`scripts/download-frank-dragoon-kingjones-uncensored.sh`](../../scripts/download-frank-dragoon-kingjones-uncensored.sh) — hardlinks the draft/projector from the aligned dir when present, otherwise fetches; `-f`, byte-count + LFS sha256 verified, `--verify-only`, token never on the command line |
| Seam | `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV dtype) |
## Why this file
`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF` ships the same
"protected heads" construction as the aligned repo, in the uncensored tuning.
Parsed header of the deployed file:
| Field | Value |
|---|---|
| ftype | **114** — the fork's 64-bit-block family, ≈7.44 bpw |
| Tensor mix | 353 F32 + **339 type-102 (`GGML_TYPE_Q6_0_ROCMFPX`)** + **159 type-103 (`Q8_0_ROCMFPX`)** |
| Protected heads | 64 output-head tensors kept at plain precision — the AGENT construction from the aligned repo, at the 6-bit rung |
| Arch | qwen35, 64 layers, `full_attention_interval 4`, `context_length 262144` |
| Embedded sampling recipe | temp 1.0 / top_p 0.95 / top_k 20 |
The same lock-in law as the Q8 era applies: types 102 and 103 exist in no
loader but this fork; stock llama.cpp and gufo refuse the file at load
(`unknown tensor type`). The loader was not re-litigated — the file picks the
runtime, and the pin in `scripts/build-rocmfpx.sh` (commit by SHA) stays.
Rejected sibling: `Q6_0_ROCMFPX_LEAN` (6.45 bpw, no protected heads) — the
protection is what keeps the output-head logits at full precision and the
drafter accurate, so it stays deployed.
## Shape and budget
Unchanged from the Q8 era — the swap was payload-only:
| Dial | Value | Why |
|---|---|---|
| `--ctx-size` | 262,144 | the native window, no rope scaling |
| `--parallel` | 1 | one conversation, all memory to it |
| KV dtype | **f16** (both K and V) | ~64 KiB/token; free inside a 114 GiB aperture |
| `--fit off` | on | the auto-fitter would silently resize the window |
| `--flash-attn on` | on | required for the long-window attention path |
| `--direct-io`, `--ctx-checkpoints 16`, `--cache-ram 8192`, `--cache-idle-slots`, `--jinja`, `--reasoning on --reasoning-format deepseek --reasoning-budget -1` | | same as [[frank-dragoon-rocmfpx-q8agent-mtp]] |
Spec decode: `--spec-type draft-mtp`, the repo's `mtp-Qwen3.8-27B-Q4_0.gguf`,
`--spec-draft-n-max 4`, pinned `--spec-draft-device ROCm0`.
Measured steady state: GTT **~61 GiB** under live load (host baseline 16.7),
boot to `/health` **~15 s**.
## Measured on this payload (2026-10-03, production service, `/completion` timings, temp 0, `cache_prompt: false`)
| Traffic | Decode t/s | Prefill t/s | Draft acceptance |
|---|---|---|---|
| Prose, 4,813-token prompt, 512 tokens | **27.57 / 28.10** (two runs) | ~205–217 | **396/456 = 0.868** → mean acceptance length ≈ 4.4, i.e. drafting at nearly the full n-max 4 |
| Schema echo (structured repetition) | **31.44** | — | 639/639 = 1.000 |
| Block copy (text repeated verbatim) | ~28.0 | — | 0.848 |
vs the Q8 era's 0.462 acceptance at a 2,202-token prompt: the MTP head drafts
materially better on this file. The card's claimed 0.953-class acceptance is
no longer science fiction on real prose — but keep reading it as a file
property, not a throughput promise.
## The ngram-mod chain — measured and rejected
Frank's ask: layer `--spec-type draft-mtp,ngram-mod` on top of MTP for an
extra gain. Measured in three bench windows (production service stopped per
the benchmarking law; artifacts `/home/dank/fl-ngram-ab/` on Dankclaw),
comparing `draft-mtp` alone vs the chain on the same traffic:
| Traffic | MTP alone | MTP + ngram-mod | Delta |
|---|---|---|---|
| Prose | 27.57 t/s | 23.40 t/s | **−15 %** |
| Block copy | 28.0 t/s | 26.1 t/s | **−7 %** (with ngram-mod demonstrably firing: 138 proposals/run) |
Mechanism, straight from arm B's boot log:
```
recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support
```
On the qwen35 hybrid (Gated DeltaNet recurrent layers) adding the ngram layer
disables the rollback path MTP's own verification relies on, so acceptance —
and throughput — drop on *all* traffic, even where the ngram matcher fires.
**Deployed decision: `draft-mtp` alone.** Chaining is not free on recurrent
models; the fleet law in `AGENTS.md → llama.cpp serving` records it.
## Bench-window hardening laws (learned building the A/B)
1. **Health 200 is not proof of ownership.** After boot, verify the pid you
started owns the port (`ss -ltnp`); a zombie server from a prior window
answers `/health` too and silently serves your arm.
2. **Kill scratch servers with SIGKILL after a grace period.** llama-server's
graceful SIGTERM waits on open keep-alive connections — a harness holding
the socket leaves the process (and its ~24 GiB of GTT) alive after the
script exits.
3. **Prove the port is free and GTT dropped between arms**, or the second
arm measures a collision.
## Client contract
Unchanged from the Q8 era and re-verified on this payload 2026-10-03: model id
`frank-dragoon`, native tools, thinking in `reasoning_content`,
`reasoning_effort` ∈ {xhigh, medium, low} only (anything else HTTP 500),
card-recipe sampling defaults server-side, vision through the BF16 projector
(red PNG → **`Red`**, 74 prompt / 165 completion tokens). The traps list —
`content: null` when reasoning eats the budget, the self-recovering MTP
boundary log line, `enable_thinking:false` not `reasoning_effort:"none"` — is
in [[frank-dragoon-rocmfpx-q8agent-mtp]] and still applies.
Sweep key for scratch servers is now the payload name:
`pkill -f 'Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT'` — never `pkill -f
llama-server` (frank-zealot is also one).
## Operations
```bash
./scripts/deploy.sh dankclaw --service frank-dragoon
systemctl status frank-dragoon
journalctl -u frank-dragoon -n 200 --no-pager
scripts/download-frank-dragoon-kingjones-uncensored.sh --verify-only
```
## Rollback
| Target | How |
|---|---|
| **ALIGNED Q8_0_ROCMFPX_AGENT** (previous shape, 17 t/s) | the payload dir `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` is untouched on disk; revert the swap commit (unit/env/serve trio) and redeploy, or point `FRANK_DRAGOON_MODEL_FILE` + `LLM_DRAGOON_MODEL_FILE` at the old paths. Verify with `scripts/download-frank-dragoon-kingjones.sh --verify-only` |
| gufo Q4_K_XL era | git tag `frank-dragoon-gufo-q4kxl-20261001`, **but the gufo payloads (UD-Q4_K_XL 16.35 GiB, UD-Q8_K_XL + DFlash2 drafter) were removed from disk 2026-10-03 with Frank's approval** — this rollback now requires re-download per [[frank-dragoon-gufo-qwen38-27b]] |
## When it misbehaves
Same decision tree as [[frank-dragoon-rocmfpx-q8agent-mtp]]:
`unknown tensor type` ⇒ off the pinned fork; SIGSEGV in
`ggml_compute_forward_set_rows` ⇒ pre-`36e9acd` build; slow decode with no
error ⇒ draft not engaged (`draft_n` in `/completion` timings) or an orphaned
second copy eating GTT (`/sys/class/drm/card1/device/mem_info_gtt_used`);
500 naming Jinja ⇒ `--jinja` lost; 500 naming the template ⇒ unmapped
`reasoning_effort` in `models.yml`.