dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: OKF v0.1 compliance pass + fact-check + 3 research briefs

Browse Source 
- Migrate all 127 non-reserved docs to OKF v0.1 frontmatter (type/description/tags/timestamp); fix 13 YAML issues, 2 duplicate-FM blocks, strip reserved-page FM
- Restructure log.md to OKF (## YYYY-MM-DD, newest-first)
- Fact-check against scripts/AGENTS.md ground truth: fix 6 false 'Active' banners on retired runbooks, 96->114 GiB GTT, deployment.md vs deploy-plan.py/deploy-remote.sh, Deployment Flow frank-medium=Quadlet, Calibre 9.9.0->9.15.0 + CVE-2025-7404 conflation, Copyparty v1.20.24, RomM 5.3.0 breaking change
- Move *arr docs to new FrankBook-Pro/ section (git mv, link-safe)
- Write active runbook bodies from ground truth (frank-large, frank-small)
- Add 3 research briefs + HTML report: Local TTS and Voice Cloning, AI Job Search and Resume Tools, Most Popular Self-Hosted AI Apps (Jun-Sep 2026)
- Add upstream-version rows to ComfyUI/OpenWebUI/Hermes app docs
- All 336 tests pass; 136 pages, 0 unresolved wikilinks
This commit is contained in:
Frank committed 2026-09-22 02:09:41 -07:00
1 parent 24f7a7772d
commit 93d1181fee
103 files changed
+2047 -4329
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/LLM/frank-large-kairic-edge-qwen38-27b.md
+24 -188
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Kairic Edge Qwen3.8-27B IU4 (W4A4) on Dankclaw (frank-large)
description: The active frank-large runbook — the Kairic Edge runtime (ciru-ai/ROCmFPX, branch kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (27B dense, IU4 W4A4) on the Dankclaw Strix Halo iGPU: the source-built runtime, the full-window/one-stream shape, the MTP depth 4 + n-gram spec method, the three PromptForge sidecars, the compatibility mode, the payload, the client contract, and the failure modes worth knowing.
title: "Kairic Edge Qwen3.8-27B IU4 (W4A4) on Dankclaw (frank-large)"
description: "The RETIRED frank-large runbook (2026-09-21, same-day swap; replaced by [[frank-large-unsloth-qwen38-27b-q8xl]]) — the Kairic Edge runtime (ciru-ai/ROCmFPX, branch kairic-edge-qwen38-27b-v1.2, commit 205a3e5, TheRock 7.15.0, Composable Kernel) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (27B dense, IU4 W4A4) on the Dankclaw Strix Halo iGPU: the source-built runtime, the full-window/one-stream shape, the MTP depth 4 + n-gram spec method, the three PromptForge sidecars, the compatibility mode, the payload, the client contract, and the failure modes worth knowing."
tags:
- llm
- kairic-edge
@@ -18,195 +18,31 @@ tags:
timestamp: 2026-09-21
---
# Kairic Edge Qwen3.8-27B IU4 (W4A4) on Dankclaw (frank-large)
# frank-large — Kairic Edge Qwen3.8-27B IU4 (W4A4) (RETIRED)
> **ACTIVE since 2026-09-21.** It replaced the unsloth llama.cpp prebuilt
> (b11030, Qwen3.8-Flash-Next UD-IQ4_XS, 125B MoE — see
> [[frank-large-unsloth-qwen38-flash-next]], the current rollback target),
> which had replaced the CIRU v4.4.1 Orca runtime (see
> [[frank-large-ciru-orca]]). Deployed as a **root systemd unit**
> (`scripts/frank-large.service`, `User=dank`), port 7770, on the same
> gfx1151 iGPU. The RTX 3090 it does not touch belongs to
> [[frank-small-gemma4-12b]].
> **RETIRED 2026-09-21 (same-day swap).** This was the ACTIVE frank-large
> runbook on 2026-09-21 before it was replaced by
> [[frank-large-unsloth-qwen38-27b-q8xl]] later the same day. The payload
> and runtime remain on disk as the rollback target.
The model is `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` — a **27B dense** Qwen3.8
at **IU4** (W4A4, integer-weights / integer-activations). It is a
fundamentally different architecture from the 125B MoE it replaces: no
PLE, no MoE experts, no n-gram embedding. The payload is four files: the
15.5 GiB main GGUF plus three `.pfs` sidecars (FFN 7.99 GiB, GDN 1.88
GiB, GDN-Output 0.70 GiB) — the PromptForge IU4 weight shards the Kairic
Edge runtime loads alongside the main GGUF.
## What it served
The runtime is the **Kairic Edge** build of `ciru-ai/ROCmFPX` (branch
`kairic-edge-qwen38-27b-v1.2`, commit `205a3e5`), built from source
(`scripts/build-kairic-edge-gfx1151.sh`). It is a llama.cpp build with the
PromptForge sidecar env block and the `--kairic-edge` qualified serving
path.
## Why this swap
- The operator asked for the Kairic Edge 27B model at **one conversation,
full context (262,144)** (2026-09-21).
- The Kairic Edge runtime is the qualified IU4 (W4A4) serving path — no
standard llama.cpp quant, no PLE sidecar payload, no QSA hotfix: one
GGUF, three sidecars, done.
- The spec method is native MTP depth 4 + n-gram drafting (24/64/64).
DFlash is NOT available for this model (no `DRAFT_DFLASH` spec type in
the Kairic Edge runtime).
## Shape
One root unit, one process, one published port:
| half | port | reachable |
|---|---|---|
| llama-server (OpenAI front-end + engine, single process) | 7770 | published as host **7770** |
## Runtime
- **Source build**: `scripts/build-kairic-edge-gfx1151.sh` (TheRock
7.15.0a20260718 toolchain, AMD clang 23.0.0, HIP 7.15.0, GCC 13.3.0,
CMake 4.4.2, Ninja 1.13.0). Output at
`/home/dank/GitPackages/ROCmFPX-kairic/build-kairic/bin/llama-server`.
- **Composable Kernel**: `fdf4bb7fcc984811cef48ce817d89aac064b984a` +
`patches/composable-kernel-gfx1151-iu4.patch` (SHA-256
`ea57bca3793d35d884fc156b6c0bc33d8e672e3bb49f66ff67338503029d6af5`).
The CK `config.h` is generated by CMake's `configure_file` (line 334 of
CK's CMakeLists.txt) — it is untracked, so a `git clean` removes it.
- **GPU target**: `gfx1151` (Strix Halo). The build takes ~4 minutes
(`JOBS=$(nproc)`).
- **v1.2 changes**: Exact 65-row n-gram verification switched to the compact
authoritative path by default; the native IU4 M65 verifier is retained
only for diagnostics.
## Spec method
- **Native MTP depth 4** (`--spec-type draft-mtp --spec-draft-n-max 4`):
the model's MTP layers draft 4 tokens per step.
- **N-gram drafting** (24/64/64): the `ngram-mod` matcher chains behind
the MTP drafter (the `--spec-type draft-mtp` value builds both).
- **DFlash is NOT available**: the Kairic Edge runtime's spec-type
constants are `DRAFT_EAGLE`, `DRAFT_MTP`, `DRAFT_SIMPLE`,
`NGRAM_CACHE`, `NGRAM_MAP_K`, `NGRAM_MOD`, `NGRAM_SIMPLE`, `NONE` — no
`DRAFT_DFLASH`. The z-lab DFlash2-FP8 drafter (frank-medium's drafter)
cannot be used.
- **Draft device**: `--spec-draft-device ROCm0 --spec-draft-ngl 999`
(the drafter runs on the iGPU, all layers).
## Compatibility mode
The Kairic Edge runtime has two modes:
- **Fast-greedy** (`KAIRIC_EDGE_COMPATIBILITY_MODE=0`, the runner's
default): rejects request features that require full target logits,
including sampling, penalties, probabilities, grammar-constrained tool
calls, logit bias, LoRA, and a reasoning budget.
- **Compatibility** (`KAIRIC_EDGE_COMPATIBILITY_MODE=1`): enables all of
the above (9.7% slower per the build guide).
The harness (OMP) needs sampling, tools and a reasoning budget, so
`COMPATIBILITY_MODE=1` is the deployed shape. In compatibility mode:
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0` and
`PROMPTFORGE_OUTPUT_K8_STRICT_GREEDY=0`.
**Known gotcha**: the greedy-argmax fastpath (fast mode) rejects
non-greedy requests and reasoning budgets. Re-test chat before production
if you flip to fast mode.
## PromptForge sidecars
The three `.pfs` sidecars are wired via the `PROMPTFORGE_*` env block
(the certified runner's exact env):
| sidecar | env var | size |
|---|---|---|
| FFN | `PROMPTFORGE_IU4_SIDECAR` / `PROMPTFORGE_SIDECAR` | 7.99 GiB |
| GDN | `PROMPTFORGE_GDN_SIDECAR` | 1.88 GiB |
| GDN-Output | `PROMPTFORGE_GDN_OUTPUT_SIDECAR` | 0.70 GiB |
The serve script (`scripts/serve-frank-large.sh`) validates binary → main
GGUF → 3 sidecars and exits with a pointer message if any is missing, then
`exec`s `llama-server` with `--jinja --alias frank-large
--chat-template-kwargs '{"preserve_thinking":true}' --chat-template-file
$TEMPLATE_FILE` (Qwen3.8's template needs `preserve_thinking` to carry
thinking blocks across turns; a request-level value still wins).
## Chat template
The serve script passes `--chat-template-file` pointing at
`dankserver/frank-medium/qwen-sharp-v22.4.0.jinja` (the Qwen-Sharp
v22.4.0 template, the same one frank-medium uses). Without this flag the
binary falls back to the built-in `peg-native` template, which produces a
~73,600-token prompt for a ~250-token request (280x the expected size).
With the sharp template wired in, the same request produces a 62–175-token
prompt. The `Chat format: peg-native` log line is the format-type label,
not the template indicator — the sharp template is active even when that
line appears.
## Memory budget
The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR,
`gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB
"dedicated" is not the budget). The 27B dense model (26 GiB total) is
mmap'd in with `-ngl 999`; budget at 262k ≈ 30 GiB (weights + KV +
buffers), leaving ~84 GiB of aperture headroom. The 8 GiB prompt-cache
budget (`CACHE_RAM=8192`) is the Kairic Edge qualified config. The model
is 27B dense (not DeltaNet), so the fleet's 12.8 GiB DeltaNet `CACHE_RAM`
floor does not apply.
## Payload
`scripts/download-frank-large-kairic.sh` (resumable, each file verified
against its declared byte count + sha256; `--verify-only` re-checks):
| file | size | sha256 |
|---|---|---|
| `Qwen3.8-27B-IU4-Kairic-Edge.gguf` | 16,617,792,672 | `360caf73…` |
| `Qwen3.8-27B-Kairic-IU4-FFN.pfs` | 8,576,856,064 | `adcbb90a…` |
| `Qwen3.8-27B-Kairic-IU4-GDN.pfs` | 2,019,569,664 | `82f93131…` |
| `Qwen3.8-27B-Kairic-IU4-GDN-Output.pfs` | 756,953,088 | `3b07e7b1…` |
## Client contract
- **Port**: 7770 (the same port and alias the unsloth predecessor served).
- **Alias**: `frank-large`.
- **Context**: 262,144 (the model's native window), 1 slot.
- **Reasoning**: enabled (`--reasoning auto --reasoning-format deepseek
--reasoning-budget -1`), extracted to `reasoning_content`.
- **Vision**: text-only (the model has no vision projector). The fleet's
image path is [[frank-small-gemma4-12b]] either way.
- **Reasoning effort**: the Qwen3.8 template gates `reasoning_effort` to
{xhigh, medium, low} — the harness folds its levels onto that set in
`models.yml` (`reasoningEffortMap`).
## Measured performance
10-question conversation (2026-09-21, post-deploy): **23.99s total, 530
tokens, 22.1 t/s average**. Per-question: 1.05–3.57s, 16.9–29.5 t/s. The
MTP depth 4 + n-gram spec method is working (draft acceptance logged, both
`ngram-mod` and `draft-mtp` active in the journal).
**Template verification** (2026-09-21): a 250-token request produces a
62–175-token prompt (not the 73,600-token prompt the built-in
`peg-native` template produces). Draft acceptance 0.79 (mean 4.14), the
model answers correctly, and reasoning lands in `reasoning_content`.
## Failure modes worth knowing
- **CK `config.h` is untracked**: a `git clean` in the CK checkout removes
it, and the next build fails. Re-run the build script (it re-checks-out
the CK commit and re-applies the patch).
- **Fast-greedy mode rejects the harness**: if you flip to
`COMPATIBILITY_MODE=0`, the harness's sampling/tools/reasoning-budget
requests are rejected. Re-test chat before production.
- **The `charlie12345` fork is NOT the Kairic Edge repo**: the local
clone at `/home/dank/GitPackages/ROCmFPX` is the `charlie12345/ROCmFPX`
fork (no kairic branch). The Kairic Edge build uses the `ciru` remote
(`https://github.com/ciru-ai/ROCmFPX.git`), branch
`kairic-edge-qwen38-27b-v1.2`, commit `205a3e5`, in the worktree at
`/home/dank/GitPackages/ROCmFPX-kairic/`.
- **Runtime:** Kairic Edge (ciru-ai/ROCmFPX, branch
`kairic-edge-qwen38-27b-v1.2`, commit 205a3e5, TheRock 7.15.0,
Composable Kernel), root systemd unit, port 7770.
- **Model:** `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` (a 27B dense model, IU4
W4A4) — main GGUF + FFN/GDN/GDN-Output `.pfs` sidecars.
- **Shape:** one 262,144-token conversation (single stream), native MTP
depth 4 + n-gram drafting, compatibility mode.
- **Memory:** the 114 GiB GTT aperture on the Strix Halo iGPU.
## Rollback
[[frank-large-unsloth-qwen38-flash-next]] — the unsloth prebuilt payload
and runtime are intact on disk.
Payload and runtime intact on disk. See [[frank-large-unsloth-qwen38-27b-q8xl]]
for the current ACTIVE runbook and [[frank-large-unsloth-qwen38-flash-next]]
for the predecessor.
# Citations
- Kairic Edge runtime: https://github.com/ciru-ai/ROCmFPX
- Model: https://huggingface.co/jcbtc/Qwen3.8-27B-IU4-Kairic-Edge