From 43605519384c2115b0f530eee1d94405c49f9a2a Mon Sep 17 00:00:00 2001 From: Frank Date: Sat, 22 Aug 2026 02:51:39 -0700 Subject: [PATCH] docs: Kairic Edge reference page (build runbook, config.h gap, byte-identical rebuild, perf data); mark v3 doc retired; update topology --- AGENTS.md | 72 ++++--- docs/ciru-activefpx-promptforge.md | 6 +- docs/index.md | 22 ++- docs/kairic-edge.md | 296 +++++++++++++++++++++++++++++ 4 files changed, 356 insertions(+), 40 deletions(-) create mode 100644 docs/kairic-edge.md diff --git a/AGENTS.md b/AGENTS.md index 9f1da17..3de5309 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -3,20 +3,24 @@ ## Overview This project manages deployment scripts for the LLM inference fleet on two -local machines. Topology (2026-08-21): +local machines. Topology (2026-08-22): | Target | Host | Runtime | Service | Model | Port | |--------|------|---------|---------|-------|------| | Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 | -| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3, native MTP + Output-K8 proxy, PromptForge DualView, 262K context / 1 slot | 52625 | +| Dankclaw | dank@dankclaw | Kairic Edge CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-IU4-Kairic-Edge, native IU4 lane + MTP4 + ngram-mod, 262K context / 1 slot | 52625 | The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM -NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw; the -CIRU ROCmFPX llama.cpp runtime runs on the Strix Halo iGPU (gfx1151). On -2026-08-21 a DFlash2 Vulkan deployment briefly replaced CIRU, then was -reverted the same day for speed; the DFlash2 files live in `.AGENTTRASH/`. -See docs/ciru-activefpx-promptforge.md. The old scripts live in -`old/scripts/` for reference; they describe the retired PP topology. +NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw; +the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on +the Strix Halo iGPU (gfx1151). On 2026-08-22 the service swapped to the +Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`); v3 +files remain on the machine for rollback. On 2026-08-21 a DFlash2 Vulkan +deployment briefly replaced CIRU, then was reverted the same day for +speed; the DFlash2 files live in `.AGENTTRASH/`. See +docs/kairic-edge.md and docs/ciru-activefpx-promptforge.md. The old scripts +live in `old/scripts/` for reference; they describe the retired PP +topology. **Sibling project:** [../serverAppConfigs](../serverAppConfigs) manages the Podman app stack on the same hosts (OpenWebUI, Gitea, observability). Its @@ -124,30 +128,38 @@ back to a slow path and prefill collapses ~12x (measured 2026-08-17: 95 t/s vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README for the base recipe. -### CIRU ROCmFPX llama.cpp (Dankclaw) +### Kairic Edge llama.cpp (Dankclaw) -The CIRU llama-server serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` -from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`. -The runtime is the ciru-ai/ROCmFPX tag `qwen3.8-activefpx-promptforge-v2.3` -(commit `103af8b41`) **plus the v3 output-K8 runtime patch** (commit -`af0eac6` on Dankclaw) built against the private TheRock ROCm 7.15 SDK kept -in the retired ling3-ciru venv -(`.venv/lib/python3.12/site-packages/_rocm_sdk_*`). The model card requires -ROCm 7.15 for the advertised Strix Halo performance; the system ROCm 7.2.4 -selects slower kernels. The serve script exports the 7.15 SDK libraries -first in `LD_LIBRARY_PATH`. +The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` +from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server` +(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`). +The runtime routes qualified prompt and verification shapes through the +RDNA 3.5 native IU4 matrix instruction. The build uses the release script +against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru +venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in +`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. -The v3 model files (GGUF + FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj, -~36.7 GiB) live in the v3 staging directory -`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`; the v2 artifacts remain -in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` for rollback. -The full build runbook is in docs/ciru-activefpx-promptforge.md. The retired vLLM venvs -(`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to -`/home/dank/.AGENTTRASH/` on 2026-08-18. +**Release gap — CK `config.h`:** the pinned CK commit ships only +`config.h.in`; the release script and guide never generate +`third_party/composable_kernel/include/ck/config.h`. Generate it via a CK +CMake configure and copy it in before building (full runbook in +docs/kairic-edge.md). The file is untracked in CK git; a `git clean` +removes it. Local rebuilds are byte-identical (`d7625a0c...`). -**Env var naming rule still applies:** the HIP runtime reads `TEMP`, `HOST`, -and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`/`--temp` -as CLI flags only. +**Target greedy argmax fast path:** the pinned runner sets +`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`; production sets it to 0 because +the fast path rejects non-greedy requests (400) and the card's own chat +sampling is non-greedy. Do not flip it back without re-testing chat. + +**Rollback:** the v3 model files remain in +`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in +`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`. +The v3 runbook is docs/ciru-activefpx-promptforge.md; the rollback +procedure is in docs/kairic-edge.md. + +**Env var naming rule still applies:** the HIP runtime reads `TEMP`, +`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port` +as CLI flags only; bind and port come from `BIND_ADDR`/`LISTEN_PORT`. ### DFlash2 llama.cpp (Dankclaw, retired 2026-08-21) @@ -198,7 +210,7 @@ graph TD - `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '' | sudo -S `. - Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x `. -- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw CIRU loads the v3 ~36.7 GiB payload in ~5-10 minutes. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`). +- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw Kairic loads the 26.05 GiB payload in a few minutes (page-cache warm). Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`). - A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md). - Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md). diff --git a/docs/ciru-activefpx-promptforge.md b/docs/ciru-activefpx-promptforge.md index 71ff578..801662b 100644 --- a/docs/ciru-activefpx-promptforge.md +++ b/docs/ciru-activefpx-promptforge.md @@ -14,7 +14,11 @@ timestamp: 2026-08-21 # CIRU ActiveFPX PromptForge on Dankclaw -`frank-default-large` (port 52625) serves +> **Retired 2026-08-22.** `frank-default-large` now serves the Kairic Edge +> IU4 model. See [[kairic-edge]] for the active deployment and the rollback +> procedure. This page is kept as the v3 runbook and rollback reference. + +`frank-default-large` (port 52625) served `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21. diff --git a/docs/index.md b/docs/index.md index f03b0fc..a942c91 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,8 +1,8 @@ # vLLM Scripts — Knowledge Base Documentation for the LLM inference fleet on two local machines -(2026-08-21): upstream llama.cpp on Dankserver (frank-default-small) and -CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large). +(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and +Kairic Edge CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large). ## Deployment Topology @@ -19,10 +19,10 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx] subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] D[gfx1151 iGPU 64 GiB unified memory] - E[CIRU ROCmFPX llama.cpp + E[Kairic Edge CIRU ROCmFPX llama.cpp frank-default-large, port 52625] - F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge -262K ctx / 1 slot, native MTP, PromptForge DualView] + F[jcbtc Qwen3.8-27B-IU4-Kairic-Edge +262K ctx / 1 slot, native IU4 lane, MTP4 + ngram-mod] D --> E --> F end @@ -41,15 +41,19 @@ frank-default-large, port 52625] - [[deployment]] — deploy and update process for both servers - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change -- [[ciru-activefpx-promptforge]] — the CIRU runtime on Dankclaw - (frank-default-large) +- [[kairic-edge]] — the Kairic Edge IU4 runtime on Dankclaw + (frank-default-large, active since 2026-08-22) +- [[ciru-activefpx-promptforge]] — the retired v3 CIRU runtime + (frank-default-large rollback target) - [[Dankserver]] — the upstream llama.cpp box (frank-default-small) - [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large) ## CIRU Reference (Dankclaw) -- [[ciru-activefpx-promptforge]] — model, ROCm 7.15 toolchain, build, - and the frank-default-large service +- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary + identity, serving profile, rollback (active deployment) +- [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain, + and build history (retired 2026-08-22, rollback target) ## DFlash2 Reference (archived) diff --git a/docs/kairic-edge.md b/docs/kairic-edge.md new file mode 100644 index 0000000..ff8b142 --- /dev/null +++ b/docs/kairic-edge.md @@ -0,0 +1,296 @@ +--- +type: reference +title: Kairic Edge IU4 Runtime on Dankclaw +description: jcbtc/Qwen3.8-27B-IU4-Kairic-Edge served by the Kairic Edge CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Build runbook, binary identity, serving profile, and rollback. +tags: + - llama.cpp + - rocm + - dankclaw + - gfx1151 + - kairic + - iu4 + - promptforge +timestamp: 2026-08-22 +--- + +# Kairic Edge IU4 Runtime on Dankclaw + +`frank-default-large` (port 52625) serves +`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX +llama.cpp runtime. The runtime routes qualified prompt and multi-token +verification shapes through the RDNA 3.5 native +`V_WMMA_I32_16X16X16_IU4` matrix instruction. This deployment replaced the +v3 CIRU ActiveFPX PromptForge service on 2026-08-22. See +[[ciru-activefpx-promptforge]] for the retired v3 runbook and rollback +target. + +## Identity + +The release pins every input. Verify these values before trusting a build. + +| Input | Pinned value | +|-------|--------------| +| Runtime repo | ciru-ai/ROCmFPX tag `kairic-edge-qwen38-27b-v1`, commit `e97b324` | +| Composable Kernel | ROCm/composable_kernel `fdf4bb7fcc984811cef48ce817d89aac064b984a` | +| CK release patch | `patches/composable-kernel-gfx1151-iu4.patch`, SHA-256 `ea57bca3793d35d884fc156b6c0bc33d8e672e3bb49f66ff67338503029d6af5` | +| Certified toolchain | TheRock 7.15.0a20260718, AMD clang 23.0.0, GCC 13.3.0, CMake 4.4.2, Ninja 1.13.0 | +| Local toolchain | TheRock 7.15 SDK (ling3-ciru venv), amdclang++ 23.0.0git, GCC 15.2.0, CMake 4.2.3, Ninja 1.13.2 | +| Frozen certified binary | `llama-server` SHA-256 `0ece51fa4489e58b7c6942ec2aa59baadca897d8cf87f8e1dc4dc0b10a6efc49` | +| Local binary | `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`, SHA-256 `d7625a0c14ac4158fd793c50ac0cefd8cdcbf76b5aca5cddafa09a353dc8f048` | + +The guide states that a portable rebuild does not match the frozen +certified binary byte for byte. The local toolchain differs from the +certified one. The local binary reproduces byte-identical across local +rebuilds (see [Build reproducibility](#build-reproducibility)). + +## Model artifacts + +Download all four files into one directory. The three `.pfs` sidecars are +accelerator-specific prepacked weight views. The runtime requires all of +them. + +| File | Size | SHA-256 | +|------|------|---------| +| `Qwen3.8-27B-IU4-Kairic-Edge.gguf` | 15.48 GiB | `360caf7381907c3eca7ac0afd1228efc016af747f3f38637fb1c7f94daabac2a` | +| `Qwen3.8-27B-Kairic-IU4-FFN.pfs` | 7.99 GiB | `adcbb90a7b429a30a2a39043366d68320d72e8b4816a0f498e882b2f80a2ba2b` | +| `Qwen3.8-27B-Kairic-IU4-GDN.pfs` | 1.88 GiB | `82f931316f1c895da104915dec4697163808d06f0e6b2dc027cee7aa3afc0f0e` | +| `Qwen3.8-27B-Kairic-IU4-GDN-Output.pfs` | 0.70 GiB | `3b07e7b176559e4402924ba0c368532fa6f02118a33c71e70974c809bf6208a3` | + +Total payload: 26.05 GiB. Local directory: +`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. All four SHA-256 values +verified on 2026-08-22. + +## Build runbook + +```mermaid +graph TD + A[apt build deps] --> B[Clone ROCmFPX at pinned tag] + B --> C[Clone CK at pinned commit] + C --> D[Apply and verify CK patch] + D --> E[Generate ck/config.h: release gap] + E --> F[Run release build script] + F --> G[Build verification commands] + G --> H[Download and verify model] + H --> I[Smoke test] +``` + +### 1. Install build dependencies + +``` +sudo apt-get update +sudo apt-get install -y build-essential cmake git ninja-build pkg-config libssl-dev +``` + +### 2. Clone the immutable release + +The local checkout is a git worktree of the pinned tag at +`/home/dank/ciru-qwen38/ROCmFPX-kairic`. + +``` +git clone https://github.com/ciru-ai/ROCmFPX.git +cd ROCmFPX +git checkout kairic-edge-qwen38-27b-v1 +git status --short # must print nothing +``` + +### 3. Prepare Composable Kernel + +``` +git clone https://github.com/ROCm/composable_kernel.git third_party/composable_kernel +git -C third_party/composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a +git -C third_party/composable_kernel apply ../../patches/composable-kernel-gfx1151-iu4.patch +sha256sum patches/composable-kernel-gfx1151-iu4.patch # ea57bca3793d...af5 +git -C third_party/composable_kernel diff --check +``` + +### 4. Generate `ck/config.h` (release gap) + +The pinned CK commit ships only `include/ck/config.h.in` (CMake template). +The release script and the guide never generate `include/ck/config.h`. A +clean build fails at the first CK include without it. Generate it with CK's +own CMake configure, then copy it into the CK source include tree: + +``` +cmake -S third_party/composable_kernel -B third_party/composable_kernel/build-ck \ + -G Ninja -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ \ + -DCMAKE_HIP_COMPILER="$ROCM_PATH/bin/amdclang++" \ + -DCMAKE_PREFIX_PATH="$ROCM_PATH" +cp third_party/composable_kernel/build-ck/include/ck/config.h \ + third_party/composable_kernel/include/ck/config.h +``` + +Notes: + +- The CK configure can fail after `config.h` generation. The generated file + is still valid. Check `build-ck/include/ck/config.h` exists. +- The `CK_USE_WMMA` flag inside `config.h` gates only CK instance-selection + code. The Kairic patched CK headers do not reference it. Local rebuilds + with the flag ON and OFF produce byte-identical binaries. +- The file is untracked in CK git. A `git clean` in the CK tree removes it. + Regenerate it before any rebuild. +- A full CK configure without a GPU target leaves `CK_USE_WMMA` undefined. + Pass `-DGPU_TARGETS=gfx1151` for the certified-style variant. Both + variants build an identical Kairic binary. + +### 5. Build + +The release script owns the certified CMake options. + +``` +export ROCM_PATH= +JOBS="$(nproc)" scripts/build-kairic-edge-gfx1151.sh +``` + +Local `ROCM_PATH` is the split SDK layout inside the retired ling3-ciru +venv: +`/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_devel`. +The serve script assembles `LD_LIBRARY_PATH` from the three split +directories (`_rocm_sdk_devel`, `_rocm_sdk_libraries`, `_rocm_sdk_core`) so +the 7.15 libraries precede the system ROCm 7.2.4. + +### 6. Build verification + +``` +./build-kairic/bin/llama-server --version +./build-kairic/bin/llama-server --help | grep -A1 -- '--kairic-edge' +ldd ./build-kairic/bin/llama-server | grep -E 'not found' # must print nothing +``` + +Expected local output: `version: 79 (e97b324)`, +`built with GNU 15.2.0 for Linux x86_64`. + +### 7. Download and verify model artifacts + +Download the four files from +[jcbtc/Qwen3.8-27B-IU4-Kairic-Edge](https://huggingface.co/jcbtc/Qwen3.8-27B-IU4-Kairic-Edge) +and verify the SHA-256 values in the table above. + +### 8. Smoke test + +Run the guide's smoke test: health, `/v1/models`, two identical +`cache_prompt` completion requests, warm request must report restored +prompt tokens. Inspect the server log for route fallback, swap activity, +and HIP errors. Confirm the IU4 lane at boot: + +``` +{"record":"promptforge_init","mode":"iu4_ffn","wmma":"v_wmma_i32_16x16x16_iu4",...} +{"record":"promptforge_gdn_init","projection":"qkvz_iu4_hadamard",...} +``` + +## Build reproducibility + +On 2026-08-22 a fresh rebuild followed the documented procedure from a +clean CK tree (pinned commit, patch re-applied, `config.h` regenerated via +CK configure). The rebuild produced a byte-identical binary: + +| Build | SHA-256 | +|-------|---------| +| Original (first deployment) | `d7625a0c14ac4158fd793c50ac0cefd8cdcbf76b5aca5cddafa09a353dc8f048` | +| Fresh rebuild (clean CK tree) | `d7625a0c14ac4158fd793c50ac0cefd8cdcbf76b5aca5cddafa09a353dc8f048` | + +The two builds used different `config.h` variants (differ only in +`/* #undef */` comment lines and the `CK_USE_WMMA` state). The identical +output proves that variant difference has no effect on this build. The +original manual `config.h` step is a neutral, minimal bridge over the +release gap. + +The original build tree stays at +`/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic-orig/` until the next +service restart confirms the fresh tree, then it is a deletion candidate +(move to `.AGENTTRASH`). + +## Serving profile + +The service runs the pinned runner profile plus the local chat +deviations below. Tunables live in +`scripts/frank-default-large.env`; flags in `scripts/serve-frank-large.sh`. +See [[deployment]] for the update process. + +| Setting | Value | Origin | +|---------|-------|--------| +| Bind | `0.0.0.0:52625` | Local (runner default `127.0.0.1:8080`) | +| Context | 262144, 1 slot | Pinned runner | +| Batch / ubatch | 4096 / 2048 | Local (runner: 2048 / 512) | +| Threads | 16 target / 32 batch | Pinned runner | +| KV | f16 target and draft | Pinned runner | +| Prompt cache | `--cache-ram 8192 --cache-prompt --cache-idle-slots -ctxcp 32` | Pinned runner | +| Kairic Edge | `--kairic-edge` | Pinned runner | +| Speculative | `draft-mtp` depth 4, `p_split 0.10`, backend sampling | Pinned runner | +| Server sampling | temp 0 / top-p 1 / top-k 0 / min-p 0 | Pinned runner | +| Reasoning | on, deepseek format, budget -1, preserve_thinking | Local | +| `LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH` | 0 | Local (runner: 1) | + +Local deviations and their reasons: + +- **Bind/port**: fleet convention; the runner binds localhost only. +- **Reasoning on + deepseek**: thinking goes to `message.reasoning_content`, + matching the retired v3 service. `preserve_thinking=true` keeps thinking + tokens in the history across turns. +- **Batch 4096 / ubatch 2048**: larger than the runner; unified memory has + headroom. Prefill and decode match the runner profile in benchmarks. +- **Target greedy argmax fast path off**: the pinned runner sets + `LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`. That fast path fails closed on + any non-greedy request (400 error). The model card's own chat + instructions send temp 0.7 / top-p 0.8 / top-k 20 / presence_penalty + 1.5, which the fast path rejects. Set to 0 for production chat on + 2026-08-22. This does not affect the Kairic IU4 lane or the MTP draft + fast path (`LLAMA_MTP_CPU_ARGMAX_FASTPATH=1` stays on). +- **`--kairic-edge` auto-injects ngram-mod** (n_match 24, n_min 64, + n_max 64). This is release design, not a misconfiguration. + +## Performance on this machine + +Benchmarks from 2026-08-22 on scratch port 7893, 512 forced tokens, +greedy, production config unless noted. Stop the service before +benchmarking (see AGENTS.md). + +| Context | Kairic prefill | Kairic decode | v3 prefill | v3 decode (200 tok) | +|---------|----------------|---------------|------------|---------------------| +| ~8k | 506 (runner) / 348 (prod) | 21.0 (runner) / 19.8 (prod) | 217.7 | 29.42 | +| ~16k | 439 (runner) / 370 (prod) | 23.5 (runner) / 19.4 (prod) | 211.2 | 26.42 | +| ~64k | 232.5 (prod) | 14.6 (prod) | 155.7 | 19.72 | + +Findings: + +- Prefill runs 1.5-2.3x faster than v3 at every depth. +- Prose decode runs 15-26% slower than v3 at every depth. +- `--no-mmap` makes no difference (19.89 vs 19.80 t/s decode). +- `-c 262144` versus `-c 8192` makes no difference to decode. +- Live service at ~116k context: 13.5-14.9 t/s, consistent with the 64k + bench. Long conversations run in the deep-context regime. +- MTP draft acceptance on prose: rate 0.31-0.35, mean accepted length + 2.6-3.0. Draft fast path active, zero failures. +- The card's 47.73 t/s aggregate comes from a 164-task coding suite with + deterministic sampling and repeated coding context. It does not + represent prose chat. The card's natural-prose slice measured 34.88 + t/s on the certified system. + +## Rollback to v3 + +The v3 payload and binary stay on the machine for rollback. + +1. Stop the service: `sudo systemctl stop frank-default-large` +2. Restore the v3 scripts: + `git revert 4c43839 7aee1df 40d50fe` (or checkout the script pair from + commit `132e147`) +3. Deploy: `FRANK_SUDO_PW=... ./scripts/deploy.sh dankclaw` +4. Verify health and the `Output-K8` boot records. + +v3 locations: model `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`, +binary `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`. +Full v3 runbook: [[ciru-activefpx-promptforge]]. + +## Citations + +- Model card: +- Build and verification guide: +- Pinned release: +- Pinned runner: `scripts/run-kairic-edge-gfx1151.sh` in the release repo + +## See Also + +- [[ciru-activefpx-promptforge]] — retired v3 runtime, rollback target +- [[deployment]] — deploy and update process +- [[Dankclaw System Specs]] — hardware and memory model +- [[qwen3.8-strix-optimizations]] — Strix Halo optimization history