serverAppConfigs/serverAppConfigs-docs/LLM/ciru-activefpx-promptforge.md

---
type: reference
title: CIRU ActiveFPX PromptForge on Dankclaw
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-large service.
tags:
  - llama.cpp
  - rocm
  - dankclaw
  - gfx1151
  - ciru
  - promptforge
timestamp: 2026-08-22
---

# CIRU ActiveFPX PromptForge on Dankclaw

> **Active 2026-08-22.** `frank-default-large` serves this v3 model again
> after a same-day Kairic Edge IU4 experiment was reverted (operator
> decision). Commit `c60156f` restored the v3 script pair. See
> [[kairic-edge]] for the retired experiment reference.

`frank-default-large` (port 7770, moved from 52625 on 2026-08-22) serves
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.

## What This Is

The model is a **GGUF**, not a vLLM checkpoint. v3 is the consolidated
quality-and-speed release of Qwen3.8-27B for AMD ROCm: an FFN-equalized
compact body, 50 retained late-writer Q6 tensors, full-Q8 output
projection, full-W8 PromptForge view (FFN + Gated DeltaNet), the
small-row W8 route, a draft-only Output-K8 shortlist proxy with Q8
reranking, native depth-4 MTP (no ngram), and the BF16 Qwen3.8 vision
tower + 5120-wide merger as a separate projector.

| Artifact | Size |
|----------|------|
| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf` | 15.44 GiB |
| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-FFN.pfs` | 15.95 GiB |
| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-GDN.pfs` | 3.75 GiB |
| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-Output-K8.pfs` | 0.67 GiB |
| `Qwen3.8-27B-mmproj-BF16.mmproj` | 0.87 GiB |
| Full v3 payload (GGUF + 3 .pfs + mmproj) | ~36.7 GiB |

The `.pfs` companions and the Output-K8 proxy are optional. Omit them to
run the GGUF alone and save ~20.6 GiB of memory. The GGUF remains
independently usable.

## Requirements

- **ROCm 7.15 is required** for the advertised Strix Halo performance.
  The system ROCm (`/opt/rocm-7.2.4`) selects slower or fallback kernels.
- The pinned runtime is `ciru-ai/ROCmFPX` tag
  `qwen3.8-activefpx-promptforge-v2.3` (commit `103af8b41`) **plus the
  v3 runtime patch** `runtime/qwen38-v3-output-k8-runtime.patch`. On
  Dankclaw the patch is commit `af0eac6` on top of the v2.3 checkout.
- Composable Kernel at `fdf4bb7fcc984811cef48ce817d89aac064b984a`.
- Validated GPU: AMD Radeon 8060S / gfx1151.

## ROCm 7.15 Toolchain

The retired `ling3-ciru` package left a complete TheRock ROCm 7.15 SDK
(`7.15.0a20260728`) inside its venv:

```
/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/
  _rocm_sdk_devel/      # hipcc, amdclang 23, cmake, headers
  _rocm_sdk_libraries/  # hipblas, hipblaslt, rocroller
  _rocm_sdk_core/       # libamdhip64, libhsa-runtime64
  rocm-sdk-device-gfx1151
```

The compiler is amdclang 23. `rocm-sdk version` reports `7.15.0a20260728`.
The binary is built against and run with these 7.15 libraries.

## Build Runbook (2026-08-18, v2.3; +v3 patch on 2026-08-21)

Step 1-4 below built v2.3 on 2026-08-18. On 2026-08-21 the v3
Output-K8 runtime patch was applied on top and the server rebuilt:

0. Apply the v3 runtime patch to the v2.3 checkout and rebuild:
   ```
   % verify the patch hash: 8ba7d721ea380b060aeb0c7595b8b05f1582c0f7bf7a4b89e0910bc8e72465d5
   git am runtime/qwen38-v3-output-k8-runtime.patch   # -> commit af0eac6
   cmake --build build-promptforge --target llama-server -j$(nproc)
   ```
   The pre-built binary is kept as `build-promptforge/bin/llama-server.v2.3-bak`
   for rollback.

1. Install host build tools (ninja-build was missing): `sudo apt install ninja-build`.
2. Clone the runtime and CK at their pinned revisions:
   ```
   git clone --depth 1 --branch qwen3.8-activefpx-promptforge-v2.3 \
     https://github.com/ciru-ai/ROCmFPX.git
   git clone --depth 1 https://github.com/ROCm/composable_kernel.git
   git -C composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a
   ```
3. CK headers are generated at CK configure time (`ck/config.h`,
   `ck/version.h`). The llama.cpp build includes the CK source tree, so run
   CK's configure once and copy the headers into `include/ck/`:
   ```
   cmake -S composable_kernel -B build-ck -DAMDGPU_TARGETS=gfx1151 -DCK_ENABLE_WMMA=ON
   cp build-ck/include/ck/config.h build-ck/include/ck/version.h composable_kernel/include/ck/
   ```
4. Configure the llama.cpp build against the venv ROCm 7.15 SDK:
   ```
   export ROCM_PATH=$SDK/_rocm_sdk_devel
   cmake -S ROCmFPX -B build-promptforge -G Ninja \
     -DCMAKE_BUILD_TYPE=Release \
     -DCMAKE_C_COMPILER=$ROCM_PATH/llvm/bin/clang \
     -DCMAKE_CXX_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
     -DCMAKE_HIP_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
     -DCMAKE_PREFIX_PATH=$ROCM_PATH \
     -DGGML_HIP=ON -DGGML_CUDA=OFF -DGGML_VULKAN=OFF \
     -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \
     -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \
     -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \
     -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \
     -DPROMPTFORGE_CK_ROOT=$CK_ROOT \
     -DLLAMA_BUILD_SERVER=ON \
     -DLLAMA_BUILD_WEBUI=OFF
   cmake --build build-promptforge --target llama-server -j$(nproc)
   ```
   `LLAMA_BUILD_WEBUI=OFF` is required. The embedded browser UI downloads
   assets from an HF bucket (`llama-ui`) that returns 404; the service is
   consumed through the OpenAI-compatible API, so the UI is not needed.

Build output: `build-promptforge/bin/llama-server`.

```mermaid
graph TD
    A[Clone ROCmFPX v2.3 + CK pinned] --> B[Configure CK, copy generated headers]
    B --> C[Configure llama-server vs ROCm 7.15 SDK]
    C --> D[Build llama-server]
    D --> E[Verify --list-devices shows gfx1151]
    E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj]
    F --> G[Deploy frank-default-large]
    G --> H[Health check + chat completion]
```

## Model Download

With the HF token to the v3 staging directory
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. The v3 files live there
so the v2 artifacts in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` stay
intact for rollback. Download ran on Dankclaw with the background script
`/tmp/download-v3.sh`; all five files + the patch passed SHA-256 checks
against the model card before the service restart.

## Service

- Unit: `frank-default-large.service`, port 7770
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env`
- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
  the system ROCm. `ROCM_PATH` points at the devel SDK.

### Launch profile

- 262144 context (native max), 1 slot (user request 2026-08-19; parallel
  slots degraded performance), f16 KV (~16 GiB), context checkpoints (16),
  prompt cache. Concurrent requests queue on the single slot; the server
  never preempts a running generation
- Native MTP draft, depth 4, device ROCm0
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
  puts `<think>` reasoning in `message.reasoning_content`)
- v3 runtime: Output-K8 draft proxy (top-8 shortlist, Q8 rerank) +
  small-row W8 prompt route, full-W8 FFN/GDN views
- PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`,
  `GGML_CUDA_GRAPH_OPT=0`, `PROMPTFORGE_MTP_OUTPUT_K8=1`,
  `PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0`,
  `PROMPTFORGE_ENABLE_SMALLM_W8=1`

### Required environment

```
export PROMPTFORGE_SIDECAR=...FFN.pfs
export PROMPTFORGE_GDN_SIDECAR=...GDN.pfs
export PROMPTFORGE_MODE=m2048_fused_tail1476
export GGML_CUDA_GRAPH_OPT=0
export PROMPTFORGE_ENABLE_SMALLM_W8=1
export PROMPTFORGE_MTP_OUTPUT_K8=1
export PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0
export PROMPTFORGE_MTP_OUTPUT_K8_PROXY=...Output-K8.pfs
# v3 qualified profile: disable the CPU argmax draft fast path and any
# IU4/ngram lane. The Output-K8 proxy + backend sampling replace them.
unset LLAMA_MTP_CPU_ARGMAX_FASTPATH LLAMA_MTP_BACKEND_GREEDY
unset PROMPTFORGE_IU4_SIDECAR PROMPTFORGE_GDN_IU4_SIDECAR
unset PROMPTFORGE_ENABLE_NGRAM_MOD PROMPTFORGE_ENABLE_NGRAM_M65_IU4
```

The v3 qualified profile runs the MTP draft through the Output-K8 proxy,
not the CPU argmax fast path. The old `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1`
profile is retired for this slot. A TOP_K sampler warning may still appear
in the journal once per start; the Output-K8 draft path does not depend on
it and generation proceeds.

## Operational Notes

- Health: `curl http://127.0.0.1:7770/health`. Returns 503 while loading.
- Model load is slow (v3 five-file payload ~36.7 GiB); do not kill the
  process during warmup. The page cache makes a repeat load fast right after
  download, but a later cold start takes several minutes.
- `deploy.sh dankclaw` waits 20 minutes for health.
- The systemd unit must not set `HOST` or `TEMP` env vars (HIP collision
  rule). `--host`/`--port`/`--temp` are CLI flags only.
- v3 adds the BF16 vision tower over v2. The service reports `multimodal`
  capability; mmproj loads with the language model (CLIP on ROCm0 backend).

## ngram-mod + MTP Stacking (benchmarked 2026-08-21)

This page records a research question: does ngram-mod add value when
stacked under the native MTP draft? The v3 profile runs MTP alone; the
benchmark below compares that against the stacked pair.

### Feasibility

ngram-mod DOES stack with MTP in this CIRU fork. The v3 build compiles in
both `ngram-mod` and `draft-mtp` (`common/speculative.cpp`). Pass
`--spec-type draft-mtp,ngram-mod` (or `ngram-mod,draft-mtp`) and the fork
initializes both implementations in the same process. The boot log shows:

```
adding speculative implementation 'ngram-mod'
initialized ngram_mod with n_match=24, size=4194304 (16.000 MB)
adding speculative implementation 'draft-mtp'
```

The model card's "no ngram subsystem" line refers to the CIRU PromptForge
env lanes (`PROMPTFORGE_*_NGRAM_*`, `PROMPTFORGE_ENABLE_NGRAM_MOD`), not
the upstream llama.cpp spec chain. Those env lanes must stay unset; the
`--spec-type ngram-mod` chain is separate and works.

### Ordering — the CLI order does not matter

The `--spec-type` string order is discarded. `common_speculative_init`
collapses the requested types into a bitmask, then rebuilds the chain in a
hard-coded priority: ngram types first, then draft-simple, then MTP.
So `draft-mtp,ngram-mod` and `ngram-mod,draft-mtp` produce the same
internal order (`ngram-mod` then `draft-mtp`) and the same runtimes.
Verified: both orders decode ~25.3 t/s at 8k.

### Results (decode generation t/s, 200 tokens, prose prompt)

| ctx | MTP only | MTP + ngram-mod | verdict |
|-----|----------|-----------------|---------|
| 8k | 29.42 | 25.27 | ngram hurts -4.15 |
| 16k | 26.42 | 21.13 | ngram hurts -5.29 |
| 64k | 19.72 | 18.00 | ngram hurts -1.72 |

At every depth, adding ngram-mod under MTP REDUCED decode throughput. The
ngram draft costs its own forward pass / occupancy without adding enough
accepted tokens on top of MTP. Baseline (no ngram) remains the production
config (`--spec-type draft-mtp` only). Keep ngram-mod off.

The historical harness was moved to `.AGENTTRASH/bench-scripts-20260901/` and is retained for audit only.
it boots the v3 stack on a scratch port and reads authoritative
`print_timing` lines. It requires the production service to be stopped.

## See Also

- [[frank-redo-2026-08-17]] — the fleet topologies this slot has served.
- [[Shared Memory Architecture Dankclaw]] — unified memory sizing.
- [[qwen3.8-strix-optimizations]] — llama.cpp ROCm0 baselines on this iGPU.
- [[VLLM Overview]] — vLLM reference (retired for this slot).