--- type: reference title: CIRU ActiveFPX PromptForge on Dankclaw description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-dragoon service. tags: - llama.cpp - rocm - dankclaw - gfx1151 - ciru - promptforge timestamp: 2026-08-22 --- # CIRU ActiveFPX PromptForge on Dankclaw > **RETIRED 2026-08-24.** `frank-dragoon` served this v3 model from > 2026-08-22 (restored after a same-day Kairic Edge IU4 experiment was > reverted — operator decision, commit `c60156f`; see [[kairic-edge]]) > until [[eemin-rocmi4-mtp]] took over on 2026-08-24. Kept as the > historical record of the v3 deployment. `frank-dragoon` (port 7770, moved from 52625 on 2026-08-22) serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21. ## What This Is The model is a **GGUF**, not a vLLM checkpoint. v3 is the consolidated quality-and-speed release of Qwen3.8-27B for AMD ROCm: an FFN-equalized compact body, 50 retained late-writer Q6 tensors, full-Q8 output projection, full-W8 PromptForge view (FFN + Gated DeltaNet), the small-row W8 route, a draft-only Output-K8 shortlist proxy with Q8 reranking, native depth-4 MTP (no ngram), and the BF16 Qwen3.8 vision tower + 5120-wide merger as a separate projector. | Artifact | Size | |----------|------| | `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf` | 15.44 GiB | | `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-FFN.pfs` | 15.95 GiB | | `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-GDN.pfs` | 3.75 GiB | | `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-Output-K8.pfs` | 0.67 GiB | | `Qwen3.8-27B-mmproj-BF16.mmproj` | 0.87 GiB | | Full v3 payload (GGUF + 3 .pfs + mmproj) | ~36.7 GiB | The `.pfs` companions and the Output-K8 proxy are optional. Omit them to run the GGUF alone and save ~20.6 GiB of memory. The GGUF remains independently usable. ## Requirements - **ROCm 7.15 is required** for the advertised Strix Halo performance. The system ROCm (`/opt/rocm-7.2.4`) selects slower or fallback kernels. - The pinned runtime is `ciru-ai/ROCmFPX` tag `qwen3.8-activefpx-promptforge-v2.3` (commit `103af8b41`) **plus the v3 runtime patch** `runtime/qwen38-v3-output-k8-runtime.patch`. On Dankclaw the patch is commit `af0eac6` on top of the v2.3 checkout. - Composable Kernel at `fdf4bb7fcc984811cef48ce817d89aac064b984a`. - Validated GPU: AMD Radeon 8060S / gfx1151. ## ROCm 7.15 Toolchain The retired `ling3-ciru` package left a complete TheRock ROCm 7.15 SDK (`7.15.0a20260728`) inside its venv: ``` /home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/ _rocm_sdk_devel/ # hipcc, amdclang 23, cmake, headers _rocm_sdk_libraries/ # hipblas, hipblaslt, rocroller _rocm_sdk_core/ # libamdhip64, libhsa-runtime64 rocm-sdk-device-gfx1151 ``` The compiler is amdclang 23. `rocm-sdk version` reports `7.15.0a20260728`. The binary is built against and run with these 7.15 libraries. ## Build Runbook (2026-08-18, v2.3; +v3 patch on 2026-08-21) Step 1-4 below built v2.3 on 2026-08-18. On 2026-08-21 the v3 Output-K8 runtime patch was applied on top and the server rebuilt: 0. Apply the v3 runtime patch to the v2.3 checkout and rebuild: ``` % verify the patch hash: 8ba7d721ea380b060aeb0c7595b8b05f1582c0f7bf7a4b89e0910bc8e72465d5 git am runtime/qwen38-v3-output-k8-runtime.patch # -> commit af0eac6 cmake --build build-promptforge --target llama-server -j$(nproc) ``` The pre-built binary is kept as `build-promptforge/bin/llama-server.v2.3-bak` for rollback. 1. Install host build tools (ninja-build was missing): `sudo apt install ninja-build`. 2. Clone the runtime and CK at their pinned revisions: ``` git clone --depth 1 --branch qwen3.8-activefpx-promptforge-v2.3 \ https://github.com/ciru-ai/ROCmFPX.git git clone --depth 1 https://github.com/ROCm/composable_kernel.git git -C composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a ``` 3. CK headers are generated at CK configure time (`ck/config.h`, `ck/version.h`). The llama.cpp build includes the CK source tree, so run CK's configure once and copy the headers into `include/ck/`: ``` cmake -S composable_kernel -B build-ck -DAMDGPU_TARGETS=gfx1151 -DCK_ENABLE_WMMA=ON cp build-ck/include/ck/config.h build-ck/include/ck/version.h composable_kernel/include/ck/ ``` 4. Configure the llama.cpp build against the venv ROCm 7.15 SDK: ``` export ROCM_PATH=$SDK/_rocm_sdk_devel cmake -S ROCmFPX -B build-promptforge -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_C_COMPILER=$ROCM_PATH/llvm/bin/clang \ -DCMAKE_CXX_COMPILER=$ROCM_PATH/llvm/bin/clang++ \ -DCMAKE_HIP_COMPILER=$ROCM_PATH/llvm/bin/clang++ \ -DCMAKE_PREFIX_PATH=$ROCM_PATH \ -DGGML_HIP=ON -DGGML_CUDA=OFF -DGGML_VULKAN=OFF \ -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \ -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \ -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \ -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \ -DPROMPTFORGE_CK_ROOT=$CK_ROOT \ -DLLAMA_BUILD_SERVER=ON \ -DLLAMA_BUILD_WEBUI=OFF cmake --build build-promptforge --target llama-server -j$(nproc) ``` `LLAMA_BUILD_WEBUI=OFF` is required. The embedded browser UI downloads assets from an HF bucket (`llama-ui`) that returns 404; the service is consumed through the OpenAI-compatible API, so the UI is not needed. Build output: `build-promptforge/bin/llama-server`. ```mermaid graph TD A[Clone ROCmFPX v2.3 + CK pinned] --> B[Configure CK, copy generated headers] B --> C[Configure llama-server vs ROCm 7.15 SDK] C --> D[Build llama-server] D --> E[Verify --list-devices shows gfx1151] E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj] F --> G[Deploy frank-dragoon] G --> H[Health check + chat completion] ``` ## Model Download With the HF token to the v3 staging directory `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. The v3 files live there so the v2 artifacts in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` stay intact for rollback. Download ran on Dankclaw with the background script `/tmp/download-v3.sh`; all five files + the patch passed SHA-256 checks against the model card before the service restart. ## Service - Unit: `frank-dragoon.service`, port 7770 - Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env` - The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before the system ROCm. `ROCM_PATH` points at the devel SDK. ### Launch profile - 262144 context (native max), 1 slot (user request 2026-08-19; parallel slots degraded performance), f16 KV (~16 GiB), context checkpoints (16), prompt cache. Concurrent requests queue on the single slot; the server never preempts a running generation - Native MTP draft, depth 4, device ROCm0 - `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek` puts `` reasoning in `message.reasoning_content`) - v3 runtime: Output-K8 draft proxy (top-8 shortlist, Q8 rerank) + small-row W8 prompt route, full-W8 FFN/GDN views - PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`, `GGML_CUDA_GRAPH_OPT=0`, `PROMPTFORGE_MTP_OUTPUT_K8=1`, `PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0`, `PROMPTFORGE_ENABLE_SMALLM_W8=1` ### Required environment ``` export PROMPTFORGE_SIDECAR=...FFN.pfs export PROMPTFORGE_GDN_SIDECAR=...GDN.pfs export PROMPTFORGE_MODE=m2048_fused_tail1476 export GGML_CUDA_GRAPH_OPT=0 export PROMPTFORGE_ENABLE_SMALLM_W8=1 export PROMPTFORGE_MTP_OUTPUT_K8=1 export PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0 export PROMPTFORGE_MTP_OUTPUT_K8_PROXY=...Output-K8.pfs # v3 qualified profile: disable the CPU argmax draft fast path and any # IU4/ngram lane. The Output-K8 proxy + backend sampling replace them. unset LLAMA_MTP_CPU_ARGMAX_FASTPATH LLAMA_MTP_BACKEND_GREEDY unset PROMPTFORGE_IU4_SIDECAR PROMPTFORGE_GDN_IU4_SIDECAR unset PROMPTFORGE_ENABLE_NGRAM_MOD PROMPTFORGE_ENABLE_NGRAM_M65_IU4 ``` The v3 qualified profile runs the MTP draft through the Output-K8 proxy, not the CPU argmax fast path. The old `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1` profile is retired for this slot. A TOP_K sampler warning may still appear in the journal once per start; the Output-K8 draft path does not depend on it and generation proceeds. ## Operational Notes - Health: `curl http://127.0.0.1:7770/health`. Returns 503 while loading. - Model load is slow (v3 five-file payload ~36.7 GiB); do not kill the process during warmup. The page cache makes a repeat load fast right after download, but a later cold start takes several minutes. - `deploy.sh dankclaw` waits 20 minutes for health. - The systemd unit must not set `HOST` or `TEMP` env vars (HIP collision rule). `--host`/`--port`/`--temp` are CLI flags only. - v3 adds the BF16 vision tower over v2. The service reports `multimodal` capability; mmproj loads with the language model (CLIP on ROCm0 backend). ## ngram-mod + MTP Stacking (benchmarked 2026-08-21) This page records a research question: does ngram-mod add value when stacked under the native MTP draft? The v3 profile runs MTP alone; the benchmark below compares that against the stacked pair. ### Feasibility ngram-mod DOES stack with MTP in this CIRU fork. The v3 build compiles in both `ngram-mod` and `draft-mtp` (`common/speculative.cpp`). Pass `--spec-type draft-mtp,ngram-mod` (or `ngram-mod,draft-mtp`) and the fork initializes both implementations in the same process. The boot log shows: ``` adding speculative implementation 'ngram-mod' initialized ngram_mod with n_match=24, size=4194304 (16.000 MB) adding speculative implementation 'draft-mtp' ``` The model card's "no ngram subsystem" line refers to the CIRU PromptForge env lanes (`PROMPTFORGE_*_NGRAM_*`, `PROMPTFORGE_ENABLE_NGRAM_MOD`), not the upstream llama.cpp spec chain. Those env lanes must stay unset; the `--spec-type ngram-mod` chain is separate and works. ### Ordering — the CLI order does not matter The `--spec-type` string order is discarded. `common_speculative_init` collapses the requested types into a bitmask, then rebuilds the chain in a hard-coded priority: ngram types first, then draft-simple, then MTP. So `draft-mtp,ngram-mod` and `ngram-mod,draft-mtp` produce the same internal order (`ngram-mod` then `draft-mtp`) and the same runtimes. Verified: both orders decode ~25.3 t/s at 8k. ### Results (decode generation t/s, 200 tokens, prose prompt) | ctx | MTP only | MTP + ngram-mod | verdict | |-----|----------|-----------------|---------| | 8k | 29.42 | 25.27 | ngram hurts -4.15 | | 16k | 26.42 | 21.13 | ngram hurts -5.29 | | 64k | 19.72 | 18.00 | ngram hurts -1.72 | At every depth, adding ngram-mod under MTP REDUCED decode throughput. The ngram draft costs its own forward pass / occupancy without adding enough accepted tokens on top of MTP. Baseline (no ngram) remains the production config (`--spec-type draft-mtp` only). Keep ngram-mod off. The historical harness was moved to `.AGENTTRASH/bench-scripts-20260901/` and is retained for audit only. it boots the v3 stack on a scratch port and reads authoritative `print_timing` lines. It requires the production service to be stopped. ## See Also - [[frank-redo-2026-08-17]] — the fleet topologies this slot has served. - [[Shared Memory Architecture Dankclaw]] — unified memory sizing. - [[qwen3.8-strix-optimizations]] — llama.cpp ROCm0 baselines on this iGPU. - [[VLLM Overview]] — vLLM reference (retired for this slot).