Files

T

Frank 048e7eaaba frank-default-large: full 262144 context, 1 slot

User request 2026-08-19 (second pass): parallel slots degraded
performance. PARALLEL=1, CTX_SIZE=262144 (native max, -c equals the
per-slot context with one slot). Concurrent requests queue on the
single slot; the server never preempts a running generation.
pi models.json contextWindow/maxTokens back to 262144 (outside repo).
Docs updated to match.

2026-08-19 11:17:31 -07:00

6.5 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

CIRU ActiveFPX PromptForge on Dankclaw

frank-default-large (port 52625) serves jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge through the CIRU ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that previously occupied the slot.

What This Is

The model is a GGUF, not a vLLM checkpoint. It is the quality-and-speed release of Qwen3.8-27B for AMD ROCm. CIRU DualView keeps a compact ActiveFPX GGUF for generation and load optional PromptForge companion views to accelerate large prompts. The model carries native depth-4 multi-token prediction (MTP).

Artifact	Size
`Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v2.gguf`	15.19 GiB
`Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-FFN.pfs`	15.95 GiB
`Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-GDN.pfs`	3.75 GiB
Three-file payload	~34.89 GiB

The two .pfs companions are optional. Omit them to run the GGUF alone and save ~19.7 GiB of memory. The GGUF remains independently usable.

Requirements

ROCm 7.15 is required for the advertised Strix Halo performance. The system ROCm (/opt/rocm-7.2.4) selects slower or fallback kernels.
The pinned runtime is ciru-ai/ROCmFPX tag qwen3.8-activefpx-promptforge-v2.3 (commit 103af8b41).
Composable Kernel at fdf4bb7fcc984811cef48ce817d89aac064b984a.
Validated GPU: AMD Radeon 8060S / gfx1151.

ROCm 7.15 Toolchain

The retired ling3-ciru package left a complete TheRock ROCm 7.15 SDK (7.15.0a20260728) inside its venv:

/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/
  _rocm_sdk_devel/      # hipcc, amdclang 23, cmake, headers
  _rocm_sdk_libraries/  # hipblas, hipblaslt, rocroller
  _rocm_sdk_core/       # libamdhip64, libhsa-runtime64
  rocm-sdk-device-gfx1151

The compiler is amdclang 23. rocm-sdk version reports 7.15.0a20260728. The binary is built against and run with these 7.15 libraries.

Build Runbook (2026-08-18)

Performed on Dankclaw:

Install host build tools (ninja-build was missing): sudo apt install ninja-build.

Clone the runtime and CK at their pinned revisions:

git clone --depth 1 --branch qwen3.8-activefpx-promptforge-v2.3 \
  https://github.com/ciru-ai/ROCmFPX.git
git clone --depth 1 https://github.com/ROCm/composable_kernel.git
git -C composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a

CK headers are generated at CK configure time (ck/config.h, ck/version.h). The llama.cpp build includes the CK source tree, so run CK's configure once and copy the headers into include/ck/:
```
cmake -S composable_kernel -B build-ck -DAMDGPU_TARGETS=gfx1151 -DCK_ENABLE_WMMA=ON
cp build-ck/include/ck/config.h build-ck/include/ck/version.h composable_kernel/include/ck/
```

Configure the llama.cpp build against the venv ROCm 7.15 SDK:

export ROCM_PATH=$SDK/_rocm_sdk_devel
cmake -S ROCmFPX -B build-promptforge -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=$ROCM_PATH/llvm/bin/clang \
  -DCMAKE_CXX_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
  -DCMAKE_HIP_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
  -DCMAKE_PREFIX_PATH=$ROCM_PATH \
  -DGGML_HIP=ON -DGGML_CUDA=OFF -DGGML_VULKAN=OFF \
  -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \
  -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \
  -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \
  -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \
  -DPROMPTFORGE_CK_ROOT=$CK_ROOT \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_BUILD_WEBUI=OFF
cmake --build build-promptforge --target llama-server -j$(nproc)

LLAMA_BUILD_WEBUI=OFF is required. The embedded browser UI downloads assets from an HF bucket (llama-ui) that returns 404; the service is consumed through the OpenAI-compatible API, so the UI is not needed.

Build output: build-promptforge/bin/llama-server.

graph TD
    A[Clone ROCmFPX v2.3 + CK pinned] --> B[Configure CK, copy generated headers]
    B --> C[Configure llama-server vs ROCm 7.15 SDK]
    C --> D[Build llama-server]
    D --> E[Verify --list-devices shows gfx1151]
    E --> F[Download v2 GGUF + FFN + GDN]
    F --> G[Deploy frank-default-small]
    G --> H[Health check + chat completion]

Model Download

Downloaded with the HF token to /home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/.

Service

Unit: frank-default-large.service, port 52625
Script: scripts/serve-frank-large.sh, env scripts/frank-default-large.env
The script exports the ROCm 7.15 SDK libraries in LD_LIBRARY_PATH before the system ROCm. ROCM_PATH points at the devel SDK.

Launch profile

262144 context (native max), 1 slot (user request 2026-08-19; parallel slots degraded performance), f16 KV (~16 GiB), context checkpoints (16), prompt cache. Concurrent requests queue on the single slot; the server never preempts a running generation
Native MTP draft, depth 4, device ROCm0
--temp 0.7 --top-p 0.95, reasoning on (--reasoning-format deepseek puts <think> reasoning in message.reasoning_content)
PromptForge env: PROMPTFORGE_MODE=m2048_fused_tail1476, GGML_CUDA_GRAPH_OPT=0, LLAMA_MTP_CPU_ARGMAX_FASTPATH=1

Required environment

export PROMPTFORGE_SIDECAR=...FFN.pfs
export PROMPTFORGE_GDN_SIDECAR=...GDN.pfs
export PROMPTFORGE_MODE=m2048_fused_tail1476
export GGML_CUDA_GRAPH_OPT=0
export LLAMA_MTP_CPU_ARGMAX_FASTPATH=1

The p_min=0 MTP profile uses the CPU argmax fast path. This avoids ROCm's unsupported TOP_K operator.

Operational Notes

Health: curl http://127.0.0.1:52625/health. Returns 503 while loading.
Model load is slow (34.9 GiB payload); do not kill the process during warmup.
deploy.sh dankclaw waits 20 minutes for health.
The systemd unit must not set HOST or TEMP env vars (HIP collision rule). --host/--port/--temp are CLI flags only.

6.5 KiB Raw Blame History