frank-default-large: 200000 context per slot (400000 total, 2 slots)
User request 2026-08-19: lower per-stream context from 262144 to 200000. CTX_SIZE is the total; llama.cpp splits it across the 2 parallel slots. f16 KV drops from ~32 GiB to ~24 GiB, adding OOM headroom. Docs: sync the 2026-08-18 service swap (frank-default-large on dankclaw, frank-default-small on dankserver) across AGENTS.md, README.md, deployment.md, ciru-activefpx-promptforge.md, and index.md. The pi models.json contextWindow/maxTokens for frank-default-large move 262144 -> 200000 (separate file, outside the repo).
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: CIRU ActiveFPX PromptForge on Dankclaw
|
||||
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-small service.
|
||||
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-large service.
|
||||
tags:
|
||||
- llama.cpp
|
||||
- rocm
|
||||
@@ -9,12 +9,12 @@ tags:
|
||||
- gfx1151
|
||||
- ciru
|
||||
- promptforge
|
||||
timestamp: 2026-08-18
|
||||
timestamp: 2026-08-19
|
||||
---
|
||||
|
||||
# CIRU ActiveFPX PromptForge on Dankclaw
|
||||
|
||||
`frank-default-small` (port 52625) now serves
|
||||
`frank-default-large` (port 52625) serves
|
||||
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` through the CIRU ROCmFPX
|
||||
llama.cpp runtime. This replaced the retired vLLM service that previously
|
||||
occupied the slot.
|
||||
@@ -124,16 +124,16 @@ Downloaded with the HF token to
|
||||
|
||||
## Service
|
||||
|
||||
- Unit: `frank-default-small.service`, port 52625
|
||||
- Script: `scripts/serve-frank-small.sh`, env `scripts/frank-default-small.env`
|
||||
- Unit: `frank-default-large.service`, port 52625
|
||||
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env`
|
||||
- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
|
||||
the system ROCm. `ROCM_PATH` points at the devel SDK.
|
||||
|
||||
### Launch profile (from the model card)
|
||||
### Launch profile
|
||||
|
||||
- 128K context (user request 2026-08-18; model card default is 64K), one
|
||||
slot, f16 KV (8 GiB at 128K, ~51.5 GiB total used of 64 GiB), context
|
||||
checkpoints (32), prompt cache
|
||||
- 200K context per slot (user request 2026-08-19; native max 262144),
|
||||
2 slots, f16 KV (~24 GiB total for both slots), context checkpoints
|
||||
(16), prompt cache
|
||||
- Native MTP draft, depth 4, device ROCm0
|
||||
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
|
||||
puts `<think>` reasoning in `message.reasoning_content`)
|
||||
|
||||
+3
-3
@@ -9,7 +9,7 @@ Two servers run LLM inference services. Each server pulls its scripts from this
|
||||
| Target | Host | Runtime | Service | Port |
|
||||
|--------|------|---------|---------|------|
|
||||
| Dankserver | dank@dankserver | upstream llama.cpp (Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE on CPU) | frank-default-small | 7770 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 256K ctx / 2 slots) | frank-default-large | 52625 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 200K ctx / 2 slots) | frank-default-large | 52625 |
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -120,8 +120,8 @@ The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
|
||||
companions) under the alias `frank-default-large` on port 52625. It runs
|
||||
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
|
||||
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
|
||||
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context per
|
||||
slot (524288 total across 2 parallel slots), batch 2048 / ubatch 2048, 32
|
||||
(`--spec-type draft-mtp`), f16 KV for both K and V, 200000 context per
|
||||
slot (400000 total across 2 parallel slots), batch 2048 / ubatch 2048, 16
|
||||
context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
|
||||
+1
-1
@@ -22,7 +22,7 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
|
||||
E[CIRU ROCmFPX llama.cpp
|
||||
frank-default-large, port 52625]
|
||||
F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge
|
||||
256K ctx / 2 slots, native MTP, PromptForge DualView]
|
||||
200K ctx / 2 slots, native MTP, PromptForge DualView]
|
||||
D --> E --> F
|
||||
end
|
||||
|
||||
|
||||
Reference in New Issue
Block a user