frank-default-large: 200000 context per slot (400000 total, 2 slots)

User request 2026-08-19: lower per-stream context from 262144 to 200000.
CTX_SIZE is the total; llama.cpp splits it across the 2 parallel slots.
f16 KV drops from ~32 GiB to ~24 GiB, adding OOM headroom.

Docs: sync the 2026-08-18 service swap (frank-default-large on dankclaw,
frank-default-small on dankserver) across AGENTS.md, README.md,
deployment.md, ciru-activefpx-promptforge.md, and index.md. The pi
models.json contextWindow/maxTokens for frank-default-large move
262144 -> 200000 (separate file, outside the repo).
This commit is contained in:
Frank committed 2026-08-19 10:38:53 -07:00
1 parent 9fd82dc6e1
commit 8e4d63b68d
6 files changed
+32 -31

No files matched your search

+9 -9
View File
@@ -1,7 +1,7 @@
---
type: reference
title: CIRU ActiveFPX PromptForge on Dankclaw
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-small service.
description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-default-large service.
tags:
- llama.cpp
- rocm
@@ -9,12 +9,12 @@ tags:
- gfx1151
- ciru
- promptforge
timestamp: 2026-08-18
timestamp: 2026-08-19
---
# CIRU ActiveFPX PromptForge on Dankclaw
`frank-default-small` (port 52625) now serves
`frank-default-large` (port 52625) serves
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` through the CIRU ROCmFPX
llama.cpp runtime. This replaced the retired vLLM service that previously
occupied the slot.
@@ -124,16 +124,16 @@ Downloaded with the HF token to
## Service
- Unit: `frank-default-small.service`, port 52625
- Script: `scripts/serve-frank-small.sh`, env `scripts/frank-default-small.env`
- Unit: `frank-default-large.service`, port 52625
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env`
- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
the system ROCm. `ROCM_PATH` points at the devel SDK.
### Launch profile (from the model card)
### Launch profile
- 128K context (user request 2026-08-18; model card default is 64K), one
slot, f16 KV (8 GiB at 128K, ~51.5 GiB total used of 64 GiB), context
checkpoints (32), prompt cache
- 200K context per slot (user request 2026-08-19; native max 262144),
2 slots, f16 KV (~24 GiB total for both slots), context checkpoints
(16), prompt cache
- Native MTP draft, depth 4, device ROCm0
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
puts `<think>` reasoning in `message.reasoning_content`)