dankclaw: replace vLLM with CIRU ROCmFPX llama.cpp runtime
Retire the vLLM service (frank-default-small, port 52625) on Dankclaw and serve jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge through the CIRU ROCmFPX llama.cpp runtime instead. - Build llama-server from ciru-ai/ROCmFPX tag qwen3.8-activefpx-promptforge-v2.3 (commit 103af8b41) against the private TheRock ROCm 7.15 SDK in the retired ling3-ciru venv. The model card requires ROCm 7.15 for the advertised Strix Halo performance; the system ROCm 7.2.4 selects slower kernels. CK pinned at fdf4bb7fcc984811cef48ce817d89aac064b984a; generated ck/config.h + ck/version.h copied into the CK source tree so the ggml-hip build can include them. LLAMA_BUILD_WEBUI=OFF (HF bucket llama-ui returns 404; server is consumed via the OpenAI API). - New scripts/serve-frank-small.sh + frank-default-small.env: 64K ctx, one slot, f16 KV, context checkpoints, prompt cache, native depth-4 MTP on ROCm0, PromptForge DualView sidecars (FFN + GDN), temp 0.7 top-p 0.95, reasoning off. LD_LIBRARY_PATH puts the 7.15 SDK libs before any system ROCm. - Update the systemd unit description (same service name and port). - deploy.sh: update the dankclaw target comment. - Docs: new docs/ciru-activefpx-promptforge.md with the build runbook; update docs/index.md and AGENTS.md topology. - The vLLM venvs (vllm-venv, vllm-vk-venv) were moved to /home/dank/.AGENTTRASH/ on the target; the vllm-vulkan plugin and docs were removed from the repo in an earlier commit.
This commit is contained in:
1 parent
4941d911f2
commit
3b51b5a94a
7 files changed
+359
-157
No files matched your search
+4
-3
@@ -16,9 +16,9 @@
|
||||
# ./deploy.sh <target> [--dry-run]
|
||||
# FRANK_SUDO_PW=... ./deploy.sh <target>
|
||||
#
|
||||
# Targets (2026-08-17 topology):
|
||||
# Targets (2026-08-18 topology):
|
||||
# dankserver - llama.cpp FastMTP (frank-default-large.service, port 7770)
|
||||
# dankclaw - vLLM ROCm (frank-default-small.service, port 52625)
|
||||
# dankclaw - CIRU ROCmFPX llama.cpp (frank-default-small.service, port 52625)
|
||||
set -euo pipefail
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -41,7 +41,8 @@ REMOTE_DIR[dankclaw]="/home/dank/vllmScripts"
|
||||
SERVICE[dankclaw]="frank-default-small"
|
||||
SERVICE_HEALTH[dankclaw]="http://127.0.0.1:52625/health"
|
||||
DROP_CACHES[dankclaw]=0
|
||||
# vLLM loads a 38 GB checkpoint; allow 20 minutes.
|
||||
# CIRU llama.cpp (ROCm 7.15) loads the 34.9 GiB ActiveFPX payload + MTP
|
||||
# draft and initializes PromptForge views; allow 20 minutes.
|
||||
HEALTH_WAIT[dankclaw]=600
|
||||
|
||||
TARGET="${1:-}"
|
||||
|
||||
Reference in new issue
Block a user