From 701e742f9ff12f3c7bef4d2be517c3fb7acd85d8 Mon Sep 17 00:00:00 2001 From: Frank Date: Thu, 20 Aug 2026 22:48:25 -0700 Subject: [PATCH] docs: fix stale context sizes, batch/ubatch, MoE, and service names across docs --- AGENTS.md | 2 +- docs/Dankclaw System Specs.md | 2 +- docs/deployment.md | 8 ++++---- docs/index.md | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index c0c326c..bbac1d8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,7 +105,7 @@ When you rename a service or move a model to a new host, update `models.json`. Update the provider key, the model `id`, `contextWindow`, and `input`. If the default model moved hosts, also update `defaultProvider` in `settings.json`. Set `contextWindow` to the per-slot context (dankserver -`175104`, dankclaw `262144`). The full schema is in the agent knowledge base +`262144`, dankclaw `262144`). The full schema is in the agent knowledge base page `~/.pi/agent/docs/pi/models.md`. ## Runtime & Troubleshooting diff --git a/docs/Dankclaw System Specs.md b/docs/Dankclaw System Specs.md index 47de076..fc80d21 100644 --- a/docs/Dankclaw System Specs.md +++ b/docs/Dankclaw System Specs.md @@ -93,7 +93,7 @@ cable between claw and server is required for 10G. - Unified memory eliminates separate VRAM limits. The GPU uses system RAM. - The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget. -- The active service is `frank-default-small` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (64K ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]]. +- The active service is `frank-default-large` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (262144 ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]]. - The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`. - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18. - 1.1 TB free disk space is ample for model storage. diff --git a/docs/deployment.md b/docs/deployment.md index 8725972..d06182b 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -105,10 +105,10 @@ curl -sf http://127.0.0.1:52625/health The active service on Dankserver is upstream llama.cpp (b10121) serving `Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the alias `frank-default-small` on port 7770. It runs the stock -`/home/dank/llama.cpp/build/bin/llama-server` binary with the 36 MoE -experts on CPU (`--n-cpu-moe 36`) and 7 CPU threads, ngram-mod +`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE +experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both -K and V, context 700000 (4 slots of ~175K each), batch 8192 / ubatch 3072, +K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048, `--no-mmap`. The unit file is `scripts/frank-default-small.service`. It launches through `scripts/serve-frank-small.sh`, which reads tunables from `scripts/frank-default-small.env`. @@ -122,7 +122,7 @@ companions) under the alias `frank-default-large` on port 52625. It runs 7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP (`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context (1 slot; concurrent requests queue, the server never preempts a running -generation), batch 2048 / ubatch 2048, 16 +generation), batch 4096 / ubatch 4096, 16 context checkpoints, 8 GiB prompt cache, and reasoning in deepseek format. The unit file is `scripts/frank-default-large.service`. It launches through diff --git a/docs/index.md b/docs/index.md index 7d374de..8df3bf3 100644 --- a/docs/index.md +++ b/docs/index.md @@ -18,7 +18,7 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx] end subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] - D[gfx1151 iGPU 96GB unified memory] + D[gfx1151 iGPU 64 GiB unified memory] E[CIRU ROCmFPX llama.cpp frank-default-large, port 52625] F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge