docs: fix stale context sizes, batch/ubatch, MoE, and service names across docs

This commit is contained in:
Frank
2026-08-20 22:48:25 -07:00
parent cd6202f4a0
commit 701e742f9f
4 changed files with 7 additions and 7 deletions
+1 -1
View File
@@ -105,7 +105,7 @@ When you rename a service or move a model to a new host, update
`models.json`. Update the provider key, the model `id`, `contextWindow`, and
`input`. If the default model moved hosts, also update `defaultProvider` in
`settings.json`. Set `contextWindow` to the per-slot context (dankserver
`175104`, dankclaw `262144`). The full schema is in the agent knowledge base
`262144`, dankclaw `262144`). The full schema is in the agent knowledge base
page `~/.pi/agent/docs/pi/models.md`.
## Runtime & Troubleshooting
+1 -1
View File
@@ -93,7 +93,7 @@ cable between claw and server is required for 10G.
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget.
- The active service is `frank-default-small` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (64K ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]].
- The active service is `frank-default-large` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (262144 ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]].
- The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 1.1 TB free disk space is ample for model storage.
+4 -4
View File
@@ -105,10 +105,10 @@ curl -sf http://127.0.0.1:52625/health
The active service on Dankserver is upstream llama.cpp (b10121) serving
`Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the
alias `frank-default-small` on port 7770. It runs the stock
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 36 MoE
experts on CPU (`--n-cpu-moe 36`) and 7 CPU threads, ngram-mod
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE
experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod
speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both
K and V, context 700000 (4 slots of ~175K each), batch 8192 / ubatch 3072,
K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048,
`--no-mmap`. The unit file is `scripts/frank-default-small.service`. It
launches through `scripts/serve-frank-small.sh`, which reads tunables from
`scripts/frank-default-small.env`.
@@ -122,7 +122,7 @@ companions) under the alias `frank-default-large` on port 52625. It runs
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
(1 slot; concurrent requests queue, the server never preempts a running
generation), batch 2048 / ubatch 2048, 16
generation), batch 4096 / ubatch 4096, 16
context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
+1 -1
View File
@@ -18,7 +18,7 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
D[gfx1151 iGPU 96GB unified memory]
D[gfx1151 iGPU 64 GiB unified memory]
E[CIRU ROCmFPX llama.cpp
frank-default-large, port 52625]
F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge