docs: fix stale context sizes, batch/ubatch, MoE, and service names across docs
This commit is contained in:
@@ -105,7 +105,7 @@ When you rename a service or move a model to a new host, update
|
||||
`models.json`. Update the provider key, the model `id`, `contextWindow`, and
|
||||
`input`. If the default model moved hosts, also update `defaultProvider` in
|
||||
`settings.json`. Set `contextWindow` to the per-slot context (dankserver
|
||||
`175104`, dankclaw `262144`). The full schema is in the agent knowledge base
|
||||
`262144`, dankclaw `262144`). The full schema is in the agent knowledge base
|
||||
page `~/.pi/agent/docs/pi/models.md`.
|
||||
|
||||
## Runtime & Troubleshooting
|
||||
|
||||
@@ -93,7 +93,7 @@ cable between claw and server is required for 10G.
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget.
|
||||
- The active service is `frank-default-small` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (64K ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]].
|
||||
- The active service is `frank-default-large` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (262144 ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]].
|
||||
- The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`.
|
||||
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
|
||||
- 1.1 TB free disk space is ample for model storage.
|
||||
|
||||
+4
-4
@@ -105,10 +105,10 @@ curl -sf http://127.0.0.1:52625/health
|
||||
The active service on Dankserver is upstream llama.cpp (b10121) serving
|
||||
`Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the
|
||||
alias `frank-default-small` on port 7770. It runs the stock
|
||||
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 36 MoE
|
||||
experts on CPU (`--n-cpu-moe 36`) and 7 CPU threads, ngram-mod
|
||||
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE
|
||||
experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod
|
||||
speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both
|
||||
K and V, context 700000 (4 slots of ~175K each), batch 8192 / ubatch 3072,
|
||||
K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048,
|
||||
`--no-mmap`. The unit file is `scripts/frank-default-small.service`. It
|
||||
launches through `scripts/serve-frank-small.sh`, which reads tunables from
|
||||
`scripts/frank-default-small.env`.
|
||||
@@ -122,7 +122,7 @@ companions) under the alias `frank-default-large` on port 52625. It runs
|
||||
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
|
||||
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
|
||||
(1 slot; concurrent requests queue, the server never preempts a running
|
||||
generation), batch 2048 / ubatch 2048, 16
|
||||
generation), batch 4096 / ubatch 4096, 16
|
||||
context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
|
||||
end
|
||||
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
D[gfx1151 iGPU 96GB unified memory]
|
||||
D[gfx1151 iGPU 64 GiB unified memory]
|
||||
E[CIRU ROCmFPX llama.cpp
|
||||
frank-default-large, port 52625]
|
||||
F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge
|
||||
|
||||
Reference in New Issue
Block a user