dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: document frank-default-large v3 upgrade (Output-K8 runtime, full-W8 views, BF16 vision); update AGENTS/deploy payload sizes

Browse Source
This commit is contained in:
Frank committed 2026-08-21 20:18:57 -07:00
1 parent 132e1478cf
commit 8652116cec
4 files changed
+81 -38
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+9 -6
View File
@@ -8,7 +8,7 @@ local machines. Topology (2026-08-21):
| Target | Host | Runtime | Service | Model | Port |
|--------|------|---------|---------|-------|------|
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP, PromptForge DualView, 262K context / 1 slot | 52625 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3, native MTP + Output-K8 proxy, PromptForge DualView, 262K context / 1 slot | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw; the
@@ -129,16 +129,19 @@ for the base recipe.
The CIRU llama-server serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge`
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
The runtime is the ciru-ai/ROCmFPX tag `qwen3.8-activefpx-promptforge-v2.3`
(commit `103af8b41`) built against the private TheRock ROCm 7.15 SDK kept
(commit `103af8b41`) **plus the v3 output-K8 runtime patch** (commit
`af0eac6` on Dankclaw) built against the private TheRock ROCm 7.15 SDK kept
in the retired ling3-ciru venv
(`.venv/lib/python3.12/site-packages/_rocm_sdk_*`). The model card requires
ROCm 7.15 for the advertised Strix Halo performance; the system ROCm 7.2.4
selects slower kernels. The serve script exports the 7.15 SDK libraries
first in `LD_LIBRARY_PATH`.
The model files (v2 GGUF + FFN.pfs + GDN.pfs, ~34.9 GiB) live in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/`. The full build runbook is
in docs/ciru-activefpx-promptforge.md. The retired vLLM venvs
The v3 model files (GGUF + FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj,
~36.7 GiB) live in the v3 staging directory
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`; the v2 artifacts remain
in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` for rollback.
The full build runbook is in docs/ciru-activefpx-promptforge.md. The retired vLLM venvs
(`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to
`/home/dank/.AGENTTRASH/` on 2026-08-18.
@@ -195,7 +198,7 @@ graph TD
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw CIRU loads the 34.9 GiB ActiveFPX payload in ~5-10 minutes. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw CIRU loads the v3 ~36.7 GiB payload in ~5-10 minutes. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).