dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: reflect the 2026-08-18 second swap (rename + Qwen3.6 restore)

Browse Source 
- AGENTS.md: topology table, second-swap note, benchmark policy service
  names, mark FastMTP section retired, update model-load quirk.
- docs/index.md: intro, topology diagram, active-deployment + CIRU
  reference service names.
- docs/deployment.md: overview table, manual-update paths, rewrite both
  service-unit descriptions (Dankserver=Qwen3.6 Q6_K_P frank-default-small,
  Dankclaw=CIRU frank-default-large), validation model name.
This commit is contained in:
Frank committed 2026-08-18 01:29:21 -07:00
1 parent 1ba9618fcc
commit 819b9d448a
3 files changed
+65 -64
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+18 -13
View File
@@ -7,16 +7,19 @@ local machines. Topology (2026-08-18):
| Target | Host | Runtime | Service | Model | Port |
|--------|------|---------|---------|-------|------|
| Dankserver | dank@dankserver | llama.cpp (patched FastMTP build) | frank-default-large | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS + FastMTP 32K draft + ngram-mod | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-small | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP, PromptForge DualView | 52625 |
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP, PromptForge DualView, 256K context / 2 slots | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
NPU endpoint. The RTX 3090 runs the whole 27B model standalone
(no RPC worker). The 2026-08-18 swap retired the vLLM service on Dankclaw;
frank-default-small now runs the CIRU ROCmFPX llama.cpp runtime on the
Strix Halo iGPU (gfx1151). See docs/ciru-activefpx-promptforge.md.
The old scripts live in `old/scripts/` for reference; they describe the
retired PP topology.
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw; the
CIRU ROCmFPX llama.cpp runtime now runs on the Strix Halo iGPU (gfx1151).
A second 2026-08-18 swap renamed the services and restored the Qwen3.6 35B
A3B Q6_K_P model on Dankserver: Dankserver is now `frank-default-small`
(Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE experts on CPU) and Dankclaw is
now `frank-default-large` (CIRU Qwen3.8-27B, 256K context / 2 slots). The
retired Qwen3.8-27B IQ4_XS FastMTP files live in `.AGENTTRASH/`. See
docs/ciru-activefpx-promptforge.md. The old scripts live in `old/scripts/`
for reference; they describe the retired PP topology.
**Sibling project:** [../serverAppConfigs](../serverAppConfigs) manages the
Podman app stack on the same hosts (OpenWebUI, Gitea, observability). Its
@@ -33,8 +36,8 @@ when the bench finishes.
Rules:
1. Stop the service before any benchmark:
`sudo systemctl stop frank-default-large` (Dankserver) or
`sudo systemctl stop frank-default-small` (Dankclaw). Use the password
`sudo systemctl stop frank-default-small` (Dankserver) or
`sudo systemctl stop frank-default-large` (Dankclaw). Use the password
`Assface.3241` with `sudo -S` when no tty is available.
2. Run the bench on a scratch port (never the service ports 7770/52625).
3. Restart the service when the bench completes:
@@ -83,9 +86,11 @@ graph LR
## Runtime & Troubleshooting
### FastMTP patched llama.cpp (Dankserver)
### FastMTP patched llama.cpp (retired, 2026-08-18)
The HauhauCS FastMTP draft sidecar requires a llama.cpp build from commit
Historical: the Qwen3.8-27B IQ4_XS FastMTP service was retired on 2026-08-18
(files in `.AGENTTRASH/`). The build notes below are kept for reference.
The HauhauCS FastMTP draft sidecar required a llama.cpp build from commit
`4df29be4f4c3673f428170fda944a5b19f743bb8` plus the
`HauhauCS-FastMTP-llama.cpp.patch` (from the HF repo). The build lives at
`/home/dank/llama.cpp-fastmtp/build` on Dankserver. Loading the draft with
@@ -163,7 +168,7 @@ graph TD
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver ~5 minutes cold (15.7 GB IQ4_XS through RAM, `--no-mmap`); Dankclaw CIRU ~5-10 minutes (34.9 GiB ActiveFPX payload). Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 10 minutes on Dankserver and 20 minutes on Dankclaw (see `HEALTH_WAIT`).
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw CIRU loads the 34.9 GiB ActiveFPX payload in ~5-10 minutes. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).