docs: qwen3.6-moe unreachable incident (2026-08-16) + topology reconciliation
- FastFlowLM doc: incident section (EXEC-203 exec-bit loop, client model ID 'frank-default-small' rejected, swap churn 14:48-15:03), FLM unknown-tag fallback behavior (silently loads llama3.2:1b default), working models.json entry recorded, promotion status -> qwen3.6-moe - AGENTS.md: Overview + target map now reflect qwen3.6-moe:35b-a3b on the NPU endpoint (Dankserver llama.cpp 35B service stopped) - deployment.md: topology table current; Dankserver frank-default-small section marked STOPPED 2026-08-16 (alias moved to the NPU endpoint)
This commit is contained in:
1 parent
50c55ee220
commit
9ec5678ef0
3 files changed
+34
-7
No files matched your search
+8
-3
@@ -8,8 +8,11 @@ Two servers run LLM inference services. Each server pulls its scripts from this
|
||||
|
||||
| Target | Host | Runtime | Service | Port |
|
||||
|--------|------|---------|---------|------|
|
||||
| Dankclaw | dank@dankclaw | vLLM-ROCm (CIRU INT4) | frank-default-large | 7770 |
|
||||
| Dankserver | dank@dankserver | llama.cpp (Q6_K_P + ngram-mod, MoE offload) | frank-default-small | 7770 |
|
||||
| Dankclaw | dank@dankclaw | llama.cpp PP head (Qwen3.8-27B UD-Q8_K_XL) | frank-default-large | 7770 |
|
||||
| Dankclaw | dank@dankclaw | FastFlowLM NPU (qwen3.6-moe:35b-a3b) | frank-default-small | 52625 |
|
||||
| Dankserver | dank@dankserver | ggml-rpc worker (RTX 3090) | frank-pp-worker | 50052 |
|
||||
|
||||
The `frank-default-small` (NPU) unit is not in `deploy.sh`. Restart it by hand with `sudo systemctl restart frank-default-small`.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -155,7 +158,9 @@ from `scripts/frank-default-large.env`.
|
||||
> the bench table. Also retired earlier: bundled MTP, the vLLM-ROCm CIRU
|
||||
> INT4 Ling-3.0-flash runtime, and the llama-serve/vLLM units on Dankclaw.
|
||||
|
||||
### Dankserver — frank-default-small
|
||||
### Dankserver — frank-default-small (STOPPED 2026-08-16)
|
||||
|
||||
> Note (2026-08-16): the `frank-default-small` alias now belongs to the FastFlowLM NPU endpoint on Dankclaw (port 52625, see [[FastFlowLM NPU Inference]]). The llama.cpp Qwen3.6-35B-A3B service on Dankserver described below is stopped (verified `inactive` 2026-08-16). The text below is kept as history.
|
||||
|
||||
The active service on Dankserver runs llama.cpp (upstream, `b10108+`) and serves the HauhauCS Qwen3.6-35B-A3B uncensored `Q6_K_P` model under the alias `frank-default-small` on port 7770. It uses **ngram-mod** speculative decoding (`--spec-ngram-mod-n-min 48 / n_max 64 / n_match 24`), **4 parallel slots** at 175104 context each (pool 700416), and mmproj vision. **36 of 40 MoE expert layers** stay on CPU (`--n-cpu-moe 36`); 4 expert layers' weights are back on GPU alongside all non-expert tensors (`--gpu-layers all`). KV stays on GPU (K=`f16`, V=`f16`), refitted from 800000 on 2026-08-14 when the pool went dual-f16 (the f16/f16 pool costs 13680 MiB at 700416 cells and did not fit at 800768 cells). Batch 8192 / ubatch 3072, `--ctx-checkpoints 4`, `--no-mmap`. The unit file is `scripts/frank-default-small.service`, symlinked to `/etc/systemd/system/`. It launches through `scripts/serve-llama-frank-small.sh`, which reads tunables from `scripts/frank-default-small.env`.
|
||||
|
||||
|
||||
Reference in new issue
Block a user