docs: fleet topology update — PP split renamed frank-default-large, NPU endpoint promoted to frank-default-small, 4-model NPU bakeoff matrix (2026-08-16)

AGENTS.md

@@ -14,7 +14,7 @@ when the bench finishes. The users will call if they need the model.
 Rules:
 
 1. Stop the service before any benchmark: `systemctl stop frank-default-large`
-   (Dankclaw) or `systemctl stop frank-default-small` (Dankserver). Use the
+   (Dankclaw) or `systemctl stop frank-default-small` (Dankclaw NPU). Use the
    passwordless helper or the password `Assface.3241` with `sudo -S`.
 2. Run the bench on a scratch port (never the service port 7770).
 3. Restart the service when the bench completes: `systemctl start <svc>`.
@@ -41,8 +41,8 @@ All script edits must flow through git. Do not edit files on the remote machine
 Use `scripts/deploy.sh` to deploy committed changes to a target. It pushes `main`, pulls on the target, restarts the service, and checks the health endpoint.
 
 ```
-./scripts/deploy.sh dankclaw        # vLLM (frank-default-large, port 7770)
+./scripts/deploy.sh dankclaw        # PP head (frank-default-large, port 7770)
-./scripts/deploy.sh dankserver      # llama.cpp (frank-default-small, port 7770)
+./scripts/deploy.sh dankserver      # PP worker (frank-pp-worker, RPC 50052)
 ./scripts/deploy.sh dankclaw --dry-run   # preview without changes
 ```
 
@@ -61,8 +61,9 @@ Target map:
 
 | Target | Host | Runtime | Service | Port |
 |--------|------|---------|---------|------|
-| Dankclaw | dank@dankclaw | llama.cpp (UD-Q8_K_XL, `--spec-default` ngram-mod, Vulkan0, 2 slots) | frank-default-large | 7770 |
+| Dankclaw | dank@dankclaw | llama.cpp PP head (Qwen3.8-27B UD-Q8_K_XL, split 0.40/0.60 to the Dankserver worker via RPC, MTP+ngram-mod, f16 KV) | frank-default-large | 7770 |
-| Dankserver | dank@dankserver | llama.cpp (Q6_K_P + ngram-mod, MoE offload) | frank-default-small | 7770 |
+| Dankclaw | dank@dankclaw | FastFlowLM NPU (XDNA2 50 TOPS, gemma4-it:e4b; see [[FastFlowLM NPU Inference]]) | frank-default-small | 52625 |
+| Dankserver | dank@dankserver | ggml-rpc worker (RTX 3090, receives the head's tensor share) | frank-pp-worker | 50052 |
 
 ## Runtime & Troubleshooting
 

docs/FastFlowLM NPU Inference.md

@@ -161,7 +161,7 @@ Proposed role: a small, always-on, low-power endpoint for triage, summarization,
 - Memlock fixed: `/etc/security/limits.d/99-flm-memlock.conf` sets soft and hard memlock to unlimited. A fresh sshd session picks it up via pam_limits. No reboot needed.
 - `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
 - `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
-- The `llama-server` (frank-pp-head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
+- The `llama-server` (frank-default-large, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
 
 ## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
 
@@ -177,6 +177,30 @@ Decode is flat at 19.3 t/s across all three runs. Output includes `reasoning_con
 
 Long-context prefill is not yet measured (runs 1-3 used under 200 prompt tokens). The 35B-A3B step remains untested.
 
+## Model Bakeoff (measured 2026-08-16, 4 models)
+
+All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-default-large) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
+
+| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
+|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
+| Gemma4-E4B | `gemma4-it:e4b` | 6.8 GB + 912 MB audio | 12.2 | 582 | 9.9 | 11.7 | 4/4 | 32768 |
+| Qwen3.5-9B | `qwen3.5:9b` | 7.3 GB | 7.2 | 343 | 6.6 | 7.3 | 4/4 | 32768 |
+| GPT-OSS-20B | `gpt-oss:20b` | 13.8 GB | 19.3 | 378 | 16.9 | n/a (no vision) | 0/4 | **8192** |
+| Qwen3.6-35B-A3B | `qwen3.6-moe:35b-a3b` | 22.2 GB | 12.7 | 207 | 11.2 | 12.7 | 4/4 | 32768 |
+
+Notes:
+
+- gpt-oss:20b is the fastest decoder (MoE, 3.6B active) but its KV cache fills at ~8k tokens (98.85% occupancy at 7,926; requests past ~8k return 400). It has no vision encoder.
+- 35B-A3B loads 28-29 GiB resident. It works but leaves ~1.5 GiB headroom with the PP head running. Do not run it alongside the head at full load.
+- Vision prefill for the two vision models with ~720 token image input: gemma 141 t/s, qwen3.5 176 t/s, 35B-A3B 68 t/s.
+- All four comply with the same image at 4/4 elements except gpt-oss, which cannot see it.
+
+## Promotion (2026-08-16)
+
+`frank-default-small` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-default-small.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-default-small.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
+
+The PP split was renamed `frank-pp-head` → `frank-default-large` (2026-08-16, commit `48142f5`).
+
 ## Parallel Streams (measured 2026-08-16)
 
 The NPU serializes generations. It runs exactly one stream at a time. Extra streams queue in FIFO order. FLM server flags `--socket` (default 10) and `--q-len` (default 10) cap queue depth only. They do not add concurrency.

docs/Pipeline Parallelism/Qwen3.8 PP Deployment.md

@@ -33,9 +33,9 @@ llama-server b10433                      ggml-rpc-server b10433
 |------|---------|---------|
 | `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
 | `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
-| `scripts/frank-pp-head.env` | claw | head tunables (split, ctx, spec, KV, I/O) |
+| `scripts/frank-default-large.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
 | `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
-| `scripts/frank-pp-head.service` / `frank-pp-worker.service` | both | optional systemd units (manual install) |
+| `scripts/frank-default-large.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
 | `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
 | `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
 
@@ -89,7 +89,7 @@ FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh stop-pp
 FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
 ```
 
-Config variants: edit `scripts/frank-pp-head.env` on the claw (git commit +
+Config variants: edit `scripts/frank-default-large.env` on the claw (git commit +
 pull if the bench runs from the repo), then re-run `start` + `bench`:
 
 - No spec baseline: `SPEC_NONE=1`

docs/index.md

@@ -8,24 +8,28 @@ Documentation for running vLLM inference on two local machines.
 graph LR
     subgraph Dankserver[NVIDIA Dankserver - Ubuntu 24.04]
         A[RTX 3090 24GB + 62GB RAM]
-        B[llama.cpp ngram-mod
+        B[ggml-rpc worker
-frank-default-small]
+frank-pp-worker, port 50052]
-        C[Qwen3.6-35B-A3B uncensored Q6_K_P
+        A --> B
-MoE experts on CPU, port 7770]
-        A --> B --> C
     end
 
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         D[gfx1151 APU 30GiB phys / 96GiB GPU-visible]
-        E[llama.cpp --spec-default ngram-mod
+        E[llama.cpp PP head
-frank-default-large]
+frank-default-large, port 7770]
         F[Qwen3.8-27B UD-Q8_K_XL
-Vulkan0 backend, port 7770, PARALLEL=2]
+split 0.40/0.60 to worker, MTP+ngram-mod]
         D --> E --> F
+        G[FastFlowLM NPU XDNA2 50 TOPS
+frank-default-small, port 52625]
+        H[gemma4-it:e4b
+12.2 t/s, 32k ctx, vision]
+        D --> G --> H
     end
 
-    G[LLM Clients / API] -->|HTTP :7770| B
+    I[LLM Clients / API] -->|HTTP :7770| E
-    G -->|HTTP :7770| E
+    I -->|HTTP :52625| G
+    E -->|RPC layers 0.40| B
 ```
 
 ## Servers