Frank
136ee080c9
- Ethernet Bandwidth Math: new section 0 with live measurements (iperf 281-294 MB/s, ping RTT both paths, prefill 457.9 t/s at 2.5GbE, per-ubatch wire 20.5 MB, 98k KV snapshot 3.45 GB blocking the slot 13.6 s, validated link-speed model: 386/458/505 t/s) - System specs: NIC tables for both boxes; the 1 GbE USB-C adapter on the claw was silently routing the RPC path (unplugged 2026-08-17) - PP Deployment: fix hidden-state size (10 KiB/direction, H=5120) and RTT-vs-bandwidth claim (decode RTT-bound; prefill+snapshot BW-bound)
vLLM Scripts
Deployment scripts for the LLM inference fleet: a llama.cpp pipeline-parallel Qwen3.8-27B split across two machines, plus the FastFlowLM NPU endpoint on Dankclaw.
Targets
| Host | User | Codename | Architecture |
|---|---|---|---|
| dankserver | dank | Dankserver | NVIDIA RTX 3090 + 62 GB RAM |
| dankclaw | dank | Dankclaw | AMD Strix Halo APU (shared memory) |
Active Services
| Service | Host | Runtime | Model | Port |
|---|---|---|---|---|
| frank-default-large | dankclaw | llama.cpp PP head (RPC to Dankserver) | Qwen3.8-27B-Uncensored Q6_K, standalone MTP draft | 7770 |
| frank-pp-worker | dankserver | ggml-rpc worker (RTX 3090) | receives the head's tensor share | 50052 |
Boot policy (2026-08-17): only these two units are enabled. The
FastFlowLM NPU endpoint (frank-default-small, qwythos:9b, port 52625)
is disabled for boot; re-enable it from scripts/frank-default-small.service
if needed. Retired units: the Dankserver llama.cpp 35B service,
llamaserver, ollama, vllm-awq-qwen36, vllm-qwen36,
frank-large-ling (Ling trial), and the vLLM-ROCm CIRU path. Their unit
files remain in scripts/ for reference.
Structure
docs/— knowledge base: deployment runbook, PP runbook, spec matrix, NPU notes, research archivescripts/— serve scripts, env files, systemd units, deploy and test tools
Deployment
See docs/deployment.md. Run
./scripts/deploy.sh dankclaw for the PP head. The dankserver target
is broken (it targets the removed frank-default-small unit); restart
the worker with sudo systemctl restart frank-pp-worker until the
script is fixed.
Knowledge Base
See docs/index.md for the project knowledge base index.