# vLLM Scripts

Deployment scripts for the LLM inference fleet: a llama.cpp
pipeline-parallel Qwen3.8-27B split across two machines, plus the
FastFlowLM NPU endpoint on Dankclaw.

## Targets

| Host | User | Codename | Architecture |
|------|------|----------|--------------|
| dankserver | dank | Dankserver | NVIDIA RTX 3090 + 62 GB RAM |
| dankclaw | dank | Dankclaw | AMD Strix Halo APU (shared memory) |

## Active Services

| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-default-large | dankclaw | llama.cpp PP head (RPC to Dankserver) | Qwen3.8-27B-Uncensored Q6_K, standalone MTP draft | 7770 |
| frank-pp-worker | dankserver | ggml-rpc worker (RTX 3090) | receives the head's tensor share | 50052 |

Boot policy (2026-08-17): only these two units are enabled. The
FastFlowLM NPU endpoint (`frank-default-small`, qwythos:9b, port 52625)
is disabled for boot; re-enable it from `scripts/frank-default-small.service`
if needed. Retired units: the Dankserver llama.cpp 35B service,
`llamaserver`, `ollama`, `vllm-awq-qwen36`, `vllm-qwen36`,
`frank-large-ling` (Ling trial), and the vLLM-ROCm CIRU path. Their unit
files remain in `scripts/` for reference.

## Structure

- `docs/` — knowledge base: deployment runbook, PP runbook, spec matrix,
  NPU notes, research archive
- `scripts/` — serve scripts, env files, systemd units, deploy and test
  tools

## Deployment

See [docs/deployment.md](docs/deployment.md). Run
`./scripts/deploy.sh dankclaw` for the PP head. The `dankserver` target
is broken (it targets the removed `frank-default-small` unit); restart
the worker with `sudo systemctl restart frank-pp-worker` until the
script is fixed.

## Knowledge Base

See [docs/index.md](docs/index.md) for the project knowledge base index.