Frank
6fae3a9f1a
The helper scripts (/usr/local/sbin/frank-restart-<large|small>.sh) cover unit symlink, cache drop (dankclaw), daemon-reload, and restart in one root-owned script, run via sudoers NOPASSWD. deploy.sh now uses the helper when present and falls back to inline sudo commands.
vLLM Scripts
Scripts for running LLM inference on two local machines.
Targets
| Host | User | Codename | Architecture |
|---|---|---|---|
| dankserver | dank | Dankserver | NVIDIA RTX 3090 + 62 GB RAM |
| dankclaw | dank | Dankclaw | AMD Strix Halo APU (shared memory) |
Active Services
| Service | Host | Runtime | Model | Port |
|---|---|---|---|---|
| frank-default-large | dankclaw | vLLM-ROCm (CIRU) | Ling-3.0-flash CIRU INT4 MTP | 7770 |
| frank-default-small | dankserver | llama.cpp (ngram-mod) | Qwen3.6-35B-A3B uncensored Q6_K_P | 7770 |
The retired ROCmFPX llama.cpp service on Dankclaw was replaced by the vLLM service above. The retired
vllm-awq-qwen36(AWQ 4-bit) service on Dankserver was replaced by the llama.cpp service above. Legacy unit files remain inscripts/for reference.
Models
| Model | Target | Format |
|---|---|---|
| Ling-3.0-flash CIRU INT4 (MTP) | dankclaw | INT4 W4A16 (Quark/vLLM) |
| Qwen3.6 35B A3B uncensored Q6_K_P (ngram-mod speculative decoding) | dankserver | GGUF Q6_K_P (llama.cpp, MoE offload) |
| Qwen3.6 35B A3B AWQ 4bit | dankserver | AWQ (retired) |
| Qwen3.6 35B A3B FP8 | dankserver | FP8 (documented, not active) |
| Qwen3.6 27B FP8 | dankclaw | FP8 (documented, not active) |
Structure
docs/— Project knowledge base with research notes on vLLM configuration, hardware specifics, speculative decoding, and deployment notesscripts/— Deployment and launch scripts (deploy.sh, serve-llama-frank-small.sh, serve-ling3-vllm.sh, systemd unit files)
Deployment
See docs/deployment.md for the deploy/update process. Run ./scripts/deploy.sh <dankclaw|dankserver> after pushing committed changes.
Knowledge Base
See docs/index.md for the project knowledge base index.