Files

T

Frank 968066ff23 docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup

- deployment.md: Boot Policy (2026-08-17) section (disabled units table,
  re-enable recipes, stale frank-restart-small.sh helper, broken
  deploy.sh dankserver target); Update Dankserver -> frank-pp-worker;
  condensed retired 35B history block and 2026-08-13 incident
- AGENTS.md: boot-policy note, target map fix (Q6_K, split 0.68),
  deploy.sh dankserver warning, obsolete quirk entries condensed
- README.md: rewrite active services/models tables (PP head + worker)
- index.md: topology fix (qwythos:9b, Q6_K split 0.68, NPU disabled)
- qwen3.8-spec-matrix.md: superseded configs merged into compact
  history; stale inline env block removed; bundled-head MTP note
- FastFlowLM, PP Deployment: boot-policy + enablement notes
- System Specs x2, DFlash doc, 35B FP8 doc, Ling docs: stale active
  claims fixed, retired banners, never-shipped plan stubbed

2026-08-17 03:30:00 -07:00

1.8 KiB

Raw Blame History

vLLM Scripts

Deployment scripts for the LLM inference fleet: a llama.cpp pipeline-parallel Qwen3.8-27B split across two machines, plus the FastFlowLM NPU endpoint on Dankclaw.

Targets

Host	User	Codename	Architecture
dankserver	dank	Dankserver	NVIDIA RTX 3090 + 62 GB RAM
dankclaw	dank	Dankclaw	AMD Strix Halo APU (shared memory)

Active Services

Service	Host	Runtime	Model	Port
frank-default-large	dankclaw	llama.cpp PP head (RPC to Dankserver)	Qwen3.8-27B-Uncensored Q6_K, standalone MTP draft	7770
frank-pp-worker	dankserver	ggml-rpc worker (RTX 3090)	receives the head's tensor share	50052

Boot policy (2026-08-17): only these two units are enabled. The FastFlowLM NPU endpoint (frank-default-small, qwythos:9b, port 52625) is disabled for boot; re-enable it from scripts/frank-default-small.service if needed. Retired units: the Dankserver llama.cpp 35B service, llamaserver, ollama, vllm-awq-qwen36, vllm-qwen36, frank-large-ling (Ling trial), and the vLLM-ROCm CIRU path. Their unit files remain in scripts/ for reference.

Structure

docs/ — knowledge base: deployment runbook, PP runbook, spec matrix, NPU notes, research archive
scripts/ — serve scripts, env files, systemd units, deploy and test tools

Deployment

See docs/deployment.md. Run ./scripts/deploy.sh dankclaw for the PP head. The dankserver target is broken (it targets the removed frank-default-small unit); restart the worker with sudo systemctl restart frank-pp-worker until the script is fixed.

Knowledge Base

See docs/index.md for the project knowledge base index.

1.8 KiB Raw Blame History