T

Frank 1ba9618fcc Swap fleet naming + restore Qwen3.6-35B-A3B Q6_K_P on Dankserver

- Restore the pre-2026-08-17 Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
  Q6_K_P + ngram-mod setup as frank-default-small on Dankserver (port 7770,
  upstream llama.cpp b10121, MoE experts on CPU via --n-cpu-moe 36).
- Rename the CIRU Qwen3.8-27B ActiveFPX setup from frank-default-small to
  frank-default-large (Dankclaw, port 52625); bump to CTX_SIZE=262144
  (native max) and PARALLEL=2 (96 GiB unified pool).
- Retire the Qwen3.8-27B IQ4_XS FastMTP frank-default-large (Dankserver) to
  .AGENTTRASH.
- deploy.sh: SERVICE[dankserver]=frank-default-small,
  SERVICE[dankclaw]=frank-default-large; HEALTH_WAIT[dankserver]=600.

2026-08-18 00:32:26 -07:00

docs

docs: note 128K context and thinking on for frank-default-small

2026-08-17 23:12:32 -07:00

old/scripts

fleet redo 2026-08-17: retire PP+NPU, new topology

2026-08-17 15:23:46 -07:00

scripts

Swap fleet naming + restore Qwen3.6-35B-A3B Q6_K_P on Dankserver

2026-08-18 00:32:26 -07:00

.gitignore

chore: ignore obsidian workspace files

2026-08-06 00:14:06 -07:00

AGENTS.md

dankclaw: replace vLLM with CIRU ROCmFPX llama.cpp runtime

2026-08-17 21:12:35 -07:00

README.md

docs: reflect the 2026-08-17 evening changes (Quark model swap, tool calling, batch 8192)

2026-08-17 18:01:02 -07:00

README.md

vLLM Scripts

Deployment scripts for the LLM inference fleet (2026-08-17 topology): llama.cpp FastMTP on Dankserver and vLLM ROCm on Dankclaw.

Targets

Host	User	Codename	GPU
dankserver	dank	Dankserver	NVIDIA RTX 3090 24 GB + 62 GB RAM
dankclaw	dank	Dankclaw	AMD Strix Halo iGPU gfx1151 (unified memory)

Active Services

Service	Host	Runtime	Model	Port
frank-default-large	dankserver	llama.cpp (patched FastMTP build)	HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS, FastMTP 32K draft + ngram-mod, K q8_0 / V f16 KV, 128K ctx	7770
frank-default-small	dankclaw	vLLM ROCm (vllm-venv)	nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8, full 262144 ctx, no KV quantization, 4 seqs, batch 8192, ngram spec + tool calling	52625

The 2026-08-17 redo retired the pipeline-parallel split (PP head + RPC worker) and the FastFlowLM NPU endpoint. Their scripts are preserved in old/scripts/ for reference.

Structure

docs/ — knowledge base: deployment runbook, model notes, research archive
scripts/ — serve scripts, env files, systemd units, deploy and test tools
old/scripts/ — retired PP/NPU-era scripts (reference only)

Deployment

See docs/deployment.md. Run FRANK_SUDO_PW=... ./scripts/deploy.sh dankserver for the llama.cpp box or FRANK_SUDO_PW=... ./scripts/deploy.sh dankclaw for the vLLM box.

Knowledge Base

See docs/index.md for the project knowledge base index.