dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity
174 Commits 59 Branches 8 Tags
8e4d63b68dce05ab7e16cecaa9cf3cffd150cf6a
Commit Graph
5 Commits
Author SHA1 Message Date
Frank 8c391fe120 Reduce ctxcp from 32 to 16 to free ~16 GiB VRAM headroom and prevent OOM crashes during prompt processing 2026-08-18 16:11:58 -07:00
Frank a08be1641f frank-default-large: full native 262144 context per slot (524288 total) 
llama.cpp splits -c across -np slots; to give each of the 2 slots the full
262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB
unified pool. Update env default, serve-script fallback, and the deployment
doc line.
 2026-08-18 08:01:17 -07:00
Frank 1ba9618fcc Swap fleet naming + restore Qwen3.6-35B-A3B Q6_K_P on Dankserver 
- Restore the pre-2026-08-17 Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
  Q6_K_P + ngram-mod setup as frank-default-small on Dankserver (port 7770,
  upstream llama.cpp b10121, MoE experts on CPU via --n-cpu-moe 36).
- Rename the CIRU Qwen3.8-27B ActiveFPX setup from frank-default-small to
  frank-default-large (Dankclaw, port 52625); bump to CTX_SIZE=262144
  (native max) and PARALLEL=2 (96 GiB unified pool).
- Retire the Qwen3.8-27B IQ4_XS FastMTP frank-default-large (Dankserver) to
  .AGENTTRASH.
- deploy.sh: SERVICE[dankserver]=frank-default-small,
  SERVICE[dankclaw]=frank-default-large; HEALTH_WAIT[dankserver]=600.
 2026-08-18 00:32:26 -07:00
Frank 27788f6f2c fix: make serve scripts + deploy.sh executable 2026-08-17 15:24:40 -07:00
Frank d9af7f1d19 fleet redo 2026-08-17: retire PP+NPU, new topology 
- Move all previous scripts to old/scripts/ (retired PP/NPU-era)
- frank-default-large: llama.cpp FastMTP on dankserver (HauhauCS
  Qwen3.8-27B IQ4_XS + FastMTP 32K draft + ngram-mod, K q8_0/V f16,
  128K ctx, batch 2048/ubatch 1024, port 7770)
- frank-default-small: vLLM ROCm on dankclaw (Avesed Qwen3.6-35B-A3B
  INT8 W8A8, full 262144 ctx, no KV quant, 4 seqs, batch 16384,
  ngram spec, port 52625)
- retarget deploy.sh (dankserver/dankclaw, inline sudo, no helpers)
- docs: deployment + index + AGENTS.md + README for new topology,
  superseded banners on retired docs, add frank-redo-2026-08-17 runbook
 2026-08-17 15:23:46 -07:00