dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
67 Commits 60 Branches 8 Tags
db3eb01495d9ae6f9220ba690e0d5d05f8668e37
Commit Graph
7 Commits
Author SHA1 Message Date
Frank e92c72fd11 docs: frank-default-large llama.cpp draft-dflash deployment + benchmark results; retire Ling vLLM references 2026-08-11 17:24:59 -07:00
Frank 4c5a4a5f3b docs: frank-default-small — 4 parallel slots, batch 8192/2048, n-cpu-moe 32 
Update Dankserver System Specs and deployment to reflect the 4-stream config:
- 4 parallel slots (800000 total ctx), batch 8192 / ubatch 2048
- n-cpu-moe 32 (8 expert layers back on GPU)
- ~870 MiB VRAM headroom under 4-stream load
- Knob documented: n-cpu-moe 36 gives ~1.7 GiB more headroom if needed
 2026-08-11 03:13:36 -07:00
Frank 677b4af2a4 docs: frank-default-small now uses ngram-mod, DFlash retired 
- SWAT from DFlash (see ngram-mod benchmarks below); draft model
  not loaded. Active spec: --spec-type ngram-mod n_min=48/n_max=64/n_match=24.
- Update Dankserver specs, deployment.md, DFlash doc to ngram-mod.
- VRAM footprint: 14 GB (was 20 GB), ~10 GB free headroom.
 2026-08-11 02:41:33 -07:00
Frank 5c9e812221 docs: frank-default-small now 200000 ctx/stream, batch 3072/1536, --no-mmap 
Sync deployment.md, Dankserver specs, DFlash doc to the tuned config:
ctx reduced to 200000 per stream (pool 600000) to free KV VRAM for the larger
batch; --no-mmap eager RAM load; ctx-checkpoints 4; ~20GB VRAM / 32GB RAM.
models.json contextWindow -> 200000.
 2026-08-11 01:52:12 -07:00
Frank 35c44c4123 docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash) 
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index,
Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream
llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0
draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16).
deploy.sh now maps dankserver to frank-default-small and symlinks its unit.
models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
 2026-08-11 01:18:29 -07:00
Frank 903d54c389 docs: reflect Ling-3 vLLM (frank-default-large) on Dankclaw 
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired
ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770
with MTP speculative decoding. Align Dankserver to the running
vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs
(gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4
experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
 2026-08-10 23:37:27 -07:00
Frank 90c4883f89 feat: add consistent deployment/update process (deploy.sh + docs + AGENTS.md) 
- scripts/deploy.sh: target-aware deploy (dankclaw/dankserver). Pushes main,
  pulls on the target, installs/enables systemd unit, restarts service, and
  checks the health endpoint; --dry-run preview; pre-flight dirty-tree check
- docs/deployment.md: full deployment/update process with mermaid workflow,
  target map, manual update commands, service units, and troubleshooting
- AGENTS.md: add Deployment & Update Process section (deploy.sh usage, target
  map, mermaid workflow) and a Summary table row
- docs/index.md: crosslink deployment doc
 2026-08-10 21:04:37 -07:00