dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
61 Commits 67 Branches 8 Tags
bf64c0d9c28f2759960a007bbb5edd40f8f7deba
Commit Graph
6 Commits
Author SHA1 Message Date
Frank aa2f4bfe39 feat: frank-default-large → llama.cpp Qwen3.6-27B Q8_K_P + DFlash draft + ngram-mod (bench pending) 2026-08-11 13:47:37 -07:00
Frank 677b4af2a4 docs: frank-default-small now uses ngram-mod, DFlash retired 
- SWAT from DFlash (see ngram-mod benchmarks below); draft model
  not loaded. Active spec: --spec-type ngram-mod n_min=48/n_max=64/n_match=24.
- Update Dankserver specs, deployment.md, DFlash doc to ngram-mod.
- VRAM footprint: 14 GB (was 20 GB), ~10 GB free headroom.
 2026-08-11 02:41:33 -07:00
Frank 5c9e812221 docs: frank-default-small now 200000 ctx/stream, batch 3072/1536, --no-mmap 
Sync deployment.md, Dankserver specs, DFlash doc to the tuned config:
ctx reduced to 200000 per stream (pool 600000) to free KV VRAM for the larger
batch; --no-mmap eager RAM load; ctx-checkpoints 4; ~20GB VRAM / 32GB RAM.
models.json contextWindow -> 200000.
 2026-08-11 01:52:12 -07:00
Frank 35c44c4123 docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash) 
Retire vllm-awq-qwen36 (AWQ) from target maps, README, deployment, KB index,
Dankserver specs, DFlash and FP8 docs. New dankserver runtime: upstream
llama.cpp, port 7770, alias frank-default-small, 3x262144 ctx, DFlash Q8_0
draft, MoE experts offloaded to CPU, attention+KV on GPU (K q8_0, V f16).
deploy.sh now maps dankserver to frank-default-small and symlinks its unit.
models.json: dankserver baseUrl -> :7770, contextWindow -> 262144.
 2026-08-11 01:18:29 -07:00
Frank 903d54c389 docs: reflect Ling-3 vLLM (frank-default-large) on Dankclaw 
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired
ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770
with MTP speculative decoding. Align Dankserver to the running
vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs
(gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4
experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
 2026-08-10 23:37:27 -07:00
Frank 993b6af576 docs: add DFlash/DSpark speculative decoding research + quantization-by-hardware guide 
- DFlash DSpark Speculative Decoding.md: verified method details (DFlash
  block diffusion from Z Lab arXiv:2602.06036; DSpark semi-autoregressive from
  DeepSeek arXiv:2607.05147), runtime support matrix (vLLM/llama.cpp/SGLang),
  exact vLLM --speculative-config recipes (incl. dflash/dspark), llama.cpp
  --spec-type recipes with sidecar auto-discovery, draft model table
  (z-lab Qwen3.6-27B-DFlash 2B, Qwen3.6-35B-A3B-DFlash 0.4B), deployment
  guidance for RTX 3090 (AWQ+Dflash) and Strix Halo (ROCmFPX MTP), measured
  45-50 t/s MTP decode on Strix Halo
- Quantization by Hardware Architecture.md: FP8/INT8/INT4/GPTQ/AWQ/GGUF/FP4/
  ROCmFP4 support per NVIDIA (Turing/Ampere/Ada/Hopper/Blackwell), AMD
  (CDNA MI250/MI300/MI350, RDNA 3/3.5/4), and CPU; recommended picks per
  Dankserver (AWQ/GPTQ 4-bit) and Dankclaw (ROCmFP4/Q4_K_M)
- index.md: link both new pages under Speculative Decoding & Quantization
 2026-08-10 21:09:17 -07:00