feat: frank-default-small = HauhauCS Qwen3.6-35B-A3B uncensored Q6_K_P + DFlash on dankserver

llama.cpp service replacing retired vllm-awq-qwen36 on port 7770 with alias frank-default-small. Wrapper sources frank-default-small.env for tunables: 3 parallel slots at 262144 ctx, draft-dflash n-max 7, KV K=q8_0 V=f16 on GPU, all MoE experts on CPU (--n-cpu-moe 40), non-expert tensors on GPU, mmproj vision + jinja. Unit: frank-default-small.service (ExecStart in dankserver repo clone /home/dank/vLLM/scripts).
2026-08-11 00:57:04 -07:00
parent 903d54c389
commit f93096f2bf
3 changed files with 99 additions and 0 deletions
@@ -0,0 +1,18 @@
+# Tunables for frank-default-small (HauhauCS Q6_K_P + DFlash draft, dankserver)
+# Runtime: llama-server b10108; RTX 3090 24GB; Ryzen 7 7700X; 62GB RAM.
+MODEL_DIR=/home/dank/models
+ALIAS=frank-default-small
+HOST=0.0.0.0
+PORT=7770
+CTX_SIZE=262144
+PARALLEL=3
+SPEC_TYPE=draft-dflash
+SPEC_N_MAX=7
+CACHE_TYPE_K=q8_0
+CACHE_TYPE_V=f16
+# All 40 layers keep their MoE expert weights on CPU (targeted expert offload).
+N_CPU_MOE=40
+# Every non-expert tensor (attention, norms, embedding, LM head) goes to GPU.
+N_GPU_LAYERS=all
+THREADS=8
+THREADS_HTTP=8