feat: frank-default-small = HauhauCS Qwen3.6-35B-A3B uncensored Q6_K_P + DFlash on dankserver
llama.cpp service replacing retired vllm-awq-qwen36 on port 7770 with alias frank-default-small. Wrapper sources frank-default-small.env for tunables: 3 parallel slots at 262144 ctx, draft-dflash n-max 7, KV K=q8_0 V=f16 on GPU, all MoE experts on CPU (--n-cpu-moe 40), non-expert tensors on GPU, mmproj vision + jinja. Unit: frank-default-small.service (ExecStart in dankserver repo clone /home/dank/vLLM/scripts).
This commit is contained in:
1 parent
903d54c389
commit
f93096f2bf
3 files changed
+99
No files matched your search
@@ -0,0 +1,14 @@
|
||||
[Unit]
|
||||
Description=llama.cpp DFlash inference (frank-default-small)
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
ExecStart=/home/dank/vLLM/scripts/serve-llama-frank-small.sh
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Reference in new issue
Block a user