dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-large: roll back to gufo Qwen3.8-27B Q4_K_XL, one session x 262144 f16 KV

Browse Source 
frank-small: llama.cpp CUDA (master, sm_86) on MiMo Q6_K GGUF, 2 x 262144,
asymmetric KV (f16 K + q8_0 V, flash-attn pinned for the V quant)
This commit is contained in:
Frank committed 2026-10-01 22:40:59 -07:00
1 parent e452a1afbf
commit 8551bb006b
11 files changed
+729 -816
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/frank-small.service
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=SGLang MiMo-V2.6-Distill-Qwen-9B-INT8 (qwen3_5 hybrid, in-checkpoint vision, no MTP) 4x151k fp8 KV (dankclaw, frank-small)
Description=llama.cpp MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (qwen35 hybrid, 2x262144, f16 K + q8_0 V) on the RTX 3090 (dankclaw, frank-small)
After=network-online.target
Wants=network-online.target