dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-small: pool 540672 (3 x 180224) so the card keeps ~2.7 GiB free

Browse Source 
Measured after the first 3-slot deploy: 3 x 196,608 booted (n_slots=3,
n_ctx_slot=196608, multimodal loaded) but used 22,972/24,576 MiB - 1,153 MiB
free, outside the 2-3 GiB band. The two measurements now recorded in
frank-small.env price the vision tower at ~1.4 GiB (non-KV 7,390 -> 8,859
MiB), so the pool drops 49,152 tokens (16,384/slot, 1,176 MiB) to land ~2.7
GiB free. The bats budget gate is recalibrated to the measured 9 GiB non-KV
cost, which now rejects the 3 x 196,608 shape instead of blessing it.
This commit is contained in:
Frank committed 2026-10-02 03:17:35 -07:00
1 parent a02850203d
commit 7fe881de01
4 files changed
+44 -37
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/frank-small.service
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=llama.cpp MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF (qwen35 hybrid, 3x196608, f16 K + q8_0 V, mmproj vision) on the RTX 3090 (dankclaw, frank-small)
Description=llama.cpp MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF (qwen35 hybrid, 3x180224, f16 K + q8_0 V, mmproj vision) on the RTX 3090 (dankclaw, frank-small)
After=network-online.target
Wants=network-online.target