Frankandomp
5278f5be26
Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 - the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer 12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the 401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj 879 + 248 MiB. Playwright gate: 33 passed. Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite caused (journal signature + the head -1 check), and that llama.cpp's "n_ctx_seq < n_ctx_train" line is informational when the context is BELOW the trained window - the Gemma4 drafter warning was the opposite case. The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/ draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed as its own restart: the next frank-secondary deploy relinks it. Co-Authored-By: omp <omp@franksdank.cloud>
20 lines
681 B
Desktop File
20 lines
681 B
Desktop File
[Unit]
|
|
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
|
# though the weights themselves are copied into VRAM.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_SECONDARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-secondary.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-secondary.sh
|
|
Restart=on-failure
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|