Frank
e5b731216c
`systemctl status` said `2x262144 slots over one 524288 pool, unquantized f16 K+V, MTP off` — three payload eras and two reslices out of date, and still wrong about the thing that decides whether the endpoint can be trusted (the KV dtype and whether the drafter is armed). It now quotes the env: 3x200704 over one 602112 pool, f16 K + q8_0 V, MTP armed, adapter, Q8_0 vision. Touching the unit selects the service in the planner, so this deliberately does NOT come with a deploy: the hub is serving live conversations. The revision marker stays behind until the next real frank-zealot change.
27 lines
1.1 KiB
Desktop File
27 lines
1.1 KiB
Desktop File
[Unit]
|
|
Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF + refusal adapter (qwen35 hybrid, 3x200704 slots over one 602112 pool, f16 K + q8_0 V, MTP armed, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
|
# though the weights themselves are copied into VRAM.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
|
|
# llama.cpp can take longer than systemd's 90 s default to drain slots and
|
|
# release the card: a real stop on 2026-10-18 hit `State 'stop-sigterm' timed
|
|
# out. Killing.` and left the unit in `failed` (result 'timeout') even though
|
|
# the process had served health correctly. The engine is SIGKILL-safe (nothing
|
|
# on disk), but a unit that always ends `failed` is a false alarm for the next
|
|
# operator, so give the drain room.
|
|
TimeoutStopSec=180
|
|
Restart=on-failure
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|