Frank
872e86399a
config.yml: vision -> llamacpp (dankclaw)/frank-large:auto, tiny -> llamacpp (dankserver)/frank-small:off, task -> llamacpp (dankclaw)/frank-large:medium (was flm dankclaw-npu), matching the flm (dankclaw-npu) provider removal in models.yml. Memories/managed skills: rollouts and learned notes from the frank-large KV-pool + sampler work and the noctalia recolor; merged the duplicate noctalia-palette-recolor skill into noctalia-custom-palette-recolor (adds the dotfiles-tracking step and the +show-config delta gotcha) and dropped the superseded legacy memory skills whose ghostty USR1 reload recipe is wrong for this fleet.
2.5 KiB
name, description
| name | description |
|---|---|
| halogen-kv-pool-fit | Resize halogen-flash-server's KV pool or prefill arena without getting silently clamped or OOMing the boot: pool, arena and vision share one device budget, HALOGEN_HOST_RESERVE_GIB only scales the fit-check ceiling and is NOT an allocation, HALOGEN_KV_POOL_FIT only ever lowers, and the sizing warning prints only when a clamp happened. |
halogen-flash-server KV pool / arena resizing
One device budget covers KV pool + prefill arena + vision tower:
budget = host RAM (121.2) - resident weights (67.7) - HALOGEN_HOST_RESERVE_GIB
need = KV (~28 KiB/position, ~11.0 GiB at 401408 as reserved) + arena + vision (1.97)
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384)
Measured ledger (Dankclaw, engine's own startup print): 262144/32768/r24 → 96.3 held, 12.1 left, 2081 blocks; 524288/16384/r24 → 94.6/13.8/3742; 524288/32768/r15 → 103.5/7.1/310 (stall NOTE prints); 524288/32768/r16 → identical (reserve is NOT an allocation); 401408/32768/r15 (deployed 2026-09-13) → 100.0/9.1/1396, NOTE gone.
Procedure:
- Edit
HALOGEN_KV_POOL_POSITIONS/HALOGEN_MAX_TOKindankclaw/frank-large/quadlet/frank-large.containeronly. Never raiseHALOGEN_HOST_RESERVE_GIB— it only moves the fit-check ceiling (at 524288 the ceiling was 16; engine needs budget ≥ 36.5 there; 18 clamps to 262144). - Over budget →
HALOGEN_KV_POOL_FITsilently LOWERS the pool and deploy still reports healthy. Way over → arena alone OOMs the boot while the unit still reads ACTIVE (max_tok 65536 did this). Alwayspodman logs frank-large | grep 'kv pool:'and match the reserved count to the requested one; /health proves nothing about pool size. - Under budget → nothing prints; absence of WARNING is not evidence of headroom.
- After restart, FIRST request can silence the engine ~45-60 s ("has not answered PING", /health engine_unresponsive) — one-time pool first-touch, self-recovers; HealthStartPeriod=300s covers it. Do not restart the unit. A completion retried after that window succeeds instantly.
To free host memory: cut the pool (262144 hands back ~3.6 from 401408) or the arena (MAX_TOK 16384 = 8.4). Host-left gain comes in below the KV delta (engine keeps freed space as working slack): the 401408 trim modelled +3.4, measured +2.0 GiB but 4.5x contiguous blocks.
Omp client config on blankpad: frank-large maxTokens must be ≤ 65536 (server 400s above the cap) and prompt+max_tokens must fit the pool or the request queues 3600 s then 503s.