dank/command-center
1
0 Fork 0
Code Issues 11 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

agent: repoint FLM-NPU model roles off the retired provider; sync memory/skills

Browse Source 
config.yml: vision -> llamacpp (dankclaw)/frank-large:auto,
tiny -> llamacpp (dankserver)/frank-small:off,
task -> llamacpp (dankclaw)/frank-large:medium (was flm dankclaw-npu),
matching the flm (dankclaw-npu) provider removal in models.yml.

Memories/managed skills: rollouts and learned notes from the frank-large
KV-pool + sampler work and the noctalia recolor; merged the duplicate
noctalia-palette-recolor skill into noctalia-custom-palette-recolor (adds
the dotfiles-tracking step and the +show-config delta gotcha) and dropped
the superseded legacy memory skills whose ghostty USR1 reload recipe is
wrong for this fleet.
This commit is contained in:
Frank committed 2026-09-13 23:51:43 -07:00
1 parent c32584e093
commit 872e86399a
56 files changed
+318 -178
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

agent/managed-skills/halogen-kv-pool-fit/SKILL.md
+13 -77
View File
@@ -3,88 +3,24 @@ name: halogen-kv-pool-fit
description: "Resize halogen-flash-server's KV pool or prefill arena without getting silently clamped or OOMing the boot: pool, arena and vision share one device budget, HALOGEN_HOST_RESERVE_GIB only scales the fit-check ceiling and is NOT an allocation, HALOGEN_KV_POOL_FIT only ever lowers, and the sizing warning prints only when a clamp happened."
---
# Sizing a halogen-flash-server KV pool
# halogen-flash-server KV pool / arena resizing
`HALOGEN_KV_POOL_FIT=1` (default) computes a device budget at startup and **only ever lowers** the configured `HALOGEN_KV_POOL_POSITIONS`. It never raises it, and never below the context.
One device budget covers KV pool + prefill arena + vision tower:
```
device budget = host RAM - resident weights - HALOGEN_HOST_RESERVE_GIB
fit check = needs budget >= (KV + arena + vision) + 1.5 GiB margin
KV = 28 KiB/position
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384) # measured, linear
vision = 1.97 GiB when HALOGEN_VISION_TOWER=1
budget = host RAM (121.2) - resident weights (67.7) - HALOGEN_HOST_RESERVE_GIB
need = KV (~28 KiB/position, ~11.0 GiB at 401408 as reserved) + arena + vision (1.97)
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384)
```
Prefer the engine's printed requirement over the model: a clamped boot states
it directly, e.g. `524288 positions need ~35.0 GiB (plus 1.5 of margin)`.
Measured ledger (Dankclaw, engine's own startup print): 262144/32768/r24 → 96.3 held, 12.1 left, 2081 blocks; 524288/16384/r24 → 94.6/13.8/3742; 524288/32768/r15 → 103.5/7.1/310 (stall NOTE prints); 524288/32768/r16 → identical (reserve is NOT an allocation); **401408/32768/r15 (deployed 2026-09-13) → 100.0/9.1/1396, NOTE gone**.
## The reserve is NOT an allocation
Procedure:
1. Edit `HALOGEN_KV_POOL_POSITIONS` / `HALOGEN_MAX_TOK` in `dankclaw/frank-large/quadlet/frank-large.container` only. Never raise `HALOGEN_HOST_RESERVE_GIB` — it only moves the fit-check ceiling (at 524288 the ceiling was 16; engine needs budget ≥ 36.5 there; 18 clamps to 262144).
2. Over budget → `HALOGEN_KV_POOL_FIT` silently LOWERS the pool and deploy still reports healthy. Way over → arena alone OOMs the boot while the unit still reads ACTIVE (max_tok 65536 did this). Always `podman logs frank-large | grep 'kv pool:'` and match the reserved count to the requested one; /health proves nothing about pool size.
3. Under budget → nothing prints; absence of WARNING is not evidence of headroom.
4. After restart, FIRST request can silence the engine ~45-60 s ("has not answered PING", /health engine_unresponsive) — one-time pool first-touch, self-recovers; HealthStartPeriod=300s covers it. Do not restart the unit. A completion retried after that window succeeds instantly.
`HALOGEN_HOST_RESERVE_GIB` is a term in the fit-check inequality, nothing more. Proved by A/B: reserve 15 and reserve 16 both held **103.5 GiB** with **7.1 GiB left for everything else** — identical footprint. Actual footprint is `weights + KV + arena + vision`, none of which the reserve touches.
To free host memory: cut the pool (262144 hands back ~3.6 from 401408) or the arena (MAX_TOK 16384 = 8.4). Host-left gain comes in below the KV delta (engine keeps freed space as working slack): the 401408 trim modelled +3.4, measured +2.0 GiB but 4.5x contiguous blocks.
So the knob has only two regimes: below the pool's requirement it changes nothing observable; above it, it silently deletes the pool. **Raising the reserve can never give memory back to co-tenants.** To free host RAM, cut the pool or the arena:
| change | RAM handed back |
|---|---|
| `HALOGEN_KV_POOL_POSITIONS` 524288 → 262144 | 7.2 GiB |
| `HALOGEN_MAX_TOK` 32768 → 16384 | 8.4 GiB |
Lowering the reserve DOES work — it buys fit margin for a bigger pool or arena. On a 121.2 GiB host with 67.7 GiB weights, for 524288 + max_tok 32768 (needs 36.5):
| reserve | budget | vs 36.5 | outcome |
|---|---|---|---|
| 24 | 29.5 | −7.0 | clamps |
| **18** | **35.5** | **−1.0** | **clamps — looks like a healthy deploy** |
| 17 | 36.5 | +0.02 | knife edge, inside the noise of the weights term |
| 16 | 37.5 | +1.0 | fits, zero benefit over 15 |
| **15** | **38.5** | +2.0 | **deployed** |
Ceiling here is 16; pick the lowest reserve that fits, not the highest.
## Gotchas
- **The sizing block and its `WARNING` print ONLY after a clamp.** A fitting boot prints no `kv pool:` lines at all, so absence of a warning in old logs is NOT evidence of headroom.
- **`deploy.sh` health-passes a clamped pool.** Container `active`, `/health` 200, in ~12 s. Only the ledger proves the pool size.
- **Beware the false positive when grepping for a clamp**: the NOTE line contains the sentence "Lowering HALOGEN_KV_POOL_POSITIONS gives some back", which matches `LOWERING` case-insensitively. Grep `LOWERING THE POOL` / `cannot spare it`.
- **`free`/`MemAvailable` cannot see this server.** Weights are pinned and the kernel counts locked pages as reclaimable page cache — `MemAvailable` read ~74 GiB while the engine held 103.5 of 121.2 GiB. Read the engine's lines instead.
- **Host RAM is rounded DOWN when displayed.** `121` is `MemTotal 127104060 kB` = `121.2159`; subtracting displayed figures gives 29.3 instead of the true 29.5.
- **Vendor "on the device" figures are pool + arena totals, not pool.** Reading them as pool-only makes a doubling look affordable.
- **A 2.0 GiB firmware iGPU carve-out** is reported by the ledger and is not free memory.
- `HALOGEN_MAX_TOK` is the arena / per-request generation budget (thinking included) — do not conflate with `HALOGEN_CTX` or the pool. `HALOGEN_KV_SLOTS` costs only ~111 MiB/slot, irrelevant to sizing.
## Three failure modes, worst last
1. **Silent clamp** — need exceeds budget; pool drops to the floor, service healthy. Caught only in the ledger.
2. **Boot OOM** — enough `HALOGEN_MAX_TOK` and the arena alone beats the budget: `dmalloc: FAILED requesting 0.250 GiB after 39.970 GiB in 566 allocations (out of memory)`. Container stays `active / Up (starting)` forever; health never passes.
3. **Minutes-long stall** — reserve cut so far that the engine holds most of a shared host. It prints its own note: neighbours and this server "can stall for minutes at a time at full CPU with no disk activity and no output, which is not a crash and needs no restart." Looks like a hang, invisible to `free`, hits co-tenants too.
## Verify
```
podman logs frank-large 2>&1 | grep -aiE 'for the device|reserving the KV|ONE pool of|memory: [0-9.]+ GiB of weights|left for everything|LOWERING THE POOL|cannot spare it|listening on 127'
```
Expect the budget line, `reserving the KV pool, <N> positions`, `ONE pool of <N> positions`, zero clamp matches, a `host memory left for everything else` figure, then `listening on 127.0.0.1:8730`. (A failing 7770 publish on restart is a stale-netns race — self-heals; `podman network reload --all` forces it.)
Then a real completion — `/health` 200 alone proves nothing:
```
curl -s http://127.0.0.1:7770/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"frank-large","messages":[{"role":"user","content":"Reply with exactly: OK"}],"max_tokens":48,"reasoning_effort":"low"}'
```
Expect `finish_reason: stop`. A silent client decodes **greedy** — this server ships no sampler defaults.
## Reference points (121.2 GiB host, 67.7 GiB weights, vision on)
| pool | max_tok | reserve | need | budget | outcome |
|---|---|---|---|---|---|
| 262144 | 32768 | 24 | 29.8 | 29.5 | boots (0.3 over), no sizing lines |
| 524288 | 16384 | 24 | 28.6 | 29.5 | fits clean |
| 524288 | 32768 | 24 | 37.0 | 29.5 | clamped, deploy green |
| 524288 | 65536 | 24 | 46.5 | 29.5 | OOM, will not boot |
| 524288 | 32768 | 15 | 37.0 | 38.5 | **both features, deployed** |
| 524288 | 32768 | 16 | 37.0 | 37.5 | fits, footprint identical |
## Procedure
Compute `need` vs `budget` with unrounded RAM, then confirm against the engine's own printed requirement. After each deploy read the ledger for actual pool, clamp matches, **and the host-left line** — and A/B it: if two settings report the same `held`, the knob you turned doesn't do what its name says. Keep reserve and pool documented as a matched pair. Roll back with `git revert` (never `--no-verify`).
Omp client config on blankpad: frank-large `maxTokens` must be ≤ 65536 (server 400s above the cap) and prompt+max_tokens must fit the pool or the request queues 3600 s then 503s.