agent: repoint FLM-NPU model roles off the retired provider; sync memory/skills
config.yml: vision -> llamacpp (dankclaw)/frank-large:auto, tiny -> llamacpp (dankserver)/frank-small:off, task -> llamacpp (dankclaw)/frank-large:medium (was flm dankclaw-npu), matching the flm (dankclaw-npu) provider removal in models.yml. Memories/managed skills: rollouts and learned notes from the frank-large KV-pool + sampler work and the noctalia recolor; merged the duplicate noctalia-palette-recolor skill into noctalia-custom-palette-recolor (adds the dotfiles-tracking step and the +show-config delta gotcha) and dropped the superseded legacy memory skills whose ghostty USR1 reload recipe is wrong for this fleet.
This commit is contained in:
Frank committed
1 parent
c32584e093
commit
872e86399a
56 files changed
+318
-178
No files matched your search
@@ -3,88 +3,24 @@ name: halogen-kv-pool-fit
|
||||
description: "Resize halogen-flash-server's KV pool or prefill arena without getting silently clamped or OOMing the boot: pool, arena and vision share one device budget, HALOGEN_HOST_RESERVE_GIB only scales the fit-check ceiling and is NOT an allocation, HALOGEN_KV_POOL_FIT only ever lowers, and the sizing warning prints only when a clamp happened."
|
||||
---
|
||||
|
||||
# Sizing a halogen-flash-server KV pool
|
||||
# halogen-flash-server KV pool / arena resizing
|
||||
|
||||
`HALOGEN_KV_POOL_FIT=1` (default) computes a device budget at startup and **only ever lowers** the configured `HALOGEN_KV_POOL_POSITIONS`. It never raises it, and never below the context.
|
||||
One device budget covers KV pool + prefill arena + vision tower:
|
||||
|
||||
```
|
||||
device budget = host RAM - resident weights - HALOGEN_HOST_RESERVE_GIB
|
||||
fit check = needs budget >= (KV + arena + vision) + 1.5 GiB margin
|
||||
KV = 28 KiB/position
|
||||
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384) # measured, linear
|
||||
vision = 1.97 GiB when HALOGEN_VISION_TOWER=1
|
||||
budget = host RAM (121.2) - resident weights (67.7) - HALOGEN_HOST_RESERVE_GIB
|
||||
need = KV (~28 KiB/position, ~11.0 GiB at 401408 as reserved) + arena + vision (1.97)
|
||||
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384)
|
||||
```
|
||||
|
||||
Prefer the engine's printed requirement over the model: a clamped boot states
|
||||
it directly, e.g. `524288 positions need ~35.0 GiB (plus 1.5 of margin)`.
|
||||
Measured ledger (Dankclaw, engine's own startup print): 262144/32768/r24 → 96.3 held, 12.1 left, 2081 blocks; 524288/16384/r24 → 94.6/13.8/3742; 524288/32768/r15 → 103.5/7.1/310 (stall NOTE prints); 524288/32768/r16 → identical (reserve is NOT an allocation); **401408/32768/r15 (deployed 2026-09-13) → 100.0/9.1/1396, NOTE gone**.
|
||||
|
||||
## The reserve is NOT an allocation
|
||||
Procedure:
|
||||
1. Edit `HALOGEN_KV_POOL_POSITIONS` / `HALOGEN_MAX_TOK` in `dankclaw/frank-large/quadlet/frank-large.container` only. Never raise `HALOGEN_HOST_RESERVE_GIB` — it only moves the fit-check ceiling (at 524288 the ceiling was 16; engine needs budget ≥ 36.5 there; 18 clamps to 262144).
|
||||
2. Over budget → `HALOGEN_KV_POOL_FIT` silently LOWERS the pool and deploy still reports healthy. Way over → arena alone OOMs the boot while the unit still reads ACTIVE (max_tok 65536 did this). Always `podman logs frank-large | grep 'kv pool:'` and match the reserved count to the requested one; /health proves nothing about pool size.
|
||||
3. Under budget → nothing prints; absence of WARNING is not evidence of headroom.
|
||||
4. After restart, FIRST request can silence the engine ~45-60 s ("has not answered PING", /health engine_unresponsive) — one-time pool first-touch, self-recovers; HealthStartPeriod=300s covers it. Do not restart the unit. A completion retried after that window succeeds instantly.
|
||||
|
||||
`HALOGEN_HOST_RESERVE_GIB` is a term in the fit-check inequality, nothing more. Proved by A/B: reserve 15 and reserve 16 both held **103.5 GiB** with **7.1 GiB left for everything else** — identical footprint. Actual footprint is `weights + KV + arena + vision`, none of which the reserve touches.
|
||||
To free host memory: cut the pool (262144 hands back ~3.6 from 401408) or the arena (MAX_TOK 16384 = 8.4). Host-left gain comes in below the KV delta (engine keeps freed space as working slack): the 401408 trim modelled +3.4, measured +2.0 GiB but 4.5x contiguous blocks.
|
||||
|
||||
So the knob has only two regimes: below the pool's requirement it changes nothing observable; above it, it silently deletes the pool. **Raising the reserve can never give memory back to co-tenants.** To free host RAM, cut the pool or the arena:
|
||||
|
||||
| change | RAM handed back |
|
||||
|---|---|
|
||||
| `HALOGEN_KV_POOL_POSITIONS` 524288 → 262144 | 7.2 GiB |
|
||||
| `HALOGEN_MAX_TOK` 32768 → 16384 | 8.4 GiB |
|
||||
|
||||
Lowering the reserve DOES work — it buys fit margin for a bigger pool or arena. On a 121.2 GiB host with 67.7 GiB weights, for 524288 + max_tok 32768 (needs 36.5):
|
||||
|
||||
| reserve | budget | vs 36.5 | outcome |
|
||||
|---|---|---|---|
|
||||
| 24 | 29.5 | −7.0 | clamps |
|
||||
| **18** | **35.5** | **−1.0** | **clamps — looks like a healthy deploy** |
|
||||
| 17 | 36.5 | +0.02 | knife edge, inside the noise of the weights term |
|
||||
| 16 | 37.5 | +1.0 | fits, zero benefit over 15 |
|
||||
| **15** | **38.5** | +2.0 | **deployed** |
|
||||
|
||||
Ceiling here is 16; pick the lowest reserve that fits, not the highest.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **The sizing block and its `WARNING` print ONLY after a clamp.** A fitting boot prints no `kv pool:` lines at all, so absence of a warning in old logs is NOT evidence of headroom.
|
||||
- **`deploy.sh` health-passes a clamped pool.** Container `active`, `/health` 200, in ~12 s. Only the ledger proves the pool size.
|
||||
- **Beware the false positive when grepping for a clamp**: the NOTE line contains the sentence "Lowering HALOGEN_KV_POOL_POSITIONS gives some back", which matches `LOWERING` case-insensitively. Grep `LOWERING THE POOL` / `cannot spare it`.
|
||||
- **`free`/`MemAvailable` cannot see this server.** Weights are pinned and the kernel counts locked pages as reclaimable page cache — `MemAvailable` read ~74 GiB while the engine held 103.5 of 121.2 GiB. Read the engine's lines instead.
|
||||
- **Host RAM is rounded DOWN when displayed.** `121` is `MemTotal 127104060 kB` = `121.2159`; subtracting displayed figures gives 29.3 instead of the true 29.5.
|
||||
- **Vendor "on the device" figures are pool + arena totals, not pool.** Reading them as pool-only makes a doubling look affordable.
|
||||
- **A 2.0 GiB firmware iGPU carve-out** is reported by the ledger and is not free memory.
|
||||
- `HALOGEN_MAX_TOK` is the arena / per-request generation budget (thinking included) — do not conflate with `HALOGEN_CTX` or the pool. `HALOGEN_KV_SLOTS` costs only ~111 MiB/slot, irrelevant to sizing.
|
||||
|
||||
## Three failure modes, worst last
|
||||
|
||||
1. **Silent clamp** — need exceeds budget; pool drops to the floor, service healthy. Caught only in the ledger.
|
||||
2. **Boot OOM** — enough `HALOGEN_MAX_TOK` and the arena alone beats the budget: `dmalloc: FAILED requesting 0.250 GiB after 39.970 GiB in 566 allocations (out of memory)`. Container stays `active / Up (starting)` forever; health never passes.
|
||||
3. **Minutes-long stall** — reserve cut so far that the engine holds most of a shared host. It prints its own note: neighbours and this server "can stall for minutes at a time at full CPU with no disk activity and no output, which is not a crash and needs no restart." Looks like a hang, invisible to `free`, hits co-tenants too.
|
||||
|
||||
## Verify
|
||||
|
||||
```
|
||||
podman logs frank-large 2>&1 | grep -aiE 'for the device|reserving the KV|ONE pool of|memory: [0-9.]+ GiB of weights|left for everything|LOWERING THE POOL|cannot spare it|listening on 127'
|
||||
```
|
||||
|
||||
Expect the budget line, `reserving the KV pool, <N> positions`, `ONE pool of <N> positions`, zero clamp matches, a `host memory left for everything else` figure, then `listening on 127.0.0.1:8730`. (A failing 7770 publish on restart is a stale-netns race — self-heals; `podman network reload --all` forces it.)
|
||||
|
||||
Then a real completion — `/health` 200 alone proves nothing:
|
||||
|
||||
```
|
||||
curl -s http://127.0.0.1:7770/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"frank-large","messages":[{"role":"user","content":"Reply with exactly: OK"}],"max_tokens":48,"reasoning_effort":"low"}'
|
||||
```
|
||||
|
||||
Expect `finish_reason: stop`. A silent client decodes **greedy** — this server ships no sampler defaults.
|
||||
|
||||
## Reference points (121.2 GiB host, 67.7 GiB weights, vision on)
|
||||
|
||||
| pool | max_tok | reserve | need | budget | outcome |
|
||||
|---|---|---|---|---|---|
|
||||
| 262144 | 32768 | 24 | 29.8 | 29.5 | boots (0.3 over), no sizing lines |
|
||||
| 524288 | 16384 | 24 | 28.6 | 29.5 | fits clean |
|
||||
| 524288 | 32768 | 24 | 37.0 | 29.5 | clamped, deploy green |
|
||||
| 524288 | 65536 | 24 | 46.5 | 29.5 | OOM, will not boot |
|
||||
| 524288 | 32768 | 15 | 37.0 | 38.5 | **both features, deployed** |
|
||||
| 524288 | 32768 | 16 | 37.0 | 37.5 | fits, footprint identical |
|
||||
|
||||
## Procedure
|
||||
|
||||
Compute `need` vs `budget` with unrounded RAM, then confirm against the engine's own printed requirement. After each deploy read the ledger for actual pool, clamp matches, **and the host-left line** — and A/B it: if two settings report the same `held`, the knob you turned doesn't do what its name says. Keep reserve and pool documented as a matched pair. Roll back with `git revert` (never `--no-verify`).
|
||||
Omp client config on blankpad: frank-large `maxTokens` must be ≤ 65536 (server 400s above the cap) and prompt+max_tokens must fit the pool or the request queues 3600 s then 503s.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: noctalia-custom-palette-recolor
|
||||
description: "Recolor a live Noctalia desktop (accent, terminal palette, terminal foreground) by cutting over to a custom palette instead of editing the community-palette cache, then re-rendering and reloading templates."
|
||||
---
|
||||
|
||||
# Noctalia custom-palette recolor
|
||||
|
||||
Recolor the running Noctalia shell + all themed apps without restarting the session.
|
||||
|
||||
## Why custom, never the community cache
|
||||
- `~/.local/state/noctalia/community-palettes/*.json` is a **sync cache**. `~/.local/state/noctalia/community-palettes/.catalog/palettes.json` carries an `md5` per palette and noctalia re-fetches from `https://api.noctalia.dev/palettes`. Edits there get reverted.
|
||||
- Custom palettes live at `~/.config/noctalia/palettes/<Name>.json` (`NOCTALIA_CONFIG_HOME` then `XDG_CONFIG_HOME` honored) and are never synced. Missing/invalid file → silent fallback to the builtin palette.
|
||||
- Palette JSON shape = the community file's shape: `dark`/`light` maps of `mPrimary`, `mOnPrimary`, `mSecondary`, `mTertiary`, `mError`, `mSurface`, `mSurfaceVariant`, `mOutline`, `mShadow`, `mHover` plus a `terminal` block (`foreground`, `background`, `normal.{black..white}`, `bright.{...}`, `cursor`, `cursorText`, `selectionFg`, `selectionBg`). NOTE: `noctalia theme -o` emits M3 snake_case tokens (`on_primary_container`, …) — that is the *generator* format, not this one. Copy an existing community JSON and mutate it with `jq`.
|
||||
|
||||
## Procedure
|
||||
1. Read the live scheme first — never assume: `noctalia msg color-scheme-get` → e.g. `community Gruber Darker`; `noctalia msg theme-mode-get`.
|
||||
2. Back up settings: `cp -a ~/.local/state/noctalia/settings.toml /tmp/noctalia-settings-backup-$(date +%s).toml`.
|
||||
3. Build the palette from the current one (jq keeps structure; no hand-editing 2 KB of JSON):
|
||||
```sh
|
||||
mkdir -p ~/.config/noctalia/palettes
|
||||
jq '.dark.mPrimary = "#f43841"
|
||||
| .dark.terminal.normal.yellow = "#f43841"
|
||||
| .dark.terminal.bright.yellow = "#ff4f58"
|
||||
| .dark.terminal.foreground = "#f43841"' \
|
||||
"$HOME/.local/state/noctalia/community-palettes/Gruber%20Darker.json" \
|
||||
> ~/.config/noctalia/palettes/"Gruber Red.json"
|
||||
```
|
||||
Prefer a hex already present in the palette (its own `mError` red / accent) so the theme stays internally consistent. Set BOTH `normal` and `bright` variants, and both `dark` and `light` if the user toggles modes.
|
||||
4. Switch (persists `theme.source = custom` + `theme.custom_palette`):
|
||||
`noctalia msg color-scheme-set custom "Gruber Red"` → `ok`; confirm with `noctalia msg color-scheme-get`.
|
||||
5. Re-render every configured template: `noctalia msg templates-apply`. Regenerates `~/.config/ghostty/themes/noctalia`, `~/.config/gtk-{3,4}.0/noctalia.css`, kcolorscheme, foot, wezterm, starship, etc. Some of those paths are symlinked into the dotfiles repo → they show up as modified files worth committing.
|
||||
6. Reload surfaces:
|
||||
- Compositor (focus border, `noctalia.kdl`): `niri msg action load-config-file`. `reload-config` is NOT a valid niri subcommand.
|
||||
- Ghostty: see the reload trap below.
|
||||
|
||||
## Verification
|
||||
- `ghostty +show-config | grep -E '^(foreground|palette)'` — proves the theme file resolves into the config (e.g. `foreground = #f43841`, `palette 3/11` red).
|
||||
- `noctalia config validate` → `✓ Config is valid`.
|
||||
- Visual: `noctalia msg screenshot-fullscreen` saves to `~/Pictures/screenshot_<ts>.png` (it ALSO overwrites the clipboard with the PNG — warn the user). `read` that PNG to confirm accent/toggle/border color.
|
||||
- `grep -c <old-hex> ~/.config/noctalia/palettes/*.json` → 0 for no leftovers.
|
||||
|
||||
## Track it in the dotfiles repo (command-center `~/.omp`)
|
||||
`scripts/deploy.sh` is a stow-equivalent: packages are home-relative trees under
|
||||
`dotfiles/<pkg>/`, and deploying replaces live files with symlinks into the repo. Mirror that
|
||||
shape for the new palette so `git pull` + `deploy.sh` is the whole propagation story:
|
||||
```sh
|
||||
mkdir -p ~/.omp/dotfiles/noctalia/.config/noctalia/palettes
|
||||
mv ~/.config/noctalia/palettes/"<Name>.json" ~/.omp/dotfiles/noctalia/.config/noctalia/palettes/
|
||||
ln -s "$HOME/.omp/dotfiles/noctalia/.config/noctalia/palettes/<Name>.json" \
|
||||
~/.config/noctalia/palettes/"<Name>.json"
|
||||
noctalia msg color-scheme-set custom "<Name>" # re-reads through the symlink; expect ok
|
||||
```
|
||||
Do the link by hand for one file instead of running `deploy.sh` — sync mode quarantines and
|
||||
replaces every divergent live file, which is broader than a single-palette change.
|
||||
|
||||
## Traps
|
||||
- **NEVER `kill -USR1` ghostty** to reload. It does not reload config on this fleet — it kills/respawns the terminal (took out Frank's window 2026-09-13; he said "stop killing the terminal"). Correct: the default keybind `ctrl+shift+,` = `reload_config`, per `ghostty +show-config --default`. Existing windows keep old colors until reloaded; new windows inherit immediately.
|
||||
- Palette hex values are NOT a proxy for what the user sees: in `Gruber Darker` the *accent* (`mPrimary`) is yellow `#ffdd33`, so "change the yellow" means the accent too, not just terminal index 3.
|
||||
- Terminal text turned into a saturated color makes ANSI 1/9 indistinguishable from default text (grep highlights flatten) — flag it and offer a lighter red.
|
||||
- Contrast sanity: `#f43841` on `#181818` ≈ 4.6:1 (dark-on-color beats white-on-color here, ~3.9:1, so keep `mOnPrimary` dark).
|
||||
- Revert = `noctalia msg color-scheme-set community "<original name>"`; the community cache is untouched by this whole flow.
|
||||
- `ghostty +show-config` prints only the delta from Ghostty defaults — a key whose value equals
|
||||
the default (e.g. `font-size = 12`) simply does not appear. Cross-check with
|
||||
`ghostty +show-config --default` before concluding a key is unset.
|
||||
- Generated theme files that are symlinked into the dotfiles repo (`gtk-3.0`/`gtk-4.0`
|
||||
`noctalia.css`, ghostty theme) show up as tracked modifications after `templates-apply`;
|
||||
commit them together with the new palette JSON so every machine gets the same recolor.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: omp-per-model-sampler-config
|
||||
description: "Set per-model sampler params and preserve_thinking in OMP's models.yml via compat.extraBody, and verify the exact wire body with an echo-capture profile."
|
||||
---
|
||||
|
||||
# OMP Per-Model Sampler Config (`compat.extraBody`)
|
||||
|
||||
`~/.omp/agent/models.yml` has **no per-model sampler keys**. Sampling is global (`config.yml`: `temperature`, `topP`, `topK`, `minP`, `presencePenalty`, `repetitionPenalty`; default `-1` = **not sent**). The per-model hook is `compat.extraBody`.
|
||||
|
||||
## Why it works
|
||||
`applyOpenAIExtraBody` runs **last** in the openai-completions builder — after `applyChatCompletionsCompatPolicy` — via `Object.assign(params, extraBody)`. So extraBody overrides global settings and compat-policy output.
|
||||
|
||||
## Recipe
|
||||
Use **wire names**, not OMP camelCase:
|
||||
|
||||
```yaml
|
||||
- id: frank-large
|
||||
compat:
|
||||
reasoningEffortMap:
|
||||
max: xhigh
|
||||
extraBody:
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
min_p: 0.0
|
||||
presence_penalty: 0.0
|
||||
repetition_penalty: 1.0
|
||||
preserve_thinking: true
|
||||
```
|
||||
|
||||
## Two traps
|
||||
|
||||
1. **`preserve_thinking` is not user-settable.** `qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the *auto-detected* `thinkingFormat` + local baseUrl, **before** `applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in models.yml does **not** enable it. Custom-named models (`frank-large`) match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
||||
2. **Never put `chat_template_kwargs` in `extraBody`.** `Object.assign` is shallow — it replaces the whole object and silently drops the `reasoning_effort` kwarg the compat policy set. Top-level `preserve_thinking` is honoured by halogen; use that.
|
||||
|
||||
## Gotcha: silent greedy decode
|
||||
halogen-flash-server has **no server-side sampler defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the endpoint decodes greedy while looking configured.
|
||||
|
||||
## Verify the exact wire body (echo-capture)
|
||||
Don't trust the YAML — capture what OMP actually emits.
|
||||
|
||||
1. Echo server on `127.0.0.1:18099` that dumps the JSON body to a file and returns a canned `chat.completion` (include `reasoning_content` so thinking paths exercise).
|
||||
2. Copy the **real** models.yml into a throwaway profile with only the baseUrl swapped:
|
||||
```bash
|
||||
sed 's|http://<host>:7770/v1|http://127.0.0.1:18099/v1|' \
|
||||
~/.omp/agent/models.yml > ~/.omp/profiles/wiretest/agent/models.yml
|
||||
```
|
||||
3. Run and inspect:
|
||||
```bash
|
||||
omp --profile wiretest --model "<provider>/<model>" -p "hi" \
|
||||
--no-tools --no-session --no-extensions --no-skills --no-rules
|
||||
jq 'del(.messages, .tools)' last-body.json
|
||||
```
|
||||
Confirm every intended field is present with the right value.
|
||||
|
||||
## Verify a field is *honoured*, not just accepted
|
||||
Unknown fields often return 200 anyway. Prove behavior with a **prompt-token delta**: replay an assistant turn carrying a long `reasoning_content`, then compare `usage.prompt_tokens` with the flag true vs false. A large delta means the field changed template rendering; zero delta means it was ignored.
|
||||
|
||||
## Cleanup
|
||||
Trash the probe profile and echo server (`profiles/.AGENTTRASH`), never `rm`.
|
||||
Reference in new issue
Block a user