chore: track live agent config (config, models, append-system, managed-skills, memories, plugin lock)

This commit is contained in:
Frank committed 2026-08-28 07:56:47 -07:00
1 parent 772607434a
commit 552d1ef8bf
23 files changed
+640

No files matched your search

@@ -0,0 +1,69 @@
---
name: bench-llama-server-params
description: "Benchmark llama.cpp server-side parameters (--threads-batch, --ubatch-size, --spec-type) using HTTP API with max_tokens=1 for prefill, max_tokens=16384+ for decode"
---
# Benchmarking llama.cpp Server-Side Parameters
## Problem
`llama-bench` benchmarks the bench binary itself, not the running server.
It cannot measure the effect of server-side parameters like `--threads-batch`,
`--ubatch-size`, `--spec-type`, or `--parallel`.
## Solution: HTTP-Based Benchmarking
### Prefill throughput (prompt processing)
1. Start the server with the target parameter value
2. Send a large prompt via the chat API with `max_tokens=1`
3. Measure `prompt_tokens / wall_time` from the response usage
`max_tokens=1` is correct here — you want only prompt processing time,
not generation. The server crashes on `max_tokens=0`.
### Decode throughput (generation)
1. Start the server with the target parameter value
2. Send a prompt that generates a long response
3. Use `max_tokens=16384` minimum (16k) — **never `max_tokens=1`** for decode
benchmarking. One token tells you nothing about sustained generation rate.
4. Measure `predicted_per_second` from the response `timings` object, or
compute `completion_tokens / wall_time`
### Why at least 16k tokens for decode
- Decode throughput varies over the generation (warmup, cache growth, MoE
routing). A single token is noise.
- Speculative decoding acceptance rates stabilize over thousands of tokens.
- Short generations finish before the draft head reaches steady state.
- 16k tokens gives a stable average at ~22 tok/s (≈12 min per run).
### Key Pattern
```python
# Decode benchmark — measure generation throughput
body = json.dumps({
"model": "<alias>",
"messages": [{"role": "user", "content": "<prompt that generates 10k+ tokens>"}],
"max_tokens": 16384, # 16k minimum for decode; use max_tokens=1 ONLY for prefill
"temperature": 0, # deterministic for reproducibility
"stream": False
})
# Measure wall time from request send to response receipt
# Get predicted_per_second from timings, or completion_tokens / wall_time
```
### When to Use
- `--threads-batch` (prefill): use `max_tokens=1`
- `--ubatch-size` (decode): use `max_tokens=16384+`
- `--spec-type` / speculative decoding: use `max_tokens=16384+` (decode only)
- Any server-side parameter that `llama-bench` cannot target
### Benchmarking policy (MANDATORY)
1. **STOP the production service** before running any benchmark
2. **Mask** the service (`systemctl mask`, never `disable` — on these hosts
`disable` deletes the unit file from `/etc/systemd/system/`)
3. Run the bench
4. **Unmask + restart** the service when done
5. **Verify health** before reporting results
### Example Scripts
- `scripts/bench-threads-batch-http.py` — prefill sweep (max_tokens=1)
- `scripts/bench-decode-threads-batch.py` — decode sweep (max_tokens=256+)
- `scripts/bench-spec-decode.py` — speculative decoding config sweep
@@ -0,0 +1,29 @@
---
name: quadlet-v1-generator-quirks
description: "When writing or deploying Quadlet .container units on podman 5.x (v1 generator, e.g. Dankclaw), avoid keys the v1 generator hard-fails on and verify generated ExecStart before starting; generated units cannot be enabled."
---
# Quadlet v1 generator quirks (podman 5.x)
The podman 5.x user generator (`/usr/lib/systemd/user-generators/podman-user-generator`, quadlet **v1**) behaves differently from the v4 docs. Verified live on Dankclaw 2026-08-28 while deploying the authentik trio.
## Hard-fail keys (unit silently fails to generate — `systemctl --user cat <unit>` shows "could not be found")
- Any dash-prefixed option line in `[Container]`: `--shm-size=512m` etc. → generator logs `unsupported key '--shm-size' in group 'Container'` and skips the whole unit. Omit them; accept podman defaults.
- `Command=` is unsupported — use `Exec=` (that is what this generator accepts).
## HealthCmd space-splitting
The v1 generator splits `HealthCmd` on spaces into the healthcheck argv array (`podman inspect --format {{.Config.Healthcheck.Test}} <ctr>` shows `[CMD-SHELL pg_isready -d ... ]`). A quoted probe like `/bin/sh -c "pg_isready ..."` lands as argv token `"pg_isready` → command-not-found → container reads **unhealthy while the DB is actually up**. Rule: HealthCmd must contain only space-free argv tokens, e.g. `HealthCmd=pg_isready -d authentik -U authentik` or `python /manage.py check --database default`.
## Generated units cannot be enabled
`systemctl --user enable <generated-unit>` fails with "unit is transient or generated". Boot persistence comes from `[Install] WantedBy=default.target` inside the `.container` file, honored by the generator at user-manager start; linger must stay on for the deploy user. Deploy handlers: install unit → `systemctl --user daemon-reload` → `systemctl --user start`; no `enable`.
## Probe a new key before shipping it
Drop a throwaway `probe.container` into `~/.config/containers/systemd/`, `systemctl --user daemon-reload`, then check the generator verdict:
```bash
journalctl --user --since "1 min ago" -o cat | grep -i "unsupported key"
systemctl --user cat probe.service # shows the generated ExecStart verbatim
rm ~/.config/containers/systemd/probe.container && systemctl --user daemon-reload
```
If a healthcheck ever reads unhealthy but logs look fine, suspect this quirk first: compare the baked `Healthcheck.Test` array against the intended command before believing the status.