live: heartbeat sync 2026-10-19T01:00:41-07:00 (Refs #11)
This commit is contained in:
Frank
@@ -1,82 +1,32 @@
|
||||
---
|
||||
name: bounding-subagent-runtime
|
||||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, agentIdleTtlMs, per-agent model overrides, the cfg save-approval quirk, and the finished-work-but-never-exits nested-fanout deadlock."
|
||||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, maxConcurrency, maxRecursionDepth (the 0-trap), per-provider in-flight ceilings."
|
||||
---
|
||||
|
||||
# Bounding subagent runtime (omp harness)
|
||||
## When
|
||||
Subagent runs long, fan out too wide, or you must (re)declare the fleet's subagent limit policy. Also when asked "what limits can subagents have".
|
||||
|
||||
Use when a `task`/`eval` subagent runs absurdly long (tens of minutes on a small job),
|
||||
when it **finishes its deliverable and still never exits**, when you want a hard cap
|
||||
before spawning, or when you must tell a busy agent to stop.
|
||||
## Read first
|
||||
- Repo KB page `omp-docs/pi/subagent-limits.md` — the live policy of record (values + rationale + deliberately-not-set list). Do not re-derive; read it.
|
||||
- `cfg://task` and `cfg://providers/maxInFlightRequests` — effective values, not the stock defaults.
|
||||
|
||||
## The knobs (`cfg://task/...`)
|
||||
|
||||
| Key | Default | Meaning |
|
||||
## The knobs (cfg:// writes; `/save` suffix persists)
|
||||
| Knob | Effect | Gotcha |
|
||||
|---|---|---|
|
||||
| `maxRuntimeMs` | `0` (unlimited) | **Hard wall-clock limit per subagent (ms).** The only true time cap. |
|
||||
| `softRequestBudget` | `200` | Soft cap on assistant **requests** per run; force-stop at **1.5×**. |
|
||||
| `softRequestBudgetNotice` | `true` | Injects one "wrap up" steering message at the soft mark. |
|
||||
| `agentIdleTtlMs` | `420000` | Parks an **idle** agent after 7 min. Useless against a busy-but-slow agent. |
|
||||
| `completionProbe` | `true` | Asks a working subagent to self-report completeness at 2/5/10 requests. |
|
||||
| `maxConcurrency` | `32` (often lowered) | Concurrent subagents; sizes GPU/endpoint load. |
|
||||
| `agentModelOverrides` | `{}` | Per-agent model routing, e.g. `{"scout":"provider/model-id"}`. |
|
||||
| `tools.maxTimeout` | `0` | Caps the timeout the agent may set on a **tool**, not on the agent. |
|
||||
| `task.maxConcurrency` | session semaphore; excess queues | stock default 32 — "2" is not a default |
|
||||
| `task.maxRecursionDepth` | nesting cap | predicate is `depth < n`; **1 = children cannot spawn (correct for "no nesting"); 0 disables spawning for the MAIN session too — never reach for 0**; negative = unlimited |
|
||||
| `task.maxRuntimeMs` | hard wall clock, overrun = `aborted` | default 0 = off |
|
||||
| `task.softRequestBudget` | wrap-up notice at N, force-yield at 1.5N | scout/sonic may impose lower built-ins |
|
||||
| `task.agentIdleTtlMs` | park idle finished agents | `<= 0` disables parking |
|
||||
| `providers.maxInFlightRequests` | concurrent LLM requests per provider id | keys are the quoted block labels from `agent/models.yml` (e.g. `"llamacpp (dankclaw-3090)"`), flat map lookup; absent/≤0 = unlimited; scope = this machine's config root only — other workstations and non-omp clients are NOT counted; excess queues |
|
||||
| `tools.maxTimeout` | ceiling on any agent-requested tool timeout | kills `timeout: 0` runaways |
|
||||
| `task.isolation.enabled` | makes per-spawn `isolated` flag available | opt-in per spawn, not a blanket sandbox; needs git repo; off in plan mode |
|
||||
| `task.completionProbe` | periodic completeness side-requests | token sink on shared endpoints |
|
||||
|
||||
Reasonable fleet defaults: `maxRuntimeMs = 900000` (15 min), leave the request budget
|
||||
alone unless the failure mode is a looping search agent (then lower it — request count,
|
||||
not the clock, is what bounds a 200-call research loop).
|
||||
## Verify semantics from the bundle, not the label
|
||||
Setting descriptions lie by omission. `~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`:
|
||||
- depth gate: `grep -o '.\{130\}xC\b.\{130\}'` finds `U_(n,s)`; `grep -o 'function U_(e,t){.\{0,60\}'` → `e<0||t<e` (n=cap, s=taskDepth).
|
||||
- in-flight lookup: `grep -o '.\{150\}axInFlightRequests.\{200\}'` → flat `[providerId]` lookup, non-number/≤0 = unlimited, clamp ≥1.
|
||||
|
||||
**Verify the cap instead of assuming it.** `read cfg://task/maxRuntimeMs` at the start of
|
||||
any session that plans to rely on it. A session-only write is gone next session, and an
|
||||
unanswered `/save` approval means the value never persisted — so the fleet can run hours
|
||||
with the cap at `0` while everyone believes it is 15 minutes. Nothing else bounds a child.
|
||||
|
||||
## Three failure modes, three different fixes
|
||||
|
||||
- **Runaway loop** (hundreds of sequential tool calls): lower `softRequestBudget`; the
|
||||
clock cap only truncates it late.
|
||||
- **Slow model, legitimate work** (dozens of calls on a low-t/s endpoint = an hour of wall
|
||||
clock): fix routing, not the clock — check `cfg://task/agentModelOverrides` and move the
|
||||
agent class onto a faster/shared endpoint. Watch the slot math: subagent endpoints share
|
||||
KV slots with the interactive session, so `maxConcurrency + 1` sequences on an N-slot
|
||||
endpoint degrades everyone's decode.
|
||||
- **Finished work, no exit (nested fan-out deadlock).** A subagent that spawned **its own**
|
||||
children blocks on their barriers. If those children die out-of-band (killed, starved, or
|
||||
waiting on an answer whose subject changed mid-flight), the parent waits forever with its
|
||||
deliverable already written. No instruction text prevents this — not the repo AGENTS.md,
|
||||
not the task prompt; only the clock cap or a kill ends it.
|
||||
|
||||
Signature: parent and children report **the same elapsed wall clock**, and the parent's
|
||||
on-disk output is complete. Census (agent registry + `ps`): group by elapsed time, look
|
||||
for identical `etimes` across a parent/child set, then judge the **artifacts** instead of
|
||||
the process — balanced code fences, no truncated tables, trailing newline, front matter or
|
||||
the page's H1+status convention intact. Complete artifacts + still "running" = deadlock,
|
||||
not work.
|
||||
|
||||
Repair: kill the **subtree** and confirm the children leave (a killed parent does not
|
||||
cancel children that came from a separate `tasks[]` batch). Then **flatten**: the
|
||||
top-level session does the fan-out and children never spawn children. Re-run only the
|
||||
slices whose artifacts are missing — integrate from what is already on disk.
|
||||
|
||||
## cfg write mechanics (bit me once)
|
||||
|
||||
- `write cfg://task/maxRuntimeMs 900000` → applies **session-only**, replies "NOT saved".
|
||||
- Persist = `write cfg://task/maxRuntimeMs/save` with the same content, and that write
|
||||
needs its **own interactive approval**. If the approval prompt is not answered in time
|
||||
the value stays session-only and the tool says *NEVER retry unless the user asks* —
|
||||
report it in the reply instead.
|
||||
- Revert a session value by writing the old value back (e.g. `0`).
|
||||
|
||||
## Order of operations when one is already running late
|
||||
|
||||
1. `read proc://` → registry of running agents; `read proc://<id>` for status/output.
|
||||
2. `read agent://<id>` → live progress ("Latest assistant text") — often shows it is
|
||||
mid-batch, not hung. If the latest text is already a wrap-up/summary, treat it as
|
||||
**done** and collect its output instead of waiting for an exit event.
|
||||
3. `write agent://<id>` an explicit wrap-up order: stop searching, answer what is
|
||||
sourced, mark the rest `NOT VERIFIED — needs <what>`, cap the report length.
|
||||
4. Only then `write proc://<id>/kill` — the cap does **not** retroactively bind agents
|
||||
spawned before the change, so a long runner must be ordered or killed, never just
|
||||
re-configured. After any kill, re-read the registry: unreaped children and orphaned
|
||||
host-side processes (a scratch server holding GPU/RAM) are the expensive residue, and
|
||||
`pkill -f` patterns that also match production services are forbidden — kill by pid.
|
||||
## Landing a policy change
|
||||
cfg `/save` writes persist immediately; `agent/config.yml` is machine-live tier (heartbeat commits it). Human-facing policy changes land as an `omp-docs` page update via branch → PR (docs tier), `git worktree add` from `origin/main` if the current checkout is branch-locked by workspace policy.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792381479415,
|
||||
"lastConsolidatedAt": 1792395788647,
|
||||
"lastResult": {
|
||||
"at": 1791599865775,
|
||||
"sessions": 1,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792392829703
|
||||
"lastConsolidatedAt": 1792396789735
|
||||
}
|
||||
|
||||
+19
-2
@@ -1,3 +1,20 @@
|
||||
## Subagent routing (harness law, 2026-10-18)
|
||||
|
||||
**No subagent class may route to `frank-archon`** — the operator session lives there and
|
||||
the endpoint runs 3 sequences (`--max-num-seqs 3`), so `task.maxConcurrency + 1` children
|
||||
eat the parent's own KV slots. Violation cost: three doc children + parent = 4 sequences
|
||||
on 3 slots, children ran 2 h 27 m / 2 h 45 m, the interactive session starved.
|
||||
Routing and bounds live in `~/.omp/agent/config.yml` (`task:` block — the owner; verified
|
||||
with `read cfg://task/agentModelOverrides`): heavy writers (`task`, `scout`) → frank-dragoon
|
||||
(2 slots, idle, slow card ⇒ slice work smaller), bursty readers (`reviewer`, `sonic`,
|
||||
`designer`, `librarian`) → frank-zealot (2 slots, also the compaction hub and every
|
||||
tiny/smol/plan/commit/memory role). Caps: `maxConcurrency: 2`, `maxRuntimeMs: 1800000`,
|
||||
`softRequestBudget: 120`, `maxRecursionDepth: 1` (children never spawn children).
|
||||
`cfg://.../save` approvals auto-deny after a hardcoded 10 s dialog — after Frank has
|
||||
approved a change, edit `config.yml` directly and validate the YAML; a running session
|
||||
picks global-config edits up at once. Full procedure: managed skill
|
||||
`bounding-subagent-runtime`.
|
||||
|
||||
## Documentation law (2026-10-18)
|
||||
|
||||
**One fact, one owner.** The `serverAppConfigs` repo owns every fleet fact;
|
||||
@@ -18,8 +35,8 @@ Swap the model, don't instrument the swap: edit the trio
|
||||
probe → done. **No benchmarks, A/Bs, ladders, or acceptance tables unless
|
||||
Frank explicitly asks for data.** If a payload misbehaves, revert the commit and
|
||||
redeploy. When a bench IS ordered, the GPU-safety procedure (stop + mask, no
|
||||
scratch ports, never `systemctl disable`) is `AGENTS.md` → *Model swaps* and
|
||||
*Deployment completeness*.
|
||||
scratch ports, never `systemctl disable`) is `AGENTS.md` → *Model swaps*
|
||||
and *Deployment completeness*.
|
||||
|
||||
## Where facts live
|
||||
|
||||
|
||||
@@ -3,25 +3,22 @@
|
||||
- Custom voice source material must exclude the first 10 seconds of any audio sample file used to generate voices; this is a project-wide invariant across all source files, not a per-file exception.
|
||||
- Voice selector valid options: af_heart, af_alloy, af_aoede, af_bella, af_jessica, af_kore, af_nicole, af_nova, af_river, af_sarah, af_sky, am_adam, am_echo, am_eric, am_fenrir, am_liam, am_michael, am_onyx, am_puck, am_santa, bf_emma, bf_isabella, bm_george, bm_lewis, bf_alice, bf_lily, bm_daniel, bm_fable.
|
||||
|
||||
## Podcast
|
||||
|
||||
- Podcast functionality is sourced from open-notebook; this is a project decision established before this document was created.
|
||||
|
||||
## Audio Input Policy
|
||||
|
||||
- There is no audio input anywhere in the fleet: that capability left with the FLM NPU endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's audio encoder, and left again with the Gemma4 retirement on 2026-10-01.
|
||||
- This policy mirrors the architecture-level decision documented in `architecture.md` regarding the fleet's input capabilities.
|
||||
|
||||
## Input Capabilities
|
||||
|
||||
All endpoints accept text input with optional image input: `frank-dragoon` returned to vision with the flash-server v2 swap, while `frank-archon` takes them natively and `frank-zealot` through its BF16 mmproj.
|
||||
|
||||
There is no audio input anywhere in the fleet: that capability left with the FLM NPU endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's audio encoder, and left again with the Gemma4 retirement on 2026-10-01.
|
||||
|
||||
This rule exists because the Gemma4 projector was retired in 2026-10-01, removing the audio input channel from the fleet. The 2026-09-11 to 2026-10-01 period contained the Gemma4 projector which enabled audio, but that endpoint is now retired.
|
||||
|
||||
## Voice Usage Notes
|
||||
|
||||
- Voice generator output does not affect token counting or inference costs — all voices are rendered post-inference.
|
||||
- Voice quality varies by source sample: exclude the first 10 seconds of any audio sample file used to generate voices; this is a project-wide invariant across all source files, not a per-file exception.
|
||||
- Voice selection is client-side only; the server does not generate or return audio data.`,
|
||||
- Voice selection is client-side only; the server does not generate or return audio data.
|
||||
|
||||
## Load Management
|
||||
|
||||
- Subagent settings must prevent frank-archon from being overwhelmed; concurrent request handling prioritizes endpoint stability over throughput.
|
||||
|
||||
## Podcast
|
||||
|
||||
- Podcast functionality is sourced from open-notebook; this is a project decision established before this document was created.
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792392891753,
|
||||
"lastConsolidatedAt": 1792396748497,
|
||||
"lastResult": {
|
||||
"at": 1792379504653,
|
||||
"at": 1792393886960,
|
||||
"sessions": 1,
|
||||
"deltas": 1,
|
||||
"model": "frank-zealot"
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# Output behavior
|
||||
|
||||
Output must not be shaken off during presentation. Output remains anchored until explicitly confirmed.
|
||||
@@ -1,4 +1,10 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792392882348
|
||||
"lastConsolidatedAt": 1792396722373,
|
||||
"lastResult": {
|
||||
"at": 1792396071933,
|
||||
"sessions": 1,
|
||||
"deltas": 1,
|
||||
"model": "frank-zealot"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792392961865,
|
||||
"lastConsolidatedAt": 1792393561866,
|
||||
"lastResult": {
|
||||
"at": 1792381870670,
|
||||
"sessions": 1,
|
||||
|
||||
+1
@@ -0,0 +1 @@
|
||||
{"v":1,"kind":"product_decision","statement":"Limit on zealot set to 3, dragoon set to 2, archon set to 3","source":"explicit_user","evidence":"make the limit on zealot 3, dragoon 2, archon 3","friction":{"corrective":false,"regression":false,"subtle":false},"sessionId":"01a15302-f12d-77d8-9a33-be31a1a05528","ts":1792394470419}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792394427081
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1792396749853
|
||||
}
|
||||
@@ -275,13 +275,13 @@ transition = [ "fade" ]
|
||||
interval_seconds = 1200
|
||||
|
||||
[wallpaper.default]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_4x17w3.jpg"
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_nzz3gy.jpg"
|
||||
|
||||
[wallpaper.last]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_4x17w3.jpg"
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_nzz3gy.jpg"
|
||||
|
||||
[wallpaper.monitors.DP-1]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_4x17w3.jpg"
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_nzz3gy.jpg"
|
||||
|
||||
[wallpaper.monitors.DP-2]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_oxex69.jpg"
|
||||
@@ -290,7 +290,7 @@ transition = [ "fade" ]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_v9e2w3.jpg"
|
||||
|
||||
[wallpaper.monitors.eDP-1]
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_4x17w3.jpg"
|
||||
path = "/home/blank/.local/share/noctalia/wallpapers/wallhaven_nzz3gy.jpg"
|
||||
|
||||
[weather]
|
||||
unit = "imperial"
|
||||
|
||||
Reference in New Issue
Block a user