83 lines
5.2 KiB
Markdown
83 lines
5.2 KiB
Markdown
---
|
||
name: bounding-subagent-runtime
|
||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, agentIdleTtlMs, per-agent model overrides, the cfg save-approval quirk, and the finished-work-but-never-exits nested-fanout deadlock."
|
||
---
|
||
|
||
# Bounding subagent runtime (omp harness)
|
||
|
||
Use when a `task`/`eval` subagent runs absurdly long (tens of minutes on a small job),
|
||
when it **finishes its deliverable and still never exits**, when you want a hard cap
|
||
before spawning, or when you must tell a busy agent to stop.
|
||
|
||
## The knobs (`cfg://task/...`)
|
||
|
||
| Key | Default | Meaning |
|
||
|---|---|---|
|
||
| `maxRuntimeMs` | `0` (unlimited) | **Hard wall-clock limit per subagent (ms).** The only true time cap. |
|
||
| `softRequestBudget` | `200` | Soft cap on assistant **requests** per run; force-stop at **1.5×**. |
|
||
| `softRequestBudgetNotice` | `true` | Injects one "wrap up" steering message at the soft mark. |
|
||
| `agentIdleTtlMs` | `420000` | Parks an **idle** agent after 7 min. Useless against a busy-but-slow agent. |
|
||
| `completionProbe` | `true` | Asks a working subagent to self-report completeness at 2/5/10 requests. |
|
||
| `maxConcurrency` | `32` (often lowered) | Concurrent subagents; sizes GPU/endpoint load. |
|
||
| `agentModelOverrides` | `{}` | Per-agent model routing, e.g. `{"scout":"provider/model-id"}`. |
|
||
| `tools.maxTimeout` | `0` | Caps the timeout the agent may set on a **tool**, not on the agent. |
|
||
|
||
Reasonable fleet defaults: `maxRuntimeMs = 900000` (15 min), leave the request budget
|
||
alone unless the failure mode is a looping search agent (then lower it — request count,
|
||
not the clock, is what bounds a 200-call research loop).
|
||
|
||
**Verify the cap instead of assuming it.** `read cfg://task/maxRuntimeMs` at the start of
|
||
any session that plans to rely on it. A session-only write is gone next session, and an
|
||
unanswered `/save` approval means the value never persisted — so the fleet can run hours
|
||
with the cap at `0` while everyone believes it is 15 minutes. Nothing else bounds a child.
|
||
|
||
## Three failure modes, three different fixes
|
||
|
||
- **Runaway loop** (hundreds of sequential tool calls): lower `softRequestBudget`; the
|
||
clock cap only truncates it late.
|
||
- **Slow model, legitimate work** (dozens of calls on a low-t/s endpoint = an hour of wall
|
||
clock): fix routing, not the clock — check `cfg://task/agentModelOverrides` and move the
|
||
agent class onto a faster/shared endpoint. Watch the slot math: subagent endpoints share
|
||
KV slots with the interactive session, so `maxConcurrency + 1` sequences on an N-slot
|
||
endpoint degrades everyone's decode.
|
||
- **Finished work, no exit (nested fan-out deadlock).** A subagent that spawned **its own**
|
||
children blocks on their barriers. If those children die out-of-band (killed, starved, or
|
||
waiting on an answer whose subject changed mid-flight), the parent waits forever with its
|
||
deliverable already written. No instruction text prevents this — not the repo AGENTS.md,
|
||
not the task prompt; only the clock cap or a kill ends it.
|
||
|
||
Signature: parent and children report **the same elapsed wall clock**, and the parent's
|
||
on-disk output is complete. Census (agent registry + `ps`): group by elapsed time, look
|
||
for identical `etimes` across a parent/child set, then judge the **artifacts** instead of
|
||
the process — balanced code fences, no truncated tables, trailing newline, front matter or
|
||
the page's H1+status convention intact. Complete artifacts + still "running" = deadlock,
|
||
not work.
|
||
|
||
Repair: kill the **subtree** and confirm the children leave (a killed parent does not
|
||
cancel children that came from a separate `tasks[]` batch). Then **flatten**: the
|
||
top-level session does the fan-out and children never spawn children. Re-run only the
|
||
slices whose artifacts are missing — integrate from what is already on disk.
|
||
|
||
## cfg write mechanics (bit me once)
|
||
|
||
- `write cfg://task/maxRuntimeMs 900000` → applies **session-only**, replies "NOT saved".
|
||
- Persist = `write cfg://task/maxRuntimeMs/save` with the same content, and that write
|
||
needs its **own interactive approval**. If the approval prompt is not answered in time
|
||
the value stays session-only and the tool says *NEVER retry unless the user asks* —
|
||
report it in the reply instead.
|
||
- Revert a session value by writing the old value back (e.g. `0`).
|
||
|
||
## Order of operations when one is already running late
|
||
|
||
1. `read proc://` → registry of running agents; `read proc://<id>` for status/output.
|
||
2. `read agent://<id>` → live progress ("Latest assistant text") — often shows it is
|
||
mid-batch, not hung. If the latest text is already a wrap-up/summary, treat it as
|
||
**done** and collect its output instead of waiting for an exit event.
|
||
3. `write agent://<id>` an explicit wrap-up order: stop searching, answer what is
|
||
sourced, mark the rest `NOT VERIFIED — needs <what>`, cap the report length.
|
||
4. Only then `write proc://<id>/kill` — the cap does **not** retroactively bind agents
|
||
spawned before the change, so a long runner must be ordered or killed, never just
|
||
re-configured. After any kill, re-read the registry: unreaped children and orphaned
|
||
host-side processes (a scratch server holding GPU/RAM) are the expensive residue, and
|
||
`pkill -f` patterns that also match production services are forbidden — kill by pid.
|