live: heartbeat sync 2026-10-18T16:47:59-07:00 (Refs #11)

agent/managed-skills/bounding-subagent-runtime/SKILL.md

@@ -0,0 +1,82 @@
+---
+name: bounding-subagent-runtime
+description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, agentIdleTtlMs, per-agent model overrides, the cfg save-approval quirk, and the finished-work-but-never-exits nested-fanout deadlock."
+---
+
+# Bounding subagent runtime (omp harness)
+
+Use when a `task`/`eval` subagent runs absurdly long (tens of minutes on a small job),
+when it **finishes its deliverable and still never exits**, when you want a hard cap
+before spawning, or when you must tell a busy agent to stop.
+
+## The knobs (`cfg://task/...`)
+
+| Key | Default | Meaning |
+|---|---|---|
+| `maxRuntimeMs` | `0` (unlimited) | **Hard wall-clock limit per subagent (ms).** The only true time cap. |
+| `softRequestBudget` | `200` | Soft cap on assistant **requests** per run; force-stop at **1.5×**. |
+| `softRequestBudgetNotice` | `true` | Injects one "wrap up" steering message at the soft mark. |
+| `agentIdleTtlMs` | `420000` | Parks an **idle** agent after 7 min. Useless against a busy-but-slow agent. |
+| `completionProbe` | `true` | Asks a working subagent to self-report completeness at 2/5/10 requests. |
+| `maxConcurrency` | `32` (often lowered) | Concurrent subagents; sizes GPU/endpoint load. |
+| `agentModelOverrides` | `{}` | Per-agent model routing, e.g. `{"scout":"provider/model-id"}`. |
+| `tools.maxTimeout` | `0` | Caps the timeout the agent may set on a **tool**, not on the agent. |
+
+Reasonable fleet defaults: `maxRuntimeMs = 900000` (15 min), leave the request budget
+alone unless the failure mode is a looping search agent (then lower it — request count,
+not the clock, is what bounds a 200-call research loop).
+
+**Verify the cap instead of assuming it.** `read cfg://task/maxRuntimeMs` at the start of
+any session that plans to rely on it. A session-only write is gone next session, and an
+unanswered `/save` approval means the value never persisted — so the fleet can run hours
+with the cap at `0` while everyone believes it is 15 minutes. Nothing else bounds a child.
+
+## Three failure modes, three different fixes
+
+- **Runaway loop** (hundreds of sequential tool calls): lower `softRequestBudget`; the
+  clock cap only truncates it late.
+- **Slow model, legitimate work** (dozens of calls on a low-t/s endpoint = an hour of wall
+  clock): fix routing, not the clock — check `cfg://task/agentModelOverrides` and move the
+  agent class onto a faster/shared endpoint. Watch the slot math: subagent endpoints share
+  KV slots with the interactive session, so `maxConcurrency + 1` sequences on an N-slot
+  endpoint degrades everyone's decode.
+- **Finished work, no exit (nested fan-out deadlock).** A subagent that spawned **its own**
+  children blocks on their barriers. If those children die out-of-band (killed, starved, or
+  waiting on an answer whose subject changed mid-flight), the parent waits forever with its
+  deliverable already written. No instruction text prevents this — not the repo AGENTS.md,
+  not the task prompt; only the clock cap or a kill ends it.
+
+  Signature: parent and children report **the same elapsed wall clock**, and the parent's
+  on-disk output is complete. Census (agent registry + `ps`): group by elapsed time, look
+  for identical `etimes` across a parent/child set, then judge the **artifacts** instead of
+  the process — balanced code fences, no truncated tables, trailing newline, front matter or
+  the page's H1+status convention intact. Complete artifacts + still "running" = deadlock,
+  not work.
+
+  Repair: kill the **subtree** and confirm the children leave (a killed parent does not
+  cancel children that came from a separate `tasks[]` batch). Then **flatten**: the
+  top-level session does the fan-out and children never spawn children. Re-run only the
+  slices whose artifacts are missing — integrate from what is already on disk.
+
+## cfg write mechanics (bit me once)
+
+- `write cfg://task/maxRuntimeMs 900000` → applies **session-only**, replies "NOT saved".
+- Persist = `write cfg://task/maxRuntimeMs/save` with the same content, and that write
+  needs its **own interactive approval**. If the approval prompt is not answered in time
+  the value stays session-only and the tool says *NEVER retry unless the user asks* —
+  report it in the reply instead.
+- Revert a session value by writing the old value back (e.g. `0`).
+
+## Order of operations when one is already running late
+
+1. `read proc://` → registry of running agents; `read proc://<id>` for status/output.
+2. `read agent://<id>` → live progress ("Latest assistant text") — often shows it is
+   mid-batch, not hung. If the latest text is already a wrap-up/summary, treat it as
+   **done** and collect its output instead of waiting for an exit event.
+3. `write agent://<id>` an explicit wrap-up order: stop searching, answer what is
+   sourced, mark the rest `NOT VERIFIED — needs <what>`, cap the report length.
+4. Only then `write proc://<id>/kill` — the cap does **not** retroactively bind agents
+   spawned before the change, so a long runner must be ordered or killed, never just
+   re-configured. After any kill, re-read the registry: unreaped children and orphaned
+   host-side processes (a scratch server holding GPU/RAM) are the expensive residue, and
+   `pkill -f` patterns that also match production services are forbidden — kill by pid.