From 9df5c50fdada5156115bb24df7bba9dcb2bd24b4 Mon Sep 17 00:00:00 2001 From: Frank Date: Tue, 27 Oct 2026 18:58:49 -0700 Subject: [PATCH] live: heartbeat sync 2026-10-27T18:58:49-07:00 (Refs #11) --- agent/config.yml | 4 +-- .../snapcompact-reader-recall-probe/SKILL.md | 32 +++++++++++++++++++ .../sharpshooter/omp-32c11n2wnglul/state.json | 2 +- .../sharpshooter/tmp-1tpw73408x3e/state.json | 2 +- .../state.json | 2 +- .../state.json | 4 +++ agent/models.yml | 12 +++++++ 7 files changed, 53 insertions(+), 5 deletions(-) create mode 100644 agent/managed-skills/snapcompact-reader-recall-probe/SKILL.md create mode 100644 agent/memories/sharpshooter/wt-20261027-181709-7c8a99d-249vx4089paox/state.json diff --git a/agent/config.yml b/agent/config.yml index bf91099..630a08d 100644 --- a/agent/config.yml +++ b/agent/config.yml @@ -85,7 +85,7 @@ compaction: - handoff - remote idleThresholdTokens: 100000 - thresholdPercent: 95 + thresholdPercent: 85 idleEnabled: true thresholdTokens: -1 dropUseless: false @@ -96,7 +96,7 @@ compaction: experimentalContextManagement: false interruptMode: immediate defaultThinkingLevel: xhigh -hideThinkingBlock: false +hideThinkingBlock: true proseOnlyThinking: false terminal: showProgress: true diff --git a/agent/managed-skills/snapcompact-reader-recall-probe/SKILL.md b/agent/managed-skills/snapcompact-reader-recall-probe/SKILL.md new file mode 100644 index 0000000..95b0576 --- /dev/null +++ b/agent/managed-skills/snapcompact-reader-recall-probe/SKILL.md @@ -0,0 +1,32 @@ +--- +name: snapcompact-reader-recall-probe +description: Measure bitmap-frame OCR recall and true per-frame token billing of a custom vision endpoint to pick snapcompact shape and price it correctly. +--- + +# Snapcompact reader recall probe + +Decide snapcompact settings for a non-frontier (custom/local) vision model: which frame shape its tower actually reads, and what the endpoint really bills per frame (vs the harness's blind `FRAME_TOKEN_ESTIMATE`=5024 floor). + +## Fact sources (in trust order) +1. `omp://compaction.md` (Snapcompact method section) + `cfg://snapcompact` + `cfg://compaction` — mechanism + resolved values/defaults. +2. Upstream ground truth: `raw.githubusercontent.com/can1357/oh-my-pi/main/packages/snapcompact/CHANGELOG.md` — per-model eval winners and the `toolbench` re-tune (spacing-tuned cells beat dense; ~16px/char OCR floor). `research/` dir has the eval scripts. +3. Live bundle truth: grep minified `dist/cli.js` (`~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/`) for string literals: `grep -o '\[\[/[^;]\{0,400\}' cli.js` shows the ideal-shape regex table (`claude→11on16-bw, gemini→8on22-bw@2048, gpt|codex→8on22-bw, kimi→8on22-bw, glm→8on16-bw`); `unknown:"8on22-bw"` is the openai-compat wire fallback. Unmeasured ids (no qwen matcher) get the wire-family shape. +4. Reachability of pricing knobs (proved 2026-10-27 on 18.12.5): the user-row merge chain (`grep -o '.\{80\}t\.imageCountBudget.\{200\}' cli.js`) forwards `imageCountBudget` ONLY — `imageTokenization` resolves through the compiled catalog identity (no user-row key reaches it, so custom endpoints can never mark frames `verified` and always eat the 5024 floor in sizing). Regimes: `openai-patch | anthropic-patch | deepseek-patch | fixed` (`"fixed":return e.tokens`) — catalog-side only. + +## Probe recipe +Scratch dir: `bun init -y && bun add @oh-my-pi/snapcompact` (ships TS source; native renderer via pi-natives). npm version may drift a patch ahead of the harness bundle — geometry caveat, note it. + +`probe.ts` legs against the endpoint's `/v1/chat/completions`: +- Build ~120k-char tool-traffic transcript in snapcompact scope format (`¶user:`, `¶call:` + `…`, `¶ai:`) with N planted unique spelled-out facts (spell digits as words so filler can't collide; keep filler digit-free); score recall by `content.includes(value)`. +- Legs: plain-text baseline, then images per candidate shape via `resolveShape({api, id}, variant)` + `renderMany(text, {shape})`; send as `{type:"image_url", image_url:{url:"data:;base64,", detail:"original"}}` with a reading-guide text part. +- Replicate the harness's real request shape: same `chat_template_kwargs` (enable_thinking/preserve_thinking) and sampling from the models.yml row. +- True per-frame billing = `(prompt_tokens − text-scaffold) / frames`, compare vs `plannedFrameTokens(shape)`/`frameBilling`. + +Then: declare `imageCountBudget: ` on the models.yml row (custom providers inherit the unknown-provider floor of 5 images/request; llama.cpp has no such limit) — that lifts the count clamp; token pricing stays floor-bound until upstream prices the wire family. Measured baseline (frank-archon, omp 18.12.5): auto `8on22-bw` 8/8 recall at 47 % of text's billed tokens, ~2,370 tok/frame true vs 5,024 charged; dense grids (8on16/11on16) drop characters — keep `shape: auto`. + +## Gotchas +- `log()` is an eval-kernel global, NOT a bun global — define a console wrapper or the script dies at first call. +- bash `async` + `name` = service mode (rejects async); drop `name` for finite jobs; `timeout: 0` disables the 600s deadline for multi-leg vision runs. +- edit tool needs the absolute path (not bare filename) for /tmp files; `PUT