command-center/agent/managed-skills/snapcompact-reader-recall-probe/SKILL.md

name, description

name	description
snapcompact-reader-recall-probe	Measure bitmap-frame OCR recall and true per-frame token billing of a custom vision endpoint to pick snapcompact shape and price it correctly.

Snapcompact reader recall probe

Decide snapcompact settings for a non-frontier (custom/local) vision model: which frame shape its tower actually reads, and what the endpoint really bills per frame (vs the harness's blind FRAME_TOKEN_ESTIMATE=5024 floor).

Fact sources (in trust order)

1. omp://compaction.md (Snapcompact method section) + cfg://snapcompact + cfg://compaction — mechanism + resolved values/defaults.
2. Upstream ground truth: raw.githubusercontent.com/can1357/oh-my-pi/main/packages/snapcompact/CHANGELOG.md — per-model eval winners and the toolbench re-tune (spacing-tuned cells beat dense; ~16px/char OCR floor). research/ dir has the eval scripts.
3. Live bundle truth: grep minified dist/cli.js (~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/) for string literals: grep -o '\[\[/[^;]\{0,400\}' cli.js shows the ideal-shape regex table (claude→11on16-bw, gemini→8on22-bw@2048, gpt|codex→8on22-bw, kimi→8on22-bw, glm→8on16-bw); unknown:"8on22-bw" is the openai-compat wire fallback. Unmeasured ids (no qwen matcher) get the wire-family shape.
4. Reachability of pricing knobs (proved 2026-10-27 on 18.12.5): the user-row merge chain (grep -o '.\{80\}t\.imageCountBudget.\{200\}' cli.js) forwards imageCountBudget ONLY — imageTokenization resolves through the compiled catalog identity (no user-row key reaches it, so custom endpoints can never mark frames verified and always eat the 5024 floor in sizing). Regimes: openai-patch | anthropic-patch | deepseek-patch | fixed ("fixed":return e.tokens) — catalog-side only.

Probe recipe

Scratch dir: bun init -y && bun add @oh-my-pi/snapcompact (ships TS source; native renderer via pi-natives). npm version may drift a patch ahead of the harness bundle — geometry caveat, note it.

probe.ts legs against the endpoint's /v1/chat/completions:

• Build ~120k-char tool-traffic transcript in snapcompact scope format (¶user:, ¶call: + <out>…</out>, ¶ai:) with N planted unique spelled-out facts (spell digits as words so filler can't collide; keep filler digit-free); score recall by content.includes(value).
• Legs: plain-text baseline, then images per candidate shape via resolveShape({api, id}, variant) + renderMany(text, {shape}); send as {type:"image_url", image_url:{url:"data:<mime>;base64,<data>", detail:"original"}} with a reading-guide text part.
• Replicate the harness's real request shape: same chat_template_kwargs (enable_thinking/preserve_thinking) and sampling from the models.yml row.
• True per-frame billing = (prompt_tokens − text-scaffold) / frames, compare vs plannedFrameTokens(shape)/frameBilling.

Then: declare imageCountBudget: <proven count> on the models.yml row (custom providers inherit the unknown-provider floor of 5 images/request; llama.cpp has no such limit) — that lifts the count clamp; token pricing stays floor-bound until upstream prices the wire family. Measured baseline (frank-archon, omp 18.12.5): auto 8on22-bw 8/8 recall at 47 % of text's billed tokens, ~2,370 tok/frame true vs 5,024 charged; dense grids (8on16/11on16) drop characters — keep shape: auto.

Gotchas

• log() is an eval-kernel global, NOT a bun global — define a console wrapper or the script dies at first call.
• bash async + name = service mode (rejects async); drop name for finite jobs; timeout: 0 disables the 600s deadline for multi-leg vision runs.
• edit tool needs the absolute path (not bare filename) for /tmp files; PUT <N inserts BEFORE line N.
• Vision gate: row must declare input: [text, image] or snapcompact is skipped silently.
• Fleet follow-ups live in omp-docs/core/snapcompact.md (dank/command-center).