live: heartbeat sync 2026-10-18T16:47:59-07:00 (Refs #11)

agent/managed-skills/aa-chart-label-browser-extraction/SKILL.md

@@ -0,0 +1,24 @@
+---
+name: aa-chart-label-browser-extraction
+description: "Extract per-model benchmark scores from artificialanalysis.ai pages: charts are client-rendered recharts SVGs invisible to plain HTML fetch; pair sorted label/name x-coords, then cross-check against published sentences before trusting any chart↔eval identity."
+---
+
+Extract per-eval scores for a model from artificialanalysis.ai when no API key exists. Plain HTML fetch yields NO chart data (client-side recharts; SSR has only meta + FAQ sentences; `/api/*` = 401).
+
+## Procedure
+1. Managed browser tab: `browser.open({name, url, wait_until: "domcontentloaded"})`.
+2. **Per-eval top-3 (authoritative):** open `https://artificialanalysis.ai/evaluations/<slug>`; regex `document.body.innerText` (collapse whitespace): `([A-Z][^.;]{0,60}?) scores the highest on (.+?) with a score of (-?[\d.]+%?), followed by (.+?) with a score of ... and ...` — mind decimal points truncating naive `[^.]+` regexes. Build the page script as a plain string (`String.raw`), not an interpolated template, so `\\d` survives.
+3. **All-eval per-model rows:** on `https://artificialanalysis.ai/models/<slug>`: scroll in 6 steps (0.15→0.9, ~1.3 s each) to trigger lazy charts, then per `svg` with ≥3 bars: labels = `[...svg.querySelectorAll("g.recharts-label-list text")]` (x attr), names = `[...svg.querySelectorAll("foreignObject")]` divs sorted by `g[transform] translate(x,y)`. Sort BOTH by x, **zip by index** — x-center pairing is off by a systematic half-bar.
+4. Self-row filter: exact hyphenated name (e.g. `Qwen3.8-Flash-Next`, not "Qwen3.8 Flash").
+5. Elo evals (GDPval-AA, AA-Briefcase): charts show AA's published `(Elo−500)/2000` normalization (1866→68.3).
+
+## Verification law (non-negotiable — two plausible mappings disagreed twice)
+- Zip top-3 of every chart against the eval page's published sentence (e.g. Kimi 88.7→89, Opus 66.9→67); ship only cross-checked rows.
+- Arithmetic corroboration helps: Omniscience accuracy 17% + non-hallucination 7% ↔ index −60.
+- Record provenance per row: sentence rows → eval-page URL; chart-label rows → model-page URL; `fetched_at` + `index_version`.
+- Geometry pairing (link→nearest chart below) is AMBIGUOUS when chart height > gap to next card — adjacent links share a chart. Use order + sentence tops, not geometry.
+
+## Traps
+- `tab.run` = Bun runtime; page-world JS needs `tab.evaluate`. Eval-cell `log()` does not reach display; write to a /tmp file and `cat`.
+- "Add model from specific provider" picker is not in the DOM (renders after chart data loads); UI drill-through × N evals is fragile — prefer the sentences.
+- Headline index FAQ: "scores N on the Artificial Analysis Intelligence Index" — check for "(estimated)" qualifier; est. = "independent evaluation forthcoming".