command-center/agent/managed-skills/aa-chart-model-score-extraction/SKILL.md

name, description

name	description
aa-chart-model-score-extraction	Extract per-model benchmark scores from artificialanalysis.ai pages: charts are client-rendered Recharts SVGs; zip label-list texts to foreignObject names by x-sorted index, validate chart identity against the page's published top-3 sentence, apply Elo/Omniscience normalization laws, seed frank-intelligence-bench aa-scores.yml.

Extract AI benchmark scores for a specific model from artificialanalysis.ai. Per-model per-eval numbers are NOT in SSR HTML (JS charts, API needs key); eval-page SSR text DOES carry the top-3 sentence.

Procedure (validated 5x on this repo's model pages)

1. browser.open({ name, url: "https://artificialanalysis.ai/models/<slug>", wait_until: "domcontentloaded" }), then scroll in ~6 steps (0.15..0.9 of scrollHeight, ~1.3 s pauses) — charts lazy-render.
2. Extract each chart: pair g.recharts-label-list text (values, take x attr) with foreignObject div (model names, x from parent g[transform] translate(x,y)). Sort both lists by x and zip by index — NOT nearest-x (nearest-x is off by one bar; index-zip matches published tops exactly).
3. Self row: match name loosely (Qwen3.8-Flash-Next has hyphens — test both forms).

Identity law (never ship an order-guessed chart→eval mapping)

• Model pages have ~14 charts, 18 eval cards; DOM order and geometry pairing BOTH proved ambiguous.
• Anchor every chart by its top-3 names/values vs the eval page's published sentence: https://artificialanalysis.ai/evaluations/<slug> body text: "X scores the highest on with a score of N%, followed by …". Regex must tolerate decimals: value pattern (-?[\\d.]+%?); build page scripts with String.raw (bun template literals eat \\d).
• A chart's self value on the MODEL page is page-local → trustworthy once the top-3 anchor confirms the eval (e.g. AA-LCR: Kimi 88.7 → chart top "Kimi K3 (max)=89").

Normalization laws (frank-intelligence-bench aa-scores.yml)

• Elo evals (AA-Briefcase, GDPval-AA): store (Elo−500)/2000×100 (1866 → 68.3), AA's published normalization.
• AA-Omniscience: score = Accuracy (w10) + 1−Hallucination (w5), both 0–100 → composite (10·acc + 5·nonhall)/15. NEVER feed the −100..100 Index chart value (pre-fix cost ~7 pts; calibration then reproduces AA's official index ±0.6: Flash-Next 39.85≈40, Haiku 43.4≈43, 27B 33.85≈34, 3.6-27B 21.55≈21).
• Every seeded row: source_url + fetched_at + index_version.

Seed + validate loop

Append model block to config/references/aa-scores.yml (block-insert via python string replace on the - name: <anchor> line — beware: dropping rows from a block loses the 2-space indent on the anchor line; verify with yaml.safe_load), then bun -e loadAaRefs parse-check, bun src/cli.ts report $RID, calibration check (full-row models: lite vs AA official), bun test, commit.