---
name: hf-checkpoint-profile
description: "Profile a Hugging Face safetensors checkpoint (sizes, dtypes, tensor-name space, gated/license) with compact HTTP-range reads — safe for subagents on small context slots."
---

# Profile an HF safetensors checkpoint cheaply

Goal: inventory a repo (files/bytes, dtype histogram, tensor-name patterns, per-group GiB, config keys, license/access) without pulling gigabytes and without overflowing a subagent's context slot.

## Endpoints (all public, no token needed even for gated models)
- `https://huggingface.co/api/models/<id>?blobs=true` → `siblings` (name+size), `gated`, `private`, `downloads`, `safetensors` metadata, `tags`.
- `https://huggingface.co/api/models/<id>` for tags/base model; `https://huggingface.co/api/models?author=<x>&limit=100` to find sibling repos (canonical id check: cards often link both `X-1.5-Y` and `X-Y`).
- `https://huggingface.co/api/models/<id>/resolve/main/<file>` for `config.json`, `model.safetensors.index.json`, `chat_template.jinja`, `LICENSE`, `generation_config.json`; `/raw/main/<file>` for text.

## dtype + per-group bytes without downloading shards
Parse the safetensors header over HTTP Range: first 8 bytes = little-endian u64 header length, then that many bytes of JSON (`{name: {dtype, shape, data_offsets}}`). Fetch every shard header with a thread pool (131 shards ≈ seconds), then bytes = prod(shape) × dtype size. Collapse names with `re.sub(r"\.\d+\.", ".N.", k)` (and `\.shard_\d+` → `.shard_N`) to get the pattern space; group-sum by prefix to see where the mass sits (experts / ngram-PLE / MTP / vision / trunk).

Compare the key set against the base model's index to prove "full-weight finetune, identical names" (a zero-diff key set means per-tensor recipes written for the base apply verbatim, and it reveals fused vs per-expert layout, PLE/ngram shards, MTP head, vision tower, absence of `quantization_config`).

## Guardrails
- Run python scripts from a clean directory: a stray `/tmp/inspect.py` shadows stdlib `inspect` and kills `concurrent.futures` imports.
- Never put an HF token on a command line (`/proc/<pid>/cmdline` is world-readable); use `curl -K <mode-600 file>`.
- `curl -f` / `--fail-with-body` and verify byte counts — a 401/403 writes the error body as the "artifact" and exits 0.

## Subagent context budget
Subagents on a 131k-token slot die from `exceed_context_size_error` when told to read whole doc pages. Give grep-only instructions (`| head -40`, `-o`, `-c`), demand a line cap ("report in ≤250 lines, no verbatim dumps"), and prefer `git clone --depth 5` + local grep to web-page reads. A peer scout that overflows returns exit 1 with the 400 in the harness log and NO structured output — assume nothing was learned and redo the fact inline.