Files

T

Frank a34322aeb0 refactor(skills): consolidate managed skill library 94 -> 25

The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.

Surface check: the 25 on disk are exactly the skills the harness advertises
this session.

2026-10-09 13:17:48 -07:00

2.7 KiB

Raw Blame History

name, description

name	description
hf-checkpoint-profile	Profile a Hugging Face safetensors checkpoint (sizes, dtypes, tensor-name space, gated/license) with compact HTTP-range reads — safe for subagents on small context slots.

Profile an HF safetensors checkpoint cheaply

Goal: inventory a repo (files/bytes, dtype histogram, tensor-name patterns, per-group GiB, config keys, license/access) without pulling gigabytes and without overflowing a subagent's context slot.

Endpoints (all public, no token needed even for gated models)

https://huggingface.co/api/models/<id>?blobs=true → siblings (name+size), gated, private, downloads, safetensors metadata, tags.
https://huggingface.co/api/models/<id> for tags/base model; https://huggingface.co/api/models?author=<x>&limit=100 to find sibling repos (canonical id check: cards often link both X-1.5-Y and X-Y).
https://huggingface.co/api/models/<id>/resolve/main/<file> for config.json, model.safetensors.index.json, chat_template.jinja, LICENSE, generation_config.json; /raw/main/<file> for text.

dtype + per-group bytes without downloading shards

Parse the safetensors header over HTTP Range: first 8 bytes = little-endian u64 header length, then that many bytes of JSON ({name: {dtype, shape, data_offsets}}). Fetch every shard header with a thread pool (131 shards ≈ seconds), then bytes = prod(shape) × dtype size. Collapse names with re.sub(r"\.\d+\.", ".N.", k) (and \.shard_\d+ → .shard_N) to get the pattern space; group-sum by prefix to see where the mass sits (experts / ngram-PLE / MTP / vision / trunk).

Compare the key set against the base model's index to prove "full-weight finetune, identical names" (a zero-diff key set means per-tensor recipes written for the base apply verbatim, and it reveals fused vs per-expert layout, PLE/ngram shards, MTP head, vision tower, absence of quantization_config).

Guardrails

Run python scripts from a clean directory: a stray /tmp/inspect.py shadows stdlib inspect and kills concurrent.futures imports.
Never put an HF token on a command line (/proc/<pid>/cmdline is world-readable); use curl -K <mode-600 file>.
curl -f / --fail-with-body and verify byte counts — a 401/403 writes the error body as the "artifact" and exits 0.

Subagent context budget

Subagents on a 131k-token slot die from exceed_context_size_error when told to read whole doc pages. Give grep-only instructions (| head -40, -o, -c), demand a line cap ("report in ≤250 lines, no verbatim dumps"), and prefer git clone --depth 5 + local grep to web-page reads. A peer scout that overflows returns exit 1 with the 400 in the harness log and NO structured output — assume nothing was learned and redo the fact inline.

2.7 KiB Raw Blame History Unescape Escape