command-center/agent/managed-skills/model-runtime-support-survey/SKILL.md

name, description

name	description
model-runtime-support-survey	Survey how every open-source runtime serves a given LLM (vLLM, llama.cpp, SGLang, KTransformers, overlays, vendor GGUFs) before writing a serving/feasibility plan; harvest quant recipes, measured rates, and mapped landmines from PR comments and overlay repos.

Model Runtime Support Survey

Answer "how do other engines run model X, and what did they pay to learn it?" before committing a bring-up plan. Every kernel spec, quant recipe, and landmine found upstream is one you don't pay for twice. Proven on GLM-5.3-Flash (2026-10-14): surfaced a production 8xR9700 vLLM overlay, MTP 2.3x decode numbers, and 4 landmines (fp8-KV scale calibration, prefix-cache 0-hits, KDA autotune nondeterminism, chat-template notation).

Harvest order

1. Official model card (huggingface.co/<org>/<model>) — the "Serve Locally" section lists officially supported runtimes; sampling/template contract lives here (e.g. reasoning_effort defaults, clear_thinking).
2. Per-runtime PRs (vLLM, llama.cpp, SGLang) — fetch via the GitHub-API-backed read; the comments, not the body, are the payload: field reports (prefix-cache hit censuses, ROCm/AMD enablement branches, extension rebuild recipes, crash repros, free tuning levers via --override-kv, nondeterminism root-causes). Get the PR status (merged/open/closed) — it sets what "master supports" means.
3. Overlay/serving repos on HF (search model name + your GPU family, e.g. rocm-r9700): read the tree (huggingface.co/api/models/<repo>/tree/main), then raw/main/README.md, raw/main/OVERLAY_NOTES.md, raw/main/run-*.sh. Launch scripts carry the laws in comments (KV sizing justifications, env ladders, dead knobs).
4. Vendor GGUF guides (Unsloth docs): append .md to the docs URL for clean markdown; gives quant KLD/top-1 tables, template fixes, hardware floors. Check whether their GGUF carries MTP/nextn tensors — that's often the only public source of draft-head weights.
5. KTransformers tutorials (raw.githubusercontent.com/kvcache-ai/ktransformers/main/doc/en/kt-kernel/<Model>-Tutorial.md) + SGLang cookbook (docs.sglang.io/.../<Model>.md): CPU-offload expert shape validation, 1M-ctx recipes.

Harvest checklist (per runtime)

• Measured decode/prefill + hardware shape (cards, TP) — anchor, not decoration; per-card effective GB/s is the cross-engine comparison unit.
• Quant recipe: bits-per-weight split by module class (experts/trunk/embed) — convergent recipes cross-validate your planned conversion.
• Spec-decode: acceptance length, per-position accept, when it pays (CPU-offload) vs loses (VRAM-resident).
• Landmines: KV-scale calibration reading uninitialized linear-attn state; prefix-cache semantics on hybrid KDA/DSA models; kernel autotune nondeterminism (KLD across launches); fp8 flavour bugs (FNUZ vs e4m3fn scale); template notation engines can't parse; import gates per gfx/arch.
• Harness contract: reasoning-effort defaults (16k+ hidden thinking tokens), sampling params, tool/reasoning parser names.

Output

A runtime matrix table (engine | arch status | hardware | artifact | measured) + a numbered "laws we inherit" list; fold every law into the plan's phases and risks with § cross-refs.

Gotchas

• Read PR comments — the highest-value data (crash repros, censuses) is comment-only.
• HF fetches: api/models/<repo>/tree/main for listing, raw/main/<path> for files; the HTML card view renders README inline too.
• browser.open targets the harness's headless browser, not the user's — refresh user-facing pages with xdg-open.
• Companion skill for the artifact side: radiance-model-feasibility-scout (GGUF header census).