40 lines
3.8 KiB
Markdown
40 lines
3.8 KiB
Markdown
---
|
||
name: radiance-model-feasibility-scout
|
||
description: "Scout whether a new GGUF model can run on radiance (R9700 fleet): GGUF header census, REUSE/ADAPT/NEW gap matrix vs radiance docs, ledgers (traffic/KV/RAM/disk), KLD-gate substitution, HTML report delivery."
|
||
---
|
||
|
||
# Radiance Model Feasibility Scout
|
||
|
||
Answer "can model X serve on radiance (dankserver R9700 TP=2), and what's the bill?" Produce an HTML report. Complements `gguf-header-vram-fit-analysis` (fit math), `radiance-w4-convert-playbook` (conversion mechanics), `radiance-quantize-and-score`.
|
||
|
||
## 1. GGUF header census (trust header, not the card)
|
||
- Range-parse the header: tensor count, per-type census (type → tensor count / params / bytes), layer pattern from tensor names, `general.architecture`, `llm_type`, rope dim, expert counts, `mtp` declaration.
|
||
- MTP: presence is decided by the **tensor-name census** (`nextn.*`/`mtp` tensors), never by metadata.
|
||
- Confirm tokenizer + `chat_template` KV entries ship in the file; mmproj = separate BF16 tower file.
|
||
- Cross-check total bytes vs the published file size.
|
||
|
||
## 2. Gap matrix vs radiance docs
|
||
Read `PLUGIN.md` (arch-plugin contract, quantizer registry, core/mm image-pipeline law), `OPS.md` (op catalog: attention tiles h256 gqa2/4/8 + h512 gqa8, GDN, QSA, MoE, MTP), `QSA.md` (indexer decomposition bounds), `GUIDE.md` + `gguf.cpp` (dtype table 0–44: all standard GGUF grids readable — quantization happens in rad-convert, not the loader). Classify every component REUSE / ADAPT / NEW with the specific op named. The live `qwen4exp` plugin (~2k lines, QSA+MoE+mHC+GDN) is the template and the shared-header reuse list.
|
||
|
||
## 3. Laws checklist (recurring traps)
|
||
- GGUF `ssm_a` = −exp(A_log); undo at convert/plugin time.
|
||
- One-table-one-encoding: a routed-expert table takes ONE rad encoding; mixed GGUF grids (q2/q3/q4 per layer) re-quantize to w4 GPTQ + whole-expert BF16 protected set.
|
||
- int8 trunk + int8 lm_head laws; swiglu clamp carries from the arch family; rope-free layers need no RoPE path.
|
||
- KDA ≠ GDN: per-channel decay + 3 convs + L2-normed q/k + lower-bound gate → new kernel family (FLA `chunk_kda` is the oracle); `kkt_solve`/`chunk_scan` is the critical path.
|
||
- MLA: latent-cache absorbed form (KV = latent width × dtype), needs its own attention tile (e.g. h512 gqa-64) + per-head v bmm.
|
||
- llama.cpp merged PR for the arch = the ground-truth formula oracle (sign conventions, APE layout, Sinkhorn input).
|
||
|
||
## 4. Ledgers (all in the report)
|
||
- Per-token weight traffic GB (trunk int8 + active experts w4 + lm_head) → decode anchored on frank-archon's measured 99 t/s @ ~6.3 GB/token; state [INFERENCE].
|
||
- KV: per-token bytes per pool + per-seq state terms; cross-check vs any prior fleet runbook that served the same model.
|
||
- Pinned RAM: host pool + prefix vs `LimitMEMLOCK`/`ttm-preflight.sh` law.
|
||
- Disk peak = GGUF + `.rad` (~187 GiB-class for a 300B MoE, no PLE) vs free space; options: retire rollback payload / external convert host / `--in-place`.
|
||
|
||
## 5. KLD gate feasibility
|
||
BF16 teacher = GGUF params × 2 B; if unservable (626 GB-class), the substitute gate = llama.cpp (arch PR merged in master) serving the GGUF on the Dankclaw iGPU — GTT aperture ~114 GiB — stop+mask frank-dragoon for the window, KL + top-1 census vs radiance `.rad`. Document as a gate deviation.
|
||
|
||
## 6. Deliver as self-contained HTML
|
||
- Single file, inline CSS with `prefers-color-scheme` vars, section anchors + TOC, census/gap/risk/plan tables, `[INFERENCE]` labels on arithmetic.
|
||
- Write in **chunks**: one `write` ≈ ≤8 KB (output-token limit), end each chunk with `<!--MORE-->`, then `edit` with `PUT N.=N:` where N = marker line — **prefix every body row with `+`** (bare rows get auto-prefixed; register lines like `@name` land as literal content).
|
||
- Validate: python `html.parser` tag-balance script + grep for stray markers; `xdg-open` the file.
|