Frank
a34322aeb0
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
32 lines
2.7 KiB
Markdown
32 lines
2.7 KiB
Markdown
---
|
||
name: radiance-precision-selection
|
||
description: "Choose tensor precisions for a radiance (StillDeadcode) rad-convert recipe on RDNA4: what is native, what is supported, what the hardware rewards."
|
||
---
|
||
|
||
# Radiance precision selection
|
||
|
||
Use when writing/choosing a `rad-convert` recipe for radiance on RDNA4 (gfx1201), or when arguing why a model was converted a particular way.
|
||
|
||
## Project map (which repo is authoritative)
|
||
- `codeberg.org/StillDeadcode/radiance` — standalone engine (C++/HIP). Recipes: `data/recipes/*.recipe`; docs: `docs/{GUIDE,MOE-W4,KERNELS,PLUGIN,TOOLS,OPS,ARCHITECTURES,DOCKER}.md`. Prefer the RAW form of Codeberg URLs when reading: `/raw/branch/main/<path>`.
|
||
- `codeberg.org/StillDeadcode/libr4d` — kernel library; entry points encode exact geometry and reject mismatch (`attn_decode_h256_gqa6_fp8kv`, `gdn_chunk_scan_k128_v128_c64_bf16`, `ar_oneshot_2rank_exact`). Kernel names are the ground truth for which dtypes have a fast path.
|
||
- `codeberg.org/StillDeadcode/vllm-radiance` — deprecated vLLM wrapper; README still carries the best prose on the FP8 dispatcher and kernel list.
|
||
|
||
## Precision ladder
|
||
| Precision | Role |
|
||
|---|---|
|
||
| block-FP8 (E4M3 + bf16 block scales) | native tuned GEMM/attention/KV — "preshuffled block-FP8 dispatcher" |
|
||
| MXFP4 (OCP group-32) / W4A8 | supported weight packing; activations dynamically quantized to FP8 E4M3, compute is FP8 WMMA |
|
||
| bf16 | KV (`auto`), skinny GEMMs, Gated-DeltaNet kernels |
|
||
| int8 / int4 | draft head and integer GEMM paths |
|
||
|
||
## Decision rules
|
||
- Prefer block-FP8 for anything the engine qualifies: generic upstream AITER linear in place of the FP8 dispatcher measured **9–11 % slower**; all upstream-qualified models are FP8 (Qwen3.8-27B-FP8, Qwen3.6-27B-FP8, Qwen3.6-35B-A3B-FP8, Gemma-4-31B-it-FP8).
|
||
- Weight packing is a *storage* choice, activation dtype is the *compute* choice: MXFP4 buys bytes, not an FP4 compute path.
|
||
- The matrix core is integer-biased: f16 WMMA is half the int8 rate and a quarter of the int4 rate ⇒ int4 > int8 > f16. FP8/INT8 kernels must fuse double-K WMMA to saturate the 128-bit memory interface (GPUOpen WMMA guide part 2) — a "slow" quantized kernel usually means a missing K-fused variant, not an inherent dtype cost.
|
||
- Keep low-bit tricks on the draft head (int4/int8) where verification tolerates them; keep lm_head and hard-selection tensors (QSA indexer top-k) at high precision — the shipped recipes quantify the cost (int8 lm_head beats E4M3 by ~0.6 top-1 points).
|
||
|
||
## Verification
|
||
- Read the recipe's own header comments: radiance recipes state measured KL / top-1 / prefill-decode deltas for their choice, and name the `--reuse` inputs.
|
||
- Confirm the kernel exists before promising a dtype: grep libr4d entry points for the dtype suffix (`fp8kv`, `bf16`, `exact`).
|