--- type: "reference" title: "Radiance Tools — inspect, check, tune, tokenise" description: "The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports." tags: - llm - radiance - tools - reference timestamp: 2026-10-30 --- # Radiance tools Six binaries and `rad-export` are installed beside `radiance`; `rad-avx-check`, `r4d_selftest` and `rad-chatparse` live in the build tree. Every tool that loads plugins honours `RADIANCE_HOME`, `RADIANCE_KERNELS` and `--kernels` as the engine does. A numeric option takes a whole number: `12k`, `auto` (where the option does not name it) or an empty value exits 2. `rad-convert`, `rad-kbench` and `rad-tune` run the engine's startup with a different last step: the same plugins, in the same hierarchy, through the same declare phase, so they resolve kernels exactly as a serve would. ## `rad-info` — what a container holds ``` rad-info [options] rad-info --plugins [--home DIRS] -v, --verbose list every tensor, not just the per-unit summary --meta metadata table, then stop --vocab vocab summary and normaliser chain, then stop --profile N top N expert-profile entries (default 20, 0 = all) --recipe the recipe the container was converted with, then stop --plugins every plugin the search path offers, as the engine loads it --home DIRS search path for --plugins (default $RADIANCE_HOME) ``` It opens the container read-only and loads no plugins (except for `--plugins`): a container names no kernel library. Output is aligned; byte counts are binary units and each count says what it counts. ```sh rad-info m.rad # format, architecture, model, source quant, recipe, sections rad-info --recipe m.rad # the recipe, expanded, one rule a line rad-info -v m.rad | head -40 # every tensor, encoding, shape and what made it rad-info --plugins # kind, version, declarations, reference, device fit, file ``` The section table (strings, metadata, directory, planes, encodings, vocab, profile, data) is the container's layout; the metadata table, the weight directory grouped by layer and movement unit, the vocab, and the expert profile follow. ## `rad-kbench` — correctness, completeness, speed ``` rad-kbench [options] replay the recorded fixture (no model) rad-kbench -m [options] check against a real container rad-kbench -m --record FILE record a new fixture rad-kbench -m --skeleton FILE write a metadata-only copy to record from ``` **Replay** needs no model, weights or card. Geometries, seeds and `libref`'s reduced answers are checked in as `data/kernels.rkb` (dense model, block drafter, gqa 6) and `data/kernels_moe.rkb` (routed MoE, MTP head, gqa 8). `--fixture` is repeatable; `ctest` runs three replays — the other libraries against each recording, and `libref` against both at tolerance zero (its regression test, since a live comparison moves both sides together). Each library is asked for its row as the engine would ask it with that library alone: same constraints, derived parameters and weight-encoding filter, so a row the engine would pass over does not run here either (`--all-rows` runs every row whose constraints hold). **Against a container** (`-m`), weight operands are read from it: each weight's canonical planes, which the kernel gets through its own `layout`/`relayout` hooks and the oracle gets decoded, and every expert of a weight table. This is the only test of the artifact and of the layout hooks. **Recording** takes the reference's answer once at every point of every declared op, whichever library resolved it. Weights are drawn from a seed, not read from the container, so any machine can reproduce the reference and no card is needed. A skeleton (tables intact, data a hole) records the same fixture in a few megabytes. ```sh rad-kbench -m model.rad --skeleton model.skel.rad rad-kbench -m model.skel.rad --max-cases 1 --record data/kernels.rkb rad-kbench -m moe.rad --max-cases 1 --record data/kernels_moe.rkb ``` ### Verdicts - An output is compared on sampled positions, the full-tensor norm, and 32 seeded random-sign projections (which estimate rel_l2 to about 25 %). The largest of the three is the error. - Tolerances are per op and activation dtype (`dtype`, else `q_dtype`, else `kv_dtype`). Copies, gathers, scatters and cache stores are exact. Every op in the fixtures has a row of its own. - Ids are compared exactly; where the contract leaves a tie open (`sample_argmax`, `router_topk`, `sample_topk`), an id indexing a value equal to the reference id's is accepted. - A case whose reference and output are both identically zero checked nothing: **vacuous**, not passed. - A failure, a red-zone overrun, a fusion that differs from its chain, or a host kernel that wrote into an input exits 1. `--strict` also fails a stale fixture. - `--coverage DIR` keeps one file per fixture/library of the rows a replay reaches (`data/kbench_rows/`); a listed row not reached fails, unless a memory skip kept its cases out. - Every pass ends with a line saying where its time went (draw, hashing, layout, upload, launch, read-back, comparison, cache, timing). ### Timing `--bench` times every case that passes, on the bytes it was checked on. - Device cases are timed **after** the checks, operands still on the device, after one sustained warm-up. A time is device timestamps around a batch of back-to-back launches divided by the launch count; the batch is queued behind a gate the host opens once submission is done, so neither the host rate nor a sync is in the interval. Two untimed launches precede every batch; batches repeat until the median's 95 % interval is within 1 % or `--bench-reps` is reached. - **Cold by rotation**: weights and inputs of 1/32 of the footprint or more are read from copies spanning the footprint (`--bench-flush`, default 256 MiB, at least twice an R9700's caches), so each launch reads them from memory as streamed weights do. Smaller inputs stay warm, as the previous op leaves them. - In-place operands are restored from a pristine copy before every launch; the restores are timed alone and subtracted. - Timing draws expert offsets/routing skewed (Zipf 0.5) and block tables as permutations; the checked launch keeps the recorded draw. - The header prints the calibration: launch floor and streaming-read bandwidth warm and cold. If a smaller cold probe runs >1.5× the largest's rate the rotation was cache-served and device times are withheld. Throttler bits that came on during a batch repeat it, up to three times. - Host cases are timed in their own column, after an untimed launch, weights rotated over at least 256 MiB (twice LLC), for at most two seconds a case. ### Flags | flag | meaning | |---|---| | `--fixture FILE` | replay recording (default `$RADIANCE_HOME/kernels.rkb`); repeatable | | `--record FILE` | with `-m`: write a fixture | | `--skeleton FILE` | with `-m`: write a sparse container copy | | `--rerecord FILE` | replay: write the fixtures' cases again with the reference's answers | | `--record-rows FILE` | replay: record a row-driven fixture from the cases | | `-m, --model FILE` | the `.rad` to read weight operands from | | `--home DIR`, `--kernels A:B` | plugin home; kernel libraries (default every one found) | | `--op NAME` | only this op (repeatable prefix match) | | `--ref NAME` | the oracle plugin's name (default: the library declaring itself a reference) | | `--no-ref` | replay without the reference among the tested libraries | | `--strict` | replay: a stale fixture fails | | `--all-rows` | run every row whose constraints hold | | `--coverage DIR`, `--coverage-write` | the row-coverage ratchet | | `--no-cache`, `--cache-dir DIR` | run the reference instead of reusing cached answers; cache location | | `--seed N` | base seed; every case derives its own | | `--tol X` | override every tolerance (bisecting only) | | `--max-ctx N` | deepest context considered (declare and the attention sweep) | | `--max-tok N` | `max_tok` declare runs at (default 512, small on purpose) | | `--mm-max-patches N` | encoder pass declare runs at (default 2048 when a tower is present; 0 leaves it out) | | `--kv-cache-dtype T` | `bf16` or `fp8`, as on the engine | | `--num-speculative-tokens N` | draft depth to declare at (default: the container's) | | `--max-cases N` | runs per distinct case (default 4; 0 = every one) | | `--max-bytes MIB` | skip a case needing more than this, by name (default: half available memory, 512–8192) | | `--no-fusion` | skip the fusion oracle | | `--bench`, `--bench-iters N`, `--bench-reps N`, `--bench-warm-ms N`, `--bench-flush MIB` | timing | | `--timing-only` | time every device kernel, check nothing (refuses `--record`) | | `--report FILE`, `--analysis FILE` | markdown report; splice in `tools/kbench-analysis.sh` output | | `-v` | every case, not only failures | Notes: `--max-bytes` matters on a loaded machine — one case holds each operand as drawn, as the kernel's copy, and on the device; without the cap a small machine is OOM-killed with no indication of the op. `--bench` times cold on purpose: an R9700's 52 MB bf16 weight read warm comes from cache at 1.45 TB/s against 565 GB/s from memory, so warm times rank shapes but do not size a roofline. A kernel that cannot lay out an encoding is a **skip**, not a failure (`RAD_E_DTYPE`), and the report names the kernel, operand and encoding. ## `rad-tune` — measure the instantiation space ``` rad-tune -m [options] -m, --model FILE the .rad whose declared graph supplies the shapes --home DIR $RADIANCE_HOME; cache at /tune/.tune --kernels A:B kernel libraries to load, first wins --op NAME only ops whose name starts with NAME --machine KEY override the machine key (writing a cache for another machine) --reps N timing reps, median reported (default 5) --iters N launches per rep (default 32) --max-tok N max_tok declare runs at (default 8192) --tp N tensor-parallel width to declare at (default 1) --num-speculative-tokens N|auto draft depth to declare at --dry-run list what would be measured -v, --verbose every candidate, not just the winner ``` Kernels declare tunable axes, not named variants; `rad-tune` walks the cross product at this model's shapes and writes the winner per (op, geometry) to the cache the selector reads at startup. - **Declare at the shapes you serve**: `--tp N` divides every sharded N; a declare at world size 1 measures a program no `--tp 2` serve runs. `--num-speculative-tokens` puts the drafter's kernels in the measured graph. - At `--tp > 1` every rank declares at once, each on its own thread and device: a collective's `init` is a rendezvous, so declaring rank 0 alone hangs. - A collective is measured as the exchange: per-rank operands (own seeds, so identical halves cannot pass), own scratch and stream, own instance; every rank fires at once and the clock stops when the last stream drains. Non-collective rows run on rank 0 alone. - `RADIANCE_TUNE` pins a kernel's axes to compare a tuned cache against the defaults — a measurement aid, not a deployment setting. ## `rad-schemas` — the op vocabulary diff ``` rad-schemas libref.so print one plugin's vocabulary rad-schemas libr4d.so libref.so diff them, in hierarchy order ``` The first plugin in hierarchy order to declare an op fixes its schema; a later plugin declaring the same op differently is refused at load, both named (arguments are positional, so disagreeing schemas would make one call mean different things). `rad-schemas` prints every disagreement at once, so a new plugin can be conformed before it is loaded beside another. It exits 1 on any disagreement. It links no core — it `dlopen`s plugins and reads static tables, so it works on a plugin the engine cannot load. ## `rad-avx-check` — host libraries without a card ``` rad-avx-check --ref libref.so --avx LIB.so [options] --ref PATH the oracle plugin (default libref.so) --avx PATH the plugin under test (default libavx.so) --op NAME only this op --level N only this ISA level (0 scalar, 1 avx, 2 avx2, 3 avx512) --threads N rerun every case with each parallel region on N threads (default 4; 0 skips) --tol X override every tolerance -v every case ``` Checks a host library op by op against `libref`, on the CPU, with neither engine nor card: two `dlopen`s, a geometry table (`libavx/avx_cases.cpp`), libref's operand descriptions and hand-built cases. For every case: schemas as the loader compares them, outputs within `rad-kbench`'s tolerances, guard bytes, inputs unchanged, and an oracle output that is not just the draw. `libavx` is also checked at every ISA level the CPU executes and with parallel regions forced onto several threads. A new host library is checked with the same command; it is treated as a host library (not `libavx`), so libref ops it has no row for are listed rather than failed, and the ISA/thread sweeps run only if it exports `avx_isa_force`/`avx_par_force`. A row returning `RAD_E_UNSUPPORTED` for a geometry libref serves fails: the selector would resolve to it and the step would fail at issue. ## `r4d_selftest` — libr4d's own check and timer ``` r4d_selftest [--case ] r4d_selftest --perf N K M [iters] (and the other --perf* modes in its usage) ``` With no mode it runs the catalogue and static checks, then each row's smoke and numeric cases in a child process of its own, with fusion claims compared byte for byte (`ctest` labels `slow`, `gpu`). `--case` runs one row in-process as the reproducer. `--perf` covers the fp8 decode GEMMs and head; `--perfmxd` the dense OCP-MXFP4 rows with the weight bandwidth each reaches; `--perfmx` the MXFP4 expert GEMM; `--cold MIB` times each launch alone behind a cache flush. `--perfar8` prints the eight-rank all-reduce shard weights and times `lockstep`/`ring`/`split` forms at every piece count; `R4D_SELFTEST_ONE_CARD=1` runs them on one card for their bits only. ## `rad-tokcheck` — tokeniser against its references ``` rad-tokcheck [options] ... --hf-dir DIR compare against HuggingFace ids in DIR/.ids (little-endian int32, encode(text, add_special_tokens=False)) --no-ref skip the reference encoder (slow) --user encode with parse_special off --turns N turns in the multi-turn measurement (default 40; 0 skips) --repeat N timing repeats, best taken (default 3) --context N ids shown either side of a difference (default 6) ``` Per corpus file: ids match `tools/tokref`'s reference encoder (llama.cpp's splitters and `std::regex`); ids match HuggingFace `tokenizers` read from files made elsewhere; and speed — MB/s for both encoders, plus the cost of re-tokenising a growing conversation with and without the segment cache. Any difference prints the ids and surrounding text and exits nonzero, so it gates a change. ## `rad-chatparse` — where a tool call goes wrong ``` rad-chatparse --request REQ.json --rad MODEL.rad --reply R.txt rad-chatparse --render-raw T.jinja --context CTX.json ``` Renders one OpenAI chat request as the server would (container template or `--override-chat-template`, `--chat-template-kwargs`, the plugin's declared reply format, or the template's own with `--no-plugin`), then reads each reply file as the model's output: parsed whole, parsed through the SSE stream as a client reassembles it, and run through the request's tool-call grammar. It prints one JSON object; exit 1 when the two parses disagree or the grammar refuses a token. No weights are loaded. ## The KL mode `radiance --kld-record/--kld-ref/--kld-corpus/--kld-out/--kld-lora` is a mode of the engine, not a separate tool: same placement and kernels, prefill instead of listening, then exit. The flags and the corpus format are on [[Radiance Conversion]]. ## Shell harnesses (`scripts/`) They drive a running engine: bring a model up, measure it, assert on what came back. Each header records why it measures the way it does. | script | what it does | |---|---| | `mcserve.sh`, `fnserve.sh` | bring MiniCPM5-2B+DSpark / Flash-Next up and leave it running (env knobs: `P TP MLEN SEQS M KVD HR MNBT EXTRA …`) | | `fnstop.sh` | stop whichever is running and wait for exit | | `convdspark.sh`, `convfn.sh` | rebuild the two reference containers (`convfn.sh` needs `CALIB=`) | | `hfget.sh` | resumable Hugging Face fetch with a content manifest | | `fnpf.sh`, `fnconc.sh`, `cstep2.sh`, `ident.sh`, `cardclock.py` | attach to a live server: prefill rate; prefill-vs-decode contention; steady-state decode ms/step per concurrency; run-to-run hashes; which clock limit binds and which rank the TP step waits for | | `fndrop.sh` | disconnect a client mid-request and check every other answer against a reference | | `tierident.sh`, `tiercross.sh` | whether a session that left VRAM returns byte for byte (attention and recurrent state), including two sessions in flight | | `mctools.sh`, `mcstream.sh`, `mcagent.sh`, `mcdecl.sh` | tool-call parsing shapes; streamed tool calls on raw SSE; speculation on an agentic workload; whether the drafter proposes | | `dprofd.sh`, `fntrace.sh`, `mctrace.sh`, `oracle.sh` | per-op device timing; rocprofv3 traces; every op against libref on the operands the model issued | | `kv-parity.sh`, `kv-stage-parity-64k.sh` | KV streaming: covering ring and staged prefill are byte-identical to streaming off | **Not all of them leave a server alone.** `fnserve.sh`, `mcserve.sh`, `fnstop.sh`, `fntrace.sh`, `dprofd.sh`, `mctrace.sh`, `mcagent.sh` and `mcdecl.sh` kill any running engine (`pkill` on the binary), whatever port it is on; `oracle.sh` stops whatever listens on its own port. `cstep2.sh`, `ident.sh`, `tierident.sh`, `tiercross.sh`, `mctools.sh`, `mcstream.sh`, `hfget.sh`, `convfn.sh`, `convdspark.sh`, `fnpf.sh`, `fnconc.sh`, `fndrop.sh` and `cardclock.py` leave it running (`fndrop.sh` deliberately disconnects clients). Never point the stopping ones at a production port. ## Related - [[Radiance Conversion]] — `rad-convert`, `rad-export`, the KL mode. - [[Radiance Plugins]] — what `rad-kbench`/`rad-tune`/`rad-schemas` check. - [[Radiance Operations]] — the startup receipt these tools corroborate.