323 lines
18 KiB
Markdown
323 lines
18 KiB
Markdown
---
|
||
type: "reference"
|
||
title: "Radiance Tools — inspect, check, tune, tokenise"
|
||
description: "The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports."
|
||
tags:
|
||
- llm
|
||
- radiance
|
||
- tools
|
||
- reference
|
||
timestamp: 2026-10-30
|
||
---
|
||
|
||
# Radiance tools
|
||
|
||
Six binaries and `rad-export` are installed beside `radiance`; `rad-avx-check`, `r4d_selftest` and
|
||
`rad-chatparse` live in the build tree. Every tool that loads plugins honours `RADIANCE_HOME`,
|
||
`RADIANCE_KERNELS` and `--kernels` as the engine does. A numeric option takes a whole number: `12k`,
|
||
`auto` (where the option does not name it) or an empty value exits 2.
|
||
|
||
`rad-convert`, `rad-kbench` and `rad-tune` run the engine's startup with a different last step: the
|
||
same plugins, in the same hierarchy, through the same declare phase, so they resolve kernels exactly
|
||
as a serve would.
|
||
|
||
## `rad-info` — what a container holds
|
||
|
||
```
|
||
rad-info [options] <model.rad>
|
||
rad-info --plugins [--home DIRS]
|
||
|
||
-v, --verbose list every tensor, not just the per-unit summary
|
||
--meta metadata table, then stop
|
||
--vocab vocab summary and normaliser chain, then stop
|
||
--profile N top N expert-profile entries (default 20, 0 = all)
|
||
--recipe the recipe the container was converted with, then stop
|
||
--plugins every plugin the search path offers, as the engine loads it
|
||
--home DIRS search path for --plugins (default $RADIANCE_HOME)
|
||
```
|
||
|
||
It opens the container read-only and loads no plugins (except for `--plugins`): a container names no
|
||
kernel library. Output is aligned; byte counts are binary units and each count says what it counts.
|
||
|
||
```sh
|
||
rad-info m.rad # format, architecture, model, source quant, recipe, sections
|
||
rad-info --recipe m.rad # the recipe, expanded, one rule a line
|
||
rad-info -v m.rad | head -40 # every tensor, encoding, shape and what made it
|
||
rad-info --plugins # kind, version, declarations, reference, device fit, file
|
||
```
|
||
|
||
The section table (strings, metadata, directory, planes, encodings, vocab, profile, data) is the
|
||
container's layout; the metadata table, the weight directory grouped by layer and movement unit, the
|
||
vocab, and the expert profile follow.
|
||
|
||
## `rad-kbench` — correctness, completeness, speed
|
||
|
||
```
|
||
rad-kbench [options] replay the recorded fixture (no model)
|
||
rad-kbench -m <model.rad> [options] check against a real container
|
||
rad-kbench -m <model.rad> --record FILE record a new fixture
|
||
rad-kbench -m <model.rad> --skeleton FILE write a metadata-only copy to record from
|
||
```
|
||
|
||
**Replay** needs no model, weights or card. Geometries, seeds and `libref`'s reduced answers are
|
||
checked in as `data/kernels.rkb` (dense model, block drafter, gqa 6) and `data/kernels_moe.rkb`
|
||
(routed MoE, MTP head, gqa 8). `--fixture` is repeatable; `ctest` runs three replays — the other
|
||
libraries against each recording, and `libref` against both at tolerance zero (its regression test,
|
||
since a live comparison moves both sides together).
|
||
|
||
Each library is asked for its row as the engine would ask it with that library alone: same
|
||
constraints, derived parameters and weight-encoding filter, so a row the engine would pass over does
|
||
not run here either (`--all-rows` runs every row whose constraints hold).
|
||
|
||
**Against a container** (`-m`), weight operands are read from it: each weight's canonical planes,
|
||
which the kernel gets through its own `layout`/`relayout` hooks and the oracle gets decoded, and
|
||
every expert of a weight table. This is the only test of the artifact and of the layout hooks.
|
||
|
||
**Recording** takes the reference's answer once at every point of every declared op, whichever
|
||
library resolved it. Weights are drawn from a seed, not read from the container, so any machine can
|
||
reproduce the reference and no card is needed. A skeleton (tables intact, data a hole) records the
|
||
same fixture in a few megabytes.
|
||
|
||
```sh
|
||
rad-kbench -m model.rad --skeleton model.skel.rad
|
||
rad-kbench -m model.skel.rad --max-cases 1 --record data/kernels.rkb
|
||
rad-kbench -m moe.rad --max-cases 1 --record data/kernels_moe.rkb
|
||
```
|
||
|
||
### Verdicts
|
||
|
||
- An output is compared on sampled positions, the full-tensor norm, and 32 seeded random-sign
|
||
projections (which estimate rel_l2 to about 25 %). The largest of the three is the error.
|
||
- Tolerances are per op and activation dtype (`dtype`, else `q_dtype`, else `kv_dtype`). Copies,
|
||
gathers, scatters and cache stores are exact. Every op in the fixtures has a row of its own.
|
||
- Ids are compared exactly; where the contract leaves a tie open (`sample_argmax`, `router_topk`,
|
||
`sample_topk`), an id indexing a value equal to the reference id's is accepted.
|
||
- A case whose reference and output are both identically zero checked nothing: **vacuous**, not
|
||
passed.
|
||
- A failure, a red-zone overrun, a fusion that differs from its chain, or a host kernel that wrote
|
||
into an input exits 1. `--strict` also fails a stale fixture.
|
||
- `--coverage DIR` keeps one file per fixture/library of the rows a replay reaches
|
||
(`data/kbench_rows/`); a listed row not reached fails, unless a memory skip kept its cases out.
|
||
- Every pass ends with a line saying where its time went (draw, hashing, layout, upload, launch,
|
||
read-back, comparison, cache, timing).
|
||
|
||
### Timing
|
||
|
||
`--bench` times every case that passes, on the bytes it was checked on.
|
||
|
||
- Device cases are timed **after** the checks, operands still on the device, after one sustained
|
||
warm-up. A time is device timestamps around a batch of back-to-back launches divided by the launch
|
||
count; the batch is queued behind a gate the host opens once submission is done, so neither the
|
||
host rate nor a sync is in the interval. Two untimed launches precede every batch; batches repeat
|
||
until the median's 95 % interval is within 1 % or `--bench-reps` is reached.
|
||
- **Cold by rotation**: weights and inputs of 1/32 of the footprint or more are read from copies
|
||
spanning the footprint (`--bench-flush`, default 256 MiB, at least twice an R9700's caches), so
|
||
each launch reads them from memory as streamed weights do. Smaller inputs stay warm, as the
|
||
previous op leaves them.
|
||
- In-place operands are restored from a pristine copy before every launch; the restores are timed
|
||
alone and subtracted.
|
||
- Timing draws expert offsets/routing skewed (Zipf 0.5) and block tables as permutations; the
|
||
checked launch keeps the recorded draw.
|
||
- The header prints the calibration: launch floor and streaming-read bandwidth warm and cold. If a
|
||
smaller cold probe runs >1.5× the largest's rate the rotation was cache-served and device times
|
||
are withheld. Throttler bits that came on during a batch repeat it, up to three times.
|
||
- Host cases are timed in their own column, after an untimed launch, weights rotated over at least
|
||
256 MiB (twice LLC), for at most two seconds a case.
|
||
|
||
### Flags
|
||
|
||
| flag | meaning |
|
||
|---|---|
|
||
| `--fixture FILE` | replay recording (default `$RADIANCE_HOME/kernels.rkb`); repeatable |
|
||
| `--record FILE` | with `-m`: write a fixture |
|
||
| `--skeleton FILE` | with `-m`: write a sparse container copy |
|
||
| `--rerecord FILE` | replay: write the fixtures' cases again with the reference's answers |
|
||
| `--record-rows FILE` | replay: record a row-driven fixture from the cases |
|
||
| `-m, --model FILE` | the `.rad` to read weight operands from |
|
||
| `--home DIR`, `--kernels A:B` | plugin home; kernel libraries (default every one found) |
|
||
| `--op NAME` | only this op (repeatable prefix match) |
|
||
| `--ref NAME` | the oracle plugin's name (default: the library declaring itself a reference) |
|
||
| `--no-ref` | replay without the reference among the tested libraries |
|
||
| `--strict` | replay: a stale fixture fails |
|
||
| `--all-rows` | run every row whose constraints hold |
|
||
| `--coverage DIR`, `--coverage-write` | the row-coverage ratchet |
|
||
| `--no-cache`, `--cache-dir DIR` | run the reference instead of reusing cached answers; cache location |
|
||
| `--seed N` | base seed; every case derives its own |
|
||
| `--tol X` | override every tolerance (bisecting only) |
|
||
| `--max-ctx N` | deepest context considered (declare and the attention sweep) |
|
||
| `--max-tok N` | `max_tok` declare runs at (default 512, small on purpose) |
|
||
| `--mm-max-patches N` | encoder pass declare runs at (default 2048 when a tower is present; 0 leaves it out) |
|
||
| `--kv-cache-dtype T` | `bf16` or `fp8`, as on the engine |
|
||
| `--num-speculative-tokens N` | draft depth to declare at (default: the container's) |
|
||
| `--max-cases N` | runs per distinct case (default 4; 0 = every one) |
|
||
| `--max-bytes MIB` | skip a case needing more than this, by name (default: half available memory, 512–8192) |
|
||
| `--no-fusion` | skip the fusion oracle |
|
||
| `--bench`, `--bench-iters N`, `--bench-reps N`, `--bench-warm-ms N`, `--bench-flush MIB` | timing |
|
||
| `--timing-only` | time every device kernel, check nothing (refuses `--record`) |
|
||
| `--report FILE`, `--analysis FILE` | markdown report; splice in `tools/kbench-analysis.sh` output |
|
||
| `-v` | every case, not only failures |
|
||
|
||
Notes: `--max-bytes` matters on a loaded machine — one case holds each operand as drawn, as the
|
||
kernel's copy, and on the device; without the cap a small machine is OOM-killed with no indication of
|
||
the op. `--bench` times cold on purpose: an R9700's 52 MB bf16 weight read warm comes from cache at
|
||
1.45 TB/s against 565 GB/s from memory, so warm times rank shapes but do not size a roofline. A
|
||
kernel that cannot lay out an encoding is a **skip**, not a failure (`RAD_E_DTYPE`), and the report
|
||
names the kernel, operand and encoding.
|
||
|
||
## `rad-tune` — measure the instantiation space
|
||
|
||
```
|
||
rad-tune -m <model.rad> [options]
|
||
|
||
-m, --model FILE the .rad whose declared graph supplies the shapes
|
||
--home DIR $RADIANCE_HOME; cache at <home>/tune/<machine>.tune
|
||
--kernels A:B kernel libraries to load, first wins
|
||
--op NAME only ops whose name starts with NAME
|
||
--machine KEY override the machine key (writing a cache for another machine)
|
||
--reps N timing reps, median reported (default 5)
|
||
--iters N launches per rep (default 32)
|
||
--max-tok N max_tok declare runs at (default 8192)
|
||
--tp N tensor-parallel width to declare at (default 1)
|
||
--num-speculative-tokens N|auto draft depth to declare at
|
||
--dry-run list what would be measured
|
||
-v, --verbose every candidate, not just the winner
|
||
```
|
||
|
||
Kernels declare tunable axes, not named variants; `rad-tune` walks the cross product at this model's
|
||
shapes and writes the winner per (op, geometry) to the cache the selector reads at startup.
|
||
|
||
- **Declare at the shapes you serve**: `--tp N` divides every sharded N; a declare at world size 1
|
||
measures a program no `--tp 2` serve runs. `--num-speculative-tokens` puts the drafter's kernels in
|
||
the measured graph.
|
||
- At `--tp > 1` every rank declares at once, each on its own thread and device: a collective's
|
||
`init` is a rendezvous, so declaring rank 0 alone hangs.
|
||
- A collective is measured as the exchange: per-rank operands (own seeds, so identical halves cannot
|
||
pass), own scratch and stream, own instance; every rank fires at once and the clock stops when the
|
||
last stream drains. Non-collective rows run on rank 0 alone.
|
||
- `RADIANCE_TUNE` pins a kernel's axes to compare a tuned cache against the defaults — a measurement
|
||
aid, not a deployment setting.
|
||
|
||
## `rad-schemas` — the op vocabulary diff
|
||
|
||
```
|
||
rad-schemas libref.so print one plugin's vocabulary
|
||
rad-schemas libr4d.so libref.so diff them, in hierarchy order
|
||
```
|
||
|
||
The first plugin in hierarchy order to declare an op fixes its schema; a later plugin declaring the
|
||
same op differently is refused at load, both named (arguments are positional, so disagreeing schemas
|
||
would make one call mean different things). `rad-schemas` prints every disagreement at once, so a new
|
||
plugin can be conformed before it is loaded beside another. It exits 1 on any disagreement. It links
|
||
no core — it `dlopen`s plugins and reads static tables, so it works on a plugin the engine cannot
|
||
load.
|
||
|
||
## `rad-avx-check` — host libraries without a card
|
||
|
||
```
|
||
rad-avx-check --ref libref.so --avx LIB.so [options]
|
||
|
||
--ref PATH the oracle plugin (default libref.so)
|
||
--avx PATH the plugin under test (default libavx.so)
|
||
--op NAME only this op
|
||
--level N only this ISA level (0 scalar, 1 avx, 2 avx2, 3 avx512)
|
||
--threads N rerun every case with each parallel region on N threads (default 4; 0 skips)
|
||
--tol X override every tolerance
|
||
-v every case
|
||
```
|
||
|
||
Checks a host library op by op against `libref`, on the CPU, with neither engine nor card: two
|
||
`dlopen`s, a geometry table (`libavx/avx_cases.cpp`), libref's operand descriptions and hand-built
|
||
cases. For every case: schemas as the loader compares them, outputs within `rad-kbench`'s tolerances,
|
||
guard bytes, inputs unchanged, and an oracle output that is not just the draw. `libavx` is also
|
||
checked at every ISA level the CPU executes and with parallel regions forced onto several threads.
|
||
A new host library is checked with the same command; it is treated as a host library (not `libavx`),
|
||
so libref ops it has no row for are listed rather than failed, and the ISA/thread sweeps run only if
|
||
it exports `avx_isa_force`/`avx_par_force`. A row returning `RAD_E_UNSUPPORTED` for a geometry libref
|
||
serves fails: the selector would resolve to it and the step would fail at issue.
|
||
|
||
## `r4d_selftest` — libr4d's own check and timer
|
||
|
||
```
|
||
r4d_selftest <libr4d.so> [--case <row>]
|
||
r4d_selftest <libr4d.so> --perf N K M [iters] (and the other --perf* modes in its usage)
|
||
```
|
||
|
||
With no mode it runs the catalogue and static checks, then each row's smoke and numeric cases in a
|
||
child process of its own, with fusion claims compared byte for byte (`ctest` labels `slow`, `gpu`).
|
||
`--case` runs one row in-process as the reproducer. `--perf` covers the fp8 decode GEMMs and head;
|
||
`--perfmxd` the dense OCP-MXFP4 rows with the weight bandwidth each reaches; `--perfmx` the MXFP4
|
||
expert GEMM; `--cold MIB` times each launch alone behind a cache flush. `--perfar8` prints the
|
||
eight-rank all-reduce shard weights and times `lockstep`/`ring`/`split` forms at every piece count;
|
||
`R4D_SELFTEST_ONE_CARD=1` runs them on one card for their bits only.
|
||
|
||
## `rad-tokcheck` — tokeniser against its references
|
||
|
||
```
|
||
rad-tokcheck [options] <tokenizer.json> <corpus-file>...
|
||
|
||
--hf-dir DIR compare against HuggingFace ids in DIR/<corpus basename>.ids
|
||
(little-endian int32, encode(text, add_special_tokens=False))
|
||
--no-ref skip the reference encoder (slow)
|
||
--user encode with parse_special off
|
||
--turns N turns in the multi-turn measurement (default 40; 0 skips)
|
||
--repeat N timing repeats, best taken (default 3)
|
||
--context N ids shown either side of a difference (default 6)
|
||
```
|
||
|
||
Per corpus file: ids match `tools/tokref`'s reference encoder (llama.cpp's splitters and
|
||
`std::regex`); ids match HuggingFace `tokenizers` read from files made elsewhere; and speed — MB/s
|
||
for both encoders, plus the cost of re-tokenising a growing conversation with and without the
|
||
segment cache. Any difference prints the ids and surrounding text and exits nonzero, so it gates a
|
||
change.
|
||
|
||
## `rad-chatparse` — where a tool call goes wrong
|
||
|
||
```
|
||
rad-chatparse --request REQ.json --rad MODEL.rad --reply R.txt
|
||
rad-chatparse --render-raw T.jinja --context CTX.json
|
||
```
|
||
|
||
Renders one OpenAI chat request as the server would (container template or
|
||
`--override-chat-template`, `--chat-template-kwargs`, the plugin's declared reply format, or the
|
||
template's own with `--no-plugin`), then reads each reply file as the model's output: parsed whole,
|
||
parsed through the SSE stream as a client reassembles it, and run through the request's tool-call
|
||
grammar. It prints one JSON object; exit 1 when the two parses disagree or the grammar refuses a
|
||
token. No weights are loaded.
|
||
|
||
## The KL mode
|
||
|
||
`radiance --kld-record/--kld-ref/--kld-corpus/--kld-out/--kld-lora` is a mode of the engine, not a
|
||
separate tool: same placement and kernels, prefill instead of listening, then exit. The flags and the
|
||
corpus format are on [[Radiance Conversion]].
|
||
|
||
## Shell harnesses (`scripts/`)
|
||
|
||
They drive a running engine: bring a model up, measure it, assert on what came back. Each header
|
||
records why it measures the way it does.
|
||
|
||
| script | what it does |
|
||
|---|---|
|
||
| `mcserve.sh`, `fnserve.sh` | bring MiniCPM5-2B+DSpark / Flash-Next up and leave it running (env knobs: `P TP MLEN SEQS M KVD HR MNBT EXTRA …`) |
|
||
| `fnstop.sh` | stop whichever is running and wait for exit |
|
||
| `convdspark.sh`, `convfn.sh` | rebuild the two reference containers (`convfn.sh` needs `CALIB=`) |
|
||
| `hfget.sh` | resumable Hugging Face fetch with a content manifest |
|
||
| `fnpf.sh`, `fnconc.sh`, `cstep2.sh`, `ident.sh`, `cardclock.py` | attach to a live server: prefill rate; prefill-vs-decode contention; steady-state decode ms/step per concurrency; run-to-run hashes; which clock limit binds and which rank the TP step waits for |
|
||
| `fndrop.sh` | disconnect a client mid-request and check every other answer against a reference |
|
||
| `tierident.sh`, `tiercross.sh` | whether a session that left VRAM returns byte for byte (attention and recurrent state), including two sessions in flight |
|
||
| `mctools.sh`, `mcstream.sh`, `mcagent.sh`, `mcdecl.sh` | tool-call parsing shapes; streamed tool calls on raw SSE; speculation on an agentic workload; whether the drafter proposes |
|
||
| `dprofd.sh`, `fntrace.sh`, `mctrace.sh`, `oracle.sh` | per-op device timing; rocprofv3 traces; every op against libref on the operands the model issued |
|
||
| `kv-parity.sh`, `kv-stage-parity-64k.sh` | KV streaming: covering ring and staged prefill are byte-identical to streaming off |
|
||
|
||
**Not all of them leave a server alone.** `fnserve.sh`, `mcserve.sh`, `fnstop.sh`, `fntrace.sh`,
|
||
`dprofd.sh`, `mctrace.sh`, `mcagent.sh` and `mcdecl.sh` kill any running engine (`pkill` on the
|
||
binary), whatever port it is on; `oracle.sh` stops whatever listens on its own port. `cstep2.sh`,
|
||
`ident.sh`, `tierident.sh`, `tiercross.sh`, `mctools.sh`, `mcstream.sh`, `hfget.sh`, `convfn.sh`,
|
||
`convdspark.sh`, `fnpf.sh`, `fnconc.sh`, `fndrop.sh` and `cardclock.py` leave it running
|
||
(`fndrop.sh` deliberately disconnects clients). Never point the stopping ones at a production port.
|
||
|
||
## Related
|
||
|
||
- [[Radiance Conversion]] — `rad-convert`, `rad-export`, the KL mode.
|
||
- [[Radiance Plugins]] — what `rad-kbench`/`rad-tune`/`rad-schemas` check.
|
||
- [[Radiance Operations]] — the startup receipt these tools corroborate.
|