serverAppConfigs/serverAppConfigs-docs/LLM/Radiance Tools.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	Radiance Tools — inspect, check, tune, tokenise	The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports.	llm radiance tools reference	2026-10-30

Radiance tools

Six binaries and rad-export are installed beside radiance; rad-avx-check, r4d_selftest and rad-chatparse live in the build tree. Every tool that loads plugins honours RADIANCE_HOME, RADIANCE_KERNELS and --kernels as the engine does. A numeric option takes a whole number: 12k, auto (where the option does not name it) or an empty value exits 2.

rad-convert, rad-kbench and rad-tune run the engine's startup with a different last step: the same plugins, in the same hierarchy, through the same declare phase, so they resolve kernels exactly as a serve would.

rad-info — what a container holds

rad-info [options] <model.rad>
rad-info --plugins [--home DIRS]

  -v, --verbose      list every tensor, not just the per-unit summary
      --meta         metadata table, then stop
      --vocab        vocab summary and normaliser chain, then stop
      --profile N    top N expert-profile entries (default 20, 0 = all)
      --recipe       the recipe the container was converted with, then stop
      --plugins      every plugin the search path offers, as the engine loads it
      --home DIRS    search path for --plugins (default $RADIANCE_HOME)

It opens the container read-only and loads no plugins (except for --plugins): a container names no kernel library. Output is aligned; byte counts are binary units and each count says what it counts.

rad-info m.rad                    # format, architecture, model, source quant, recipe, sections
rad-info --recipe m.rad           # the recipe, expanded, one rule a line
rad-info -v m.rad | head -40      # every tensor, encoding, shape and what made it
rad-info --plugins                # kind, version, declarations, reference, device fit, file

The section table (strings, metadata, directory, planes, encodings, vocab, profile, data) is the container's layout; the metadata table, the weight directory grouped by layer and movement unit, the vocab, and the expert profile follow.

rad-kbench — correctness, completeness, speed

rad-kbench [options]                       replay the recorded fixture (no model)
rad-kbench -m <model.rad> [options]        check against a real container
rad-kbench -m <model.rad> --record FILE    record a new fixture
rad-kbench -m <model.rad> --skeleton FILE  write a metadata-only copy to record from

Replay needs no model, weights or card. Geometries, seeds and libref's reduced answers are checked in as data/kernels.rkb (dense model, block drafter, gqa 6) and data/kernels_moe.rkb (routed MoE, MTP head, gqa 8). --fixture is repeatable; ctest runs three replays — the other libraries against each recording, and libref against both at tolerance zero (its regression test, since a live comparison moves both sides together).

Each library is asked for its row as the engine would ask it with that library alone: same constraints, derived parameters and weight-encoding filter, so a row the engine would pass over does not run here either (--all-rows runs every row whose constraints hold).

Against a container (-m), weight operands are read from it: each weight's canonical planes, which the kernel gets through its own layout/relayout hooks and the oracle gets decoded, and every expert of a weight table. This is the only test of the artifact and of the layout hooks.

Recording takes the reference's answer once at every point of every declared op, whichever library resolved it. Weights are drawn from a seed, not read from the container, so any machine can reproduce the reference and no card is needed. A skeleton (tables intact, data a hole) records the same fixture in a few megabytes.

rad-kbench -m model.rad --skeleton model.skel.rad
rad-kbench -m model.skel.rad --max-cases 1 --record data/kernels.rkb
rad-kbench -m moe.rad        --max-cases 1 --record data/kernels_moe.rkb

Verdicts

• An output is compared on sampled positions, the full-tensor norm, and 32 seeded random-sign projections (which estimate rel_l2 to about 25 %). The largest of the three is the error.
• Tolerances are per op and activation dtype (dtype, else q_dtype, else kv_dtype). Copies, gathers, scatters and cache stores are exact. Every op in the fixtures has a row of its own.
• Ids are compared exactly; where the contract leaves a tie open (sample_argmax, router_topk, sample_topk), an id indexing a value equal to the reference id's is accepted.
• A case whose reference and output are both identically zero checked nothing: vacuous, not passed.
• A failure, a red-zone overrun, a fusion that differs from its chain, or a host kernel that wrote into an input exits 1. --strict also fails a stale fixture.
• --coverage DIR keeps one file per fixture/library of the rows a replay reaches (data/kbench_rows/); a listed row not reached fails, unless a memory skip kept its cases out.
• Every pass ends with a line saying where its time went (draw, hashing, layout, upload, launch, read-back, comparison, cache, timing).

Timing

--bench times every case that passes, on the bytes it was checked on.

• Device cases are timed after the checks, operands still on the device, after one sustained warm-up. A time is device timestamps around a batch of back-to-back launches divided by the launch count; the batch is queued behind a gate the host opens once submission is done, so neither the host rate nor a sync is in the interval. Two untimed launches precede every batch; batches repeat until the median's 95 % interval is within 1 % or --bench-reps is reached.
• Cold by rotation: weights and inputs of 1/32 of the footprint or more are read from copies spanning the footprint (--bench-flush, default 256 MiB, at least twice an R9700's caches), so each launch reads them from memory as streamed weights do. Smaller inputs stay warm, as the previous op leaves them.
• In-place operands are restored from a pristine copy before every launch; the restores are timed alone and subtracted.
• Timing draws expert offsets/routing skewed (Zipf 0.5) and block tables as permutations; the checked launch keeps the recorded draw.
• The header prints the calibration: launch floor and streaming-read bandwidth warm and cold. If a smaller cold probe runs >1.5× the largest's rate the rotation was cache-served and device times are withheld. Throttler bits that came on during a batch repeat it, up to three times.
• Host cases are timed in their own column, after an untimed launch, weights rotated over at least 256 MiB (twice LLC), for at most two seconds a case.

Flags

flag	meaning
--fixture FILE	replay recording (default $RADIANCE_HOME/kernels.rkb); repeatable
--record FILE	with -m: write a fixture
--skeleton FILE	with -m: write a sparse container copy
--rerecord FILE	replay: write the fixtures' cases again with the reference's answers
--record-rows FILE	replay: record a row-driven fixture from the cases
-m, --model FILE	the .rad to read weight operands from
--home DIR, --kernels A:B	plugin home; kernel libraries (default every one found)
--op NAME	only this op (repeatable prefix match)
--ref NAME	the oracle plugin's name (default: the library declaring itself a reference)
--no-ref	replay without the reference among the tested libraries
--strict	replay: a stale fixture fails
--all-rows	run every row whose constraints hold
--coverage DIR, --coverage-write	the row-coverage ratchet
--no-cache, --cache-dir DIR	run the reference instead of reusing cached answers; cache location
--seed N	base seed; every case derives its own
--tol X	override every tolerance (bisecting only)
--max-ctx N	deepest context considered (declare and the attention sweep)
--max-tok N	max_tok declare runs at (default 512, small on purpose)
--mm-max-patches N	encoder pass declare runs at (default 2048 when a tower is present; 0 leaves it out)
--kv-cache-dtype T	bf16 or fp8, as on the engine
--num-speculative-tokens N	draft depth to declare at (default: the container's)
--max-cases N	runs per distinct case (default 4; 0 = every one)
--max-bytes MIB	skip a case needing more than this, by name (default: half available memory, 512–8192)
--no-fusion	skip the fusion oracle
--bench, --bench-iters N, --bench-reps N, --bench-warm-ms N, --bench-flush MIB	timing
--timing-only	time every device kernel, check nothing (refuses --record)
--report FILE, --analysis FILE	markdown report; splice in tools/kbench-analysis.sh output
-v	every case, not only failures

Notes: --max-bytes matters on a loaded machine — one case holds each operand as drawn, as the kernel's copy, and on the device; without the cap a small machine is OOM-killed with no indication of the op. --bench times cold on purpose: an R9700's 52 MB bf16 weight read warm comes from cache at 1.45 TB/s against 565 GB/s from memory, so warm times rank shapes but do not size a roofline. A kernel that cannot lay out an encoding is a skip, not a failure (RAD_E_DTYPE), and the report names the kernel, operand and encoding.

rad-tune — measure the instantiation space

rad-tune -m <model.rad> [options]

  -m, --model FILE   the .rad whose declared graph supplies the shapes
      --home DIR     $RADIANCE_HOME; cache at <home>/tune/<machine>.tune
      --kernels A:B  kernel libraries to load, first wins
      --op NAME      only ops whose name starts with NAME
      --machine KEY  override the machine key (writing a cache for another machine)
      --reps N       timing reps, median reported (default 5)
      --iters N      launches per rep (default 32)
      --max-tok N    max_tok declare runs at (default 8192)
      --tp N         tensor-parallel width to declare at (default 1)
      --num-speculative-tokens N|auto   draft depth to declare at
      --dry-run      list what would be measured
  -v, --verbose      every candidate, not just the winner

Kernels declare tunable axes, not named variants; rad-tune walks the cross product at this model's shapes and writes the winner per (op, geometry) to the cache the selector reads at startup.

• Declare at the shapes you serve: --tp N divides every sharded N; a declare at world size 1 measures a program no --tp 2 serve runs. --num-speculative-tokens puts the drafter's kernels in the measured graph.
• At --tp > 1 every rank declares at once, each on its own thread and device: a collective's init is a rendezvous, so declaring rank 0 alone hangs.
• A collective is measured as the exchange: per-rank operands (own seeds, so identical halves cannot pass), own scratch and stream, own instance; every rank fires at once and the clock stops when the last stream drains. Non-collective rows run on rank 0 alone.
• RADIANCE_TUNE pins a kernel's axes to compare a tuned cache against the defaults — a measurement aid, not a deployment setting.

rad-schemas — the op vocabulary diff

rad-schemas libref.so                 print one plugin's vocabulary
rad-schemas libr4d.so libref.so       diff them, in hierarchy order

The first plugin in hierarchy order to declare an op fixes its schema; a later plugin declaring the same op differently is refused at load, both named (arguments are positional, so disagreeing schemas would make one call mean different things). rad-schemas prints every disagreement at once, so a new plugin can be conformed before it is loaded beside another. It exits 1 on any disagreement. It links no core — it dlopens plugins and reads static tables, so it works on a plugin the engine cannot load.

rad-avx-check — host libraries without a card

rad-avx-check --ref libref.so --avx LIB.so [options]

  --ref PATH     the oracle plugin      (default libref.so)
  --avx PATH     the plugin under test  (default libavx.so)
  --op NAME      only this op
  --level N      only this ISA level (0 scalar, 1 avx, 2 avx2, 3 avx512)
  --threads N    rerun every case with each parallel region on N threads (default 4; 0 skips)
  --tol X        override every tolerance
  -v             every case

Checks a host library op by op against libref, on the CPU, with neither engine nor card: two dlopens, a geometry table (libavx/avx_cases.cpp), libref's operand descriptions and hand-built cases. For every case: schemas as the loader compares them, outputs within rad-kbench's tolerances, guard bytes, inputs unchanged, and an oracle output that is not just the draw. libavx is also checked at every ISA level the CPU executes and with parallel regions forced onto several threads. A new host library is checked with the same command; it is treated as a host library (not libavx), so libref ops it has no row for are listed rather than failed, and the ISA/thread sweeps run only if it exports avx_isa_force/avx_par_force. A row returning RAD_E_UNSUPPORTED for a geometry libref serves fails: the selector would resolve to it and the step would fail at issue.

r4d_selftest — libr4d's own check and timer

r4d_selftest <libr4d.so> [--case <row>]
r4d_selftest <libr4d.so> --perf N K M [iters]      (and the other --perf* modes in its usage)

With no mode it runs the catalogue and static checks, then each row's smoke and numeric cases in a child process of its own, with fusion claims compared byte for byte (ctest labels slow, gpu). --case runs one row in-process as the reproducer. --perf covers the fp8 decode GEMMs and head; --perfmxd the dense OCP-MXFP4 rows with the weight bandwidth each reaches; --perfmx the MXFP4 expert GEMM; --cold MIB times each launch alone behind a cache flush. --perfar8 prints the eight-rank all-reduce shard weights and times lockstep/ring/split forms at every piece count; R4D_SELFTEST_ONE_CARD=1 runs them on one card for their bits only.

rad-tokcheck — tokeniser against its references

rad-tokcheck [options] <tokenizer.json> <corpus-file>...

  --hf-dir DIR    compare against HuggingFace ids in DIR/<corpus basename>.ids
                  (little-endian int32, encode(text, add_special_tokens=False))
  --no-ref        skip the reference encoder (slow)
  --user          encode with parse_special off
  --turns N       turns in the multi-turn measurement (default 40; 0 skips)
  --repeat N      timing repeats, best taken (default 3)
  --context N     ids shown either side of a difference (default 6)

Per corpus file: ids match tools/tokref's reference encoder (llama.cpp's splitters and std::regex); ids match HuggingFace tokenizers read from files made elsewhere; and speed — MB/s for both encoders, plus the cost of re-tokenising a growing conversation with and without the segment cache. Any difference prints the ids and surrounding text and exits nonzero, so it gates a change.

rad-chatparse — where a tool call goes wrong

rad-chatparse --request REQ.json --rad MODEL.rad --reply R.txt
rad-chatparse --render-raw T.jinja --context CTX.json

Renders one OpenAI chat request as the server would (container template or --override-chat-template, --chat-template-kwargs, the plugin's declared reply format, or the template's own with --no-plugin), then reads each reply file as the model's output: parsed whole, parsed through the SSE stream as a client reassembles it, and run through the request's tool-call grammar. It prints one JSON object; exit 1 when the two parses disagree or the grammar refuses a token. No weights are loaded.

The KL mode

radiance --kld-record/--kld-ref/--kld-corpus/--kld-out/--kld-lora is a mode of the engine, not a separate tool: same placement and kernels, prefill instead of listening, then exit. The flags and the corpus format are on Radiance Conversion.

Shell harnesses (scripts/)

They drive a running engine: bring a model up, measure it, assert on what came back. Each header records why it measures the way it does.

script	what it does
mcserve.sh, fnserve.sh	bring MiniCPM5-2B+DSpark / Flash-Next up and leave it running (env knobs: P TP MLEN SEQS M KVD HR MNBT EXTRA …)
fnstop.sh	stop whichever is running and wait for exit
convdspark.sh, convfn.sh	rebuild the two reference containers (convfn.sh needs CALIB=)
hfget.sh	resumable Hugging Face fetch with a content manifest
fnpf.sh, fnconc.sh, cstep2.sh, ident.sh, cardclock.py	attach to a live server: prefill rate; prefill-vs-decode contention; steady-state decode ms/step per concurrency; run-to-run hashes; which clock limit binds and which rank the TP step waits for
fndrop.sh	disconnect a client mid-request and check every other answer against a reference
tierident.sh, tiercross.sh	whether a session that left VRAM returns byte for byte (attention and recurrent state), including two sessions in flight
mctools.sh, mcstream.sh, mcagent.sh, mcdecl.sh	tool-call parsing shapes; streamed tool calls on raw SSE; speculation on an agentic workload; whether the drafter proposes
dprofd.sh, fntrace.sh, mctrace.sh, oracle.sh	per-op device timing; rocprofv3 traces; every op against libref on the operands the model issued
kv-parity.sh, kv-stage-parity-64k.sh	KV streaming: covering ring and staged prefill are byte-identical to streaming off

Not all of them leave a server alone. fnserve.sh, mcserve.sh, fnstop.sh, fntrace.sh, dprofd.sh, mctrace.sh, mcagent.sh and mcdecl.sh kill any running engine (pkill on the binary), whatever port it is on; oracle.sh stops whatever listens on its own port. cstep2.sh, ident.sh, tierident.sh, tiercross.sh, mctools.sh, mcstream.sh, hfget.sh, convfn.sh, convdspark.sh, fnpf.sh, fnconc.sh, fndrop.sh and cardclock.py leave it running (fndrop.sh deliberately disconnects clients). Never point the stopping ones at a production port.

Related

• Radiance Conversion — rad-convert, rad-export, the KL mode.
• Radiance Plugins — what rad-kbench/rad-tune/rad-schemas check.
• Radiance Operations — the startup receipt these tools corroborate.