--- type: "howto" title: "Radiance Conversion — checkpoints into .rad containers" description: "Turning a checkpoint into a .rad container: rad-convert, the recipe language, quantisers, reuse and export." tags: - llm - radiance - quantization - howto timestamp: 2026-10-30 --- # Radiance conversion `rad-convert` turns a Hugging Face checkpoint into a `.rad` container, choosing each weight's encoding from a **recipe**. `rad-export` wraps it for finetunes (a merged checkpoint or a PEFT adapter) with the stock model's recipe. Neither opens a GPU: conversion is CPU-only and safe beside a running server. Fetch checkpoints with `scripts/hfget.sh ` (resumable; reads `.safetensors`, skips `.bin`/ONNX/GGUF when safetensors exist; `HF_TOKEN` for gated repos; writes a `.hfget-manifest` of content keys). `radiance --model ` runs the same fetch itself. ## What a conversion does 1. Read `config.json` (or GGUF metadata), and the shard index if present. 2. Load the architecture, kernel and quantiser plugins and run **declare**, as the engine does. 3. Resolve every op and ask each resolved kernel's `layout` hook about the planned encoding. A recipe no kernel can serve is refused **here**, before any I/O. 4. Write each logical weight once: the checkpoint's bytes where no rule matches, otherwise the planes the matching quantiser writes, a block of rows at a time. 5. Measure every quantised weight against its source through the core's decoder and print the worst. 6. Bake the tokeniser, chat template, metadata, recipe and any draft head. ```sh build/bin/rad-convert /srv/models/hf/Qwen/Qwen3.8-27B -o qwen3.8-27b-bf16.rad \ --recipe data/recipes/q38-27b-bf16.recipe build/bin/rad-convert /srv/models/hf/Qwen/Qwen3.8-Flash-Next -o flashnext-bf16.rad --max-ctx 4096 ``` ## Every `rad-convert` option | option | meaning | |---|---| | `` | a `.gguf`, a `.safetensors`, an `index.json`, or a checkpoint directory | | `-o, --out FILE` | the container to write | | `--recipe FILE` | one rule a line: `PATTERN QUANTISER key=value …`; repeatable; first match decides | | `--quant RULE` | `PATTERN=QUANTISER[:key=value,…]`, placed **ahead of** the recipe's rules; repeatable | | `--list-quantizers` | every loaded quantiser and the options it takes, then exit | | `--arch ID` | override the architecture id read from the checkpoint | | `--home DIR` | `$RADIANCE_HOME` (kernels, architectures, quantizers) | | `--kernels A:B` | kernel libraries to load, first wins; declare checks the recipe against them | | `--tokenizer F` | `tokenizer.json`; its declared chain is baked instead of reconstructed from a pre-tokenizer name. Required for a GGUF, without it the chain is a guess | | `--tokenizer2 F` | a second `tokenizer.json` baked as the second vocab (a generator reading its prompt through two tokenisers); keep `tokenizer_config.json` beside it | | `--add FILE` | merge one more `.safetensors`, tensor names kept; repeatable; a name in two files is refused | | `--config DIR` | read `config.json`, `generation_config.json`, `tokenizer.json` from DIR for a single-file checkpoint | | `--chat-template F` | a Jinja template baked in, overriding the checkpoint's (or its absence); trailing newlines stripped; without one `/v1/chat/completions` answers 501 | | `--imatrix F` | llama.cpp imatrix: expert-popularity profile, per-column importances, error weighting; required by `ggml` types `iq2_xxs`, `iq2_xs`, `iq1_s` | | `--set K=V` | a metadata key, seen by declare **and** written into the container; repeatable | | `--max-tok N` | the `max_tok` declare runs at (default 8192, or a generator's largest pass) | | `--max-ctx N` | the context declare runs at (default: the checkpoint's training context) | | `--max-spec N` | the `max_spec` declare runs at (default 4, or the block a drafter states); `0` leaves a draft head out | | `--draft-model DIR` | a drafter shipped as its own repository, merged under `draft.` | | `--kv-cache-dtype T` | `bf16` or `fp8`: the cache width declare runs at; the container is identical at either | | `--reuse FILE` | copy every weight an earlier container holds with the same encoding, quantiser, options and shape | | `--changed FILE` | with `--reuse`: checkpoint tensors (one name a line) that differ; every weight reading one is converted | | `--in-place` | with `--reuse`: extend that container; new weights go past its end, the header last; keeps `.pre-append` | | `--lora FILE\|DIR` | merge a PEFT adapter into the checkpoint tensors before converting (`W + strength · scale · B·A`); repeatable, merged in order; each merged tensor joins `--changed` | | `--lora-strength F` | strength for every `--lora` (default 1) | | `--lora-map FILE` | JSON map for adapters whose module names differ: `{"strip":[prefix…],"modules":{"":[target…]}}`; one `*` matches a block index; several targets mean a fused module (rows split in order) | | `--plan-only` | declare, plan every encoding, print the plan, stop before writing | | `--report N` | print the N worst quantisation errors (default 20) | | `-v, --verbose` | per-weight detail | `--reuse` matches a weight on its **encoding, quantiser, options and shape — not on its source bytes** — and carries whatever calibration made the old planes. Use it only when source and calibration are unchanged, or name the changed tensors with `--changed`. `--in-place` refuses to change a weight the container holds or to add an expert. ## The recipe language ``` # PATTERN QUANTISER OPTIONS blk.*.ffn_*_exps.*.weight gptq codes=i4 group=128 scale=bf16 transform=fwht128 calib=$CALIB blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4 output.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16 blk.*.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16 ``` - One rule a line: `PATTERN QUANTISER [key=value …]`. `#` starts a comment. - `PATTERN` is a glob over the **whole logical weight name**: `*` crosses dots, `?` is one character. - **The first matching rule decides.** No rule → the checkpoint's own encoding, which must be one the engine serves as is: `bf16`, `f16`, `f32`, or block-scaled FP8 (`X.weight` with `X.weight_scale_inv`). Anything else is named and refused. - `$NAME` / `${NAME}` in a value comes from the environment; an **unset variable is refused**, never expanded empty (so an empty `$CALIB` cannot silently produce an uncalibrated convert). - A quantiser may **decline** a weight (`RAD_E_UNSUPPORTED`), which keeps the checkpoint's encoding — `rtn` declines a rank-1 tensor, so a pattern that also matches a norm leaves the norm alone. - An option the quantiser does not declare is refused, by name, before anything is quantised. - Two plugins naming the same quantiser are refused, both named. ## Quantisers | quantiser | what it does | |---|---| | `rtn` | round to nearest on a grid; each block's scale chosen by `rule` | | `gptq` | error feedback against the layer's input Hessian (`calib=DIR`, damped by `damp` = 0.01 of the mean diagonal by default); `act_order=1` quantises columns by descending input energy and stores the permutation; `cd=N` adds coordinate-descent sweeps | | `awq` | per-input-column scaling to a searched power, then a per-block clip search, both against the Hessian; column scales are the last scale level | | `autoround` | sign-gradient descent on a rounding offset per value and a range factor per block; `iters` steps; slow (a 27B dense model takes most of a working day at defaults; `iters=50` a quarter) | | `paroquant` | channel scaling plus `stages` Givens rotations learned against the Hessian, then AutoRound on the rotated weight; the `givens` transform | | `m2xfp4` | M2XFP's weight-side Sg-EM: MXFP4 plus one metadata byte per group of 32; its own `m2xfp4` scheme; `adaptive=0` pins the bias, `wide=1` widens it | | `ggml` | a llama.cpp format (`q4_0`, `q4_1`, `q5_0`, `q5_1`, `q8_0`, `q2_K`…`q6_K`, `iq4_nl`, `iq4_xs`, `iq2_xxs`, `iq2_xs`, `iq2_s`, `iq3_xxs`, `iq3_s`, `iq1_s`, `iq1_m`), by llama.cpp's own search, imatrix-weighted, codes byte for byte | | `cast` | a float weight in another float dtype (`dtype=` f32, bf16, f16, fp8_e4m3, fp8_e5m2), no scale | | `import` | rad-convert's own, not a plugin: write the numbers a quantised release ships, in the spelling the kernels read, instead of quantising again | Every quantiser except `m2xfp4`, `ggml`, `cast` and `import` takes the grid options: | option | values | |---|---| | `codes` | `i2`…`i8`, `i16`, `i32` symmetric; `u1`…`u8`, `u16` with `zero=`; `fp8_e4m3`, `fp8_e5m2`, `fp6_e2m3`, `fp6_e3m2`, `fp4_e2m1` | | `table` | a codebook the codes index: `nf4`, `iq4nl`, `w4nl`, `w5nl`, `binary`, `ternary`, or `;`-separated values | | `group=N` / `block=RxC` | where the scales sit; `*` is a whole extent | | `scale` | f32, bf16, f16, fp8_e4m3, e8m0; under `scale2` also an integer sub-scale `u4`…`u8`, `i6`, `i8` | | `scale2`, `group2` / `block2` | a second scale level the first is relative to (default the whole tensor) | | `scale2_value` | the second level fixed at this value, exact in `scale2`'s dtype | | `zero` | an integer zero point per block — an asymmetric grid | | `transform=fwhtN` | rotate each row by the Hadamard first | | `rule` | `absmax`, `mx` (OCP shared exponent), `fixed` (`scale_value=`), `search` (least squared error over candidates and refits, imatrix-weighted) | | `clamp` | a signed code's range: `full` or `sym` | Named grids: MXFP4 = `codes=fp4_e2m1 group=32 scale=e8m0`; NVFP4 = `codes=fp4_e2m1 group=16 scale=fp8_e4m3 scale2=f32`; NF4 = `table=nf4 group=64`; a K-quant-like grid = `codes=i4 group=32 scale=u6 scale2=f16 group2=256`; a ternary BitNet weight = `table=ternary`. **A container can hold only what some kernel reads.** rad-convert checks every planned encoding against the `layout` hooks of the kernels declare resolved and refuses one no kernel accepts. libr4d reads the plain floats; block-scaled fp8 and fp8 rows; int8 block rows plain or rotated; routed experts as fp8, int4 (`i4*bf16[1x128]/fwht128`) or the `w4nl`/`w5nl` codebooks under E4M3 scales; the dense w4, w8 and mxfp4 GEMM rows; and the 2-bit draft head. See [[Radiance Plugins]] for the encoding model. ## Importing a release someone else quantised These forms are read weight by weight and written in the spelling the kernels read; a recipe rule re-quantises from the exact dequantisation where the family's kernels want another form. | `quantization_config` | read as | notes | |---|---|---| | `quark` MXFP4 (fp4, group 32, e8m0) | `fp4_e2m1*e8m0[1x32]` | byte for byte; also servable as it stands | | `paroquant_mxfp4` | `fp4_e2m1*e8m0[1x32]/givens` | codes/exponents byte for byte plus learned rotations as `givens` tables | | `paroquant`, `int5-bitplane`, plain `paroquant` | `u5*f16[1x128]-u8[1x128]/givens`, `u4*…` | AWQ-packed nibbles re-laid as a code bit stream; f16 scales, integer zeros, rotations | | `compressed-tensors` `pack-quantized`, 4-bit `int`, group, act-order none/static/weight | `i4*[1xG]` | `weight_packed` made signed beside `weight_scale`; group/dynamic reorder is refused | | same with `symmetric:false` (AWQ) | `u4*[1xG]-u4[1xG]` | nibbles plus `weight_zero_point`, bit for bit | | `auto-round` `auto_round:auto_gptq`, `gptq` v1 | `i4*bf16[1x128]` or `i8*bf16[1x128]` | scales rounded to bf16; convert-time; `import` keeps shipped f16 | | `nvfp4-pack-quantized` | `fp4_e2m1*f32[1x16]` | `weight_scale / weight_global_scale`; **no kernel serves NVFP4**, so it always takes a rule | | `modelopt` NVFP4 / FP8 / MIXED_PRECISION | NVFP4 `fp4_e2m1*f32[1x16]`; FP8 `fp8_e4m3*f32[1x*]` | ModelOpt's second scale **multiplies**; `input_scale` is not read | | compressed-tensors all-FP8 groups | `fp8_e4m3*[1x*]` or `[RxC]` | nothing imported: stock FP8 with its scale; a per-tensor scale is written once a row | | GGUF `Q8_0`, `Q4_K`, `Q6_K` | `i8*f16[1x32]`, affine u4/u6…, `i6*i8…` | blocks split bit for bit; **not servable as they stand** | The width is read **per tensor**, so a mixed release imports int4 experts and int8 projections together. What an import cannot read (an asymmetric GPTQ zero point, a group-reordering act-order index, a GPTQ group under 128, a missing/zero global scale, an unlisted format) is refused by name. The release's `quantization_config` names the tool, not a served form, so it is kept as `imported_quantization_config.*`. `import` options: `codes=uN` makes a symmetric grid unsigned under a zero point of `2^(N-1)`; `zero=` the zero plane's dtype (`u4`, `u8`); `scale=` f16/bf16/f32, each shipped scale rounded to nearest; `block=RxC` repeats an FP8 scale shipped per row/tensor over that grid. It reads an imported integer grid or FP8 with a per-row/per-tensor scale and keeps a plain float weight as it is. **NVFP4 to MXFP4 rounds twice** (e4m3 per 16 → power of two per 32), so it loses more than MXFP4 made from bf16. Prefer a calibrated MXFP4 container when one exists; the quality verdict is a KL gate, not a file size. ## The descriptor law (`--set`) The architecture plugin is chosen by the exact `(arch id, quant descriptor)` pair. A convert-time choice that must also hold at serve time is stated once with `--set`, which hands it to declare and writes it into the container. Storage format is not such a choice (the encoding is in the container and the plugin reads it there), but the **serving plugin** is: a `quantization_config.*` key re-derives the descriptor. ```sh rad-convert -o q38-27b-fp8.rad \ --recipe data/recipes/q38-27b-fp8block-df2.recipe \ --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3 \ --set "quantization_config.weight_block_size=128 128" \ --set quantization_config.activation_scheme=dynamic ``` A bf16 checkpoint whose recipe writes Qwen's block FP8 must therefore be declared *as that format*; otherwise the selection pair does not match any plugin and the convert (or the boot) is refused. The same law bites the other way when a release's own `quant_method` names a pair no plugin claims. ## Inspect and validate the result ```sh rad-info out.rad # architecture, source quant, recipe, sections rad-info --recipe out.rad # the expanded rules — did the intended one win? rad-info -v out.rad | head -40 # every tensor, its encoding, and what made it rad-kbench -m out.rad --home $RADIANCE_HOME --report kbench.md radiance --model out.rad --debug-graph # every op resolves ``` The converter's own error report (the worst `--report N` weights) measures each quantised weight against its source through the core decoder. `rad-kbench` adds the artifact test: weight operands read from the container and passed through each kernel's `layout`/`relayout` hooks. Both are on [[Radiance Tools]]. ## Calibration (GPTQ, AWQ, AutoRound, ParoQuant) `calib=DIR` is a directory an engine run wrote with `RADIANCE_CALIB_DIR` set: each layer's **Gram matrices of its input**, in the domain the kernel multiplies it in. A block that characterises an activation declares an f32 `[K,K]` PERSIST buffer and a `gram_accum` op; at `RADIANCE_CALIB_TOKENS` (default 65536) the run writes the files. Three-step bootstrap: convert with plain `rtn` → serve that container with `RADIANCE_CALIB_DIR` → push real prompts through prefill → convert again with the Hessians. The second moment is robust to the first container's small quantisation error. A weight with no file is quantised uncalibrated and the converter says so; `awq`, `autoround` and `paroquant` fall back to the imatrix diagonal when one is given. - The accumulator must be **PERSIST**: a transient buffer may share bytes and be overwritten between the launches it sums. - Files are per rank; the consumer sums them. Under expert parallelism a rank's `down` input covers only its experts' tokens; `gate_up` input is the full token set on every rank, so the sum scales `H` by world size — GPTQ is invariant to a positive scale. - Under tensor parallelism a column-split linear sees only its rank's columns, so a tap may write `gram.r..part.bin`; libquant places the parts on the diagonal in rank order, leaving cross-rank products zero. They must cover K and be in the model's own domain. - `--imatrix` supplies the expert-popularity profile the placement planner ranks units by, the per-column importances a quantiser uses, and the error report's weighting. ## Reuse, drafters, merging ```sh # re-quantise only what changed rad-convert $SRC -o new.rad --recipe new.recipe --reuse old.rad --changed changed.txt # merge a drafter shipped as its own repo (DFlash2, DSpark, MTP assistant) rad-convert $SRC -o out.rad --draft-model /srv/models/z-lab/Qwen3.8-27B-DFlash2 \ --recipe data/recipes/q38-27b-mxfp4-df2.recipe # merge a PEFT adapter at convert time rad-convert $SRC -o tuned.rad --lora ~/run/adapter --lora-strength 1.0 \ --recipe data/recipes/q38-27b-mxfp4-df2.recipe ``` - A drafter's tensors go in under `draft.`, its config under `draft.` keys. It has no embedding and no `lm_head` of its own; it uses the target's. - `--lora-map` exists for adapters trained on names the checkpoint does not use; a fused target (`qkv`) takes several targets and splits the adapter's rows in order. - `--add` + `--config` + `--tokenizer2` assemble a ComfyUI-style release of separate single files (Anima: diffusion model, text encoder, VAE). A name in two files is refused. ## `rad-export` — finetunes without hand-writing the command ```sh rad-export ~/outputs/my-run/checkpoint-400 --dry-run # what it would run rad-export ~/outputs/my-run/checkpoint-400 # adapter -> .rad rad-export ~/exports/merged -o ~/models/mine.rad --recipe q38-27b-fp8block-df2 ``` | flag | default | meaning | |---|---|---| | `SRC` | required | a merged 16-bit checkpoint (`config.json` + `*.safetensors`) or a PEFT adapter (`adapter_config.json` + `adapter_model.safetensors`) | | `-o, --out FILE` | SRC's folder name + `.rad`, beside SRC or in `out_dir` | the container | | `--recipe NAME\|FILE\|auto` | `auto` | a recipe name, a file, or the one recipe whose header names the base model, takes a bf16 checkpoint and needs no calibration | | `--base DIR` | the adapter's `base_model_name_or_path`, from disk or the HF cache, else downloaded | the base a LoRA merges into | | `--draft DIR\|ORG/NAME` | the config, else the Hub repo the recipe's usage line names | the drafter when the recipe merges one; a usage line with a placeholder asks for `--draft` | | `--image IMAGE` | `$RADIANCE_IMAGE`, the config's, then `stilldeadcode/radiance` | run `rad-convert` in this image (docker, CPU-only, no network, **not pulled**) | | `--rad-convert PATH` | — | run this binary instead of docker; recipes come from its `share/radiance` | | `--memory M` | — | docker memory limit for the convert | | `--dry-run` | — | resolve recipe, drafter and paths; print the `rad-convert` line; write nothing | | `--keep-merged` | — | keep the merged checkpoint beside OUT | | `--config FILE` | `~/.config/rad-export/config.json` | the config below | | `--list-recipes [--json]` | — | list the recipes the image/installed tool carries | - Input: a merged 16-bit checkpoint (`config.json` + `*.safetensors`) or a PEFT LoRA adapter (`adapter_config.json` + `adapter_model.safetensors`). - An adapter is merged **exactly as PEFT's `merge_and_unload`** computes it on CPU: `scale` = alpha/r (alpha/√r with rsLoRA), `rank_pattern`/`alpha_pattern` honoured, delta in fp32 rounded to the adapter's dtype. It streams tensor by tensor into 1 GiB shards, so a 27B merge needs a few GiB of RAM, not the model twice. Plain linear LoRA only: embedding LoRA, trained biases, DoRA, saved modules, convolution LoRA, a rank other than the config's, and a missing target are refused before anything is written. The adapter's tokeniser/template replace the base's where shipped. - The recipe comes from the image (`/opt/radiance/share/radiance/recipes`) or the installed `rad-convert`'s share; `auto` matches the base model's name in recipe headers and asks when several fit. A recipe needing calibration is listed but never picked by `auto` — a finetune's Hessians are not the stock model's. - It runs `rad-convert` on the CPU, under Docker as the caller with no network, and writes the log to `OUT.log`. - `~/.config/rad-export/config.json` pins what auto cannot know: `image`, `out_dir`, `recipes{model:recipe}`, `drafts{recipe:dir}`, `bases{repo:dir}`. - It is Python 3 with no dependencies; merging needs `torch`/`safetensors`, fetching a missing base or drafter needs `huggingface_hub`. The image carries no Python; extract the script with `docker run --rm --entrypoint busybox stilldeadcode/radiance cat /opt/radiance/bin/rad-export`. ## Scoring a container (the KL mode) ```sh # once, on the bf16 container radiance --model bf16.rad --kld-record /srv/kld/q38 --kld-corpus corpus.jsonl # each candidate, same flags otherwise radiance --model quant.rad --kld-ref /srv/kld/q38 --kld-out report.json ``` - The corpus is JSONL, one `{"prompt", "score_from"?, "source"?}` a line; only `prompt` is required. Positions `[score_from, n-1)` are scored; `source` groups the report. Keep the scoring corpus disjoint from the calibration corpus. - `--kld-record` writes the reference's log-probabilities (f16 over the vocabulary, the 16 most likely tokens exactly, the token ids). DIR is never overwritten and is read only once `kld.json` exists, which is written last. - `--kld-ref` scores this model's f32 logits against stored rows: mean, median, 99/99.9/99.99th percentile KL, top-1 agreement, and both models' perplexity. `--kld-out` writes the report; `FILE.rows` is `[positions, 4]` f32 (KL, both NLLs, argmax agreement) for position-by-position comparison. - The mode turns speculation and the prefix cache off, turns the step pipeline off, and refuses an explicit spec depth above 0. The logits buffer holds a row per token of a step, so a large `--max-num-batched-tokens` costs pinned host memory as well as VRAM. - `--kld-lora NAME` scores every document through a served adapter, against a reference recorded from the merged checkpoint; `--kld-lora-sources PREFIX` runs only matching documents through it, the rest as the base, in the same steps — each document's rows should be byte-identical to the all-adapter and all-base runs. The gate the fleet uses (mean KL ≤ 0.03, top-1 ≥ 95 %) is on [[frank-archon]] and [[Quantization Production]]; the mode here is only the instrument. ## Convert laws paid for in full - **`--plan-only` first.** A plan that resolves every op will boot; it costs seconds against an hour of I/O on a large checkpoint. - **Declare runs at the step and context bounds, and some graphs depend on them.** Qwen4-Exp sizes its QSA buffers by `--max-ctx` and refuses a `--max-tok`/context pair whose score plane would pass 2 GiB; the defaults (8192 tokens, the checkpoint's training context) fit most models, and a generator's largest pass may override `--max-tok`. - **Scale names follow the tensor.** `scale_name()` appends `_scale_inv` to the *tensor* name, so a stacked `…experts.gate_up_proj` keeps its scales at `…gate_up_proj_scale_inv`. - **File shadowing: first wins.** The reader opens index-named files sorted and emplaces every tensor each file physically holds, first-wins. A merged PLE shard must sort first (`00-ple.merged.safetensors`) or a raw shard wins the name. - **Never write through a staging symlink.** A checkpoint may symlink its raw index; `open(path,"w")` writes through it and corrupts the checkpoint. Write a temp file and `os.replace`. - **One encoding per expert table.** A whole expert kept bf16 is the only thing legal beside a table; a split tier (down=fp8, gate_up=w4) is unservable. - **The descriptor pair has no fallback** (above), and **convert is card-less**: GPTQ solves on CPU BLAS, so the GPU is irrelevant to production and the card only decides what can be decoded. ## Related - [[Radiance Model Families]] — the (arch id, quant descriptor) table and the command per model. - [[Radiance Plugins]] — quantiser and kernel-plugin contracts behind these options. - [[Radiance Tools]] — `rad-info`, `rad-kbench`, `rad-tune`. - [[Quantization Production]] — the fleet's calibration/KLD campaign and its gates. - [[frank-archon]] — the deployment whose recipes and convert laws these are.