Frank
3a04e34414
Nine pages under LLM/ document the upstream radiance engine at 8da043c: - engine shape, plugin model, RAD2 container, ABI and environment - every serve flag grouped with defaults and the law behind each knob - the HTTP surface, request fields and refusal classes - rad-convert, the recipe language, quantisers, imports, reuse, rad-export, KL scoring - the (arch id, quant) family table and the recipe of record per model - kernel/quantiser plugin authoring, encodings, build and checks - architecture plugin authoring: declare, step, probe, lane and media rules - rad-info/radbench/radtune and the shell harnesses - the container contract, host prerequisites, readiness and restart triage Also completes the dangling Related line on the frank-archon endpoint page.
141 lines
8.4 KiB
Markdown
141 lines
8.4 KiB
Markdown
---
|
||
type: "reference"
|
||
title: "Radiance Engine — the inference server behind frank-archon"
|
||
description: "The radiance inference engine: its shape, plugin model, container format, and how the reference pages fit together."
|
||
tags:
|
||
- llm
|
||
- radiance
|
||
- reference
|
||
timestamp: 2026-10-30
|
||
---
|
||
|
||
# Radiance Engine — shape, plugin model, container
|
||
|
||
Radiance is StillDeadcode's inference server for AMD GPUs, written in C++20 and HIP: language
|
||
models, plus image, video, speech and music generation and embeddings. One HTTP front end speaks
|
||
OpenAI's chat, completions, Responses, images, audio and video APIs, Anthropic's Messages API and
|
||
llama-server's native one. Nothing in the serving path is Python.
|
||
|
||
The fleet's only radiance endpoint is [[frank-archon]] (2× R9700, TP=2, Dankserver). **This page
|
||
and its siblings own the engine reference; the live deployment shape, its Eras and its measured
|
||
laws stay on the endpoint page.** The numbers here describe upstream `main` as read 2026-10-30
|
||
(release 1.14.0, plugin ABI 31). The fleet pin is older (1.8.1): a flag below that post-dates the pin
|
||
is a boot refusal, never a silent no-op, and the endpoint page records the CLI families that pin
|
||
itself retired relative to its own rollback target.
|
||
|
||
Source: `https://codeberg.org/StillDeadcode/radiance` at `8da043c`. Engine docs live in `docs/`
|
||
(`GUIDE.md` for serving, `TOOLS.md` for tools, `PLUGIN.md` for plugins, `ARCHITECTURES.md` per
|
||
family, `OPS.md` for the op vocabulary, `spec.md` for the design).
|
||
|
||
## The four moving parts
|
||
|
||
```mermaid
|
||
graph LR
|
||
A[Checkpoint<br/>safetensors / GGUF] -->|rad-convert + recipe| B[.rad container]
|
||
B --> C[radiance engine]
|
||
C --> D[OpenAI / Anthropic /<br/>llama-server wire APIs]
|
||
P1[kernel library .so<br/>libr4d, libavx, libref] --> C
|
||
P2[architecture .so<br/>one per arch id × quant descriptor] --> C
|
||
P3[quantiser .so<br/>libquant] -->|rad-convert| A
|
||
```
|
||
|
||
- **Engine** — `core/`: declare phase, placement/memory planner, scheduler, KV manager, sampler,
|
||
tokeniser, HTTP server, media preprocessing.
|
||
- **Kernel libraries** — implement ops for hardware. `libr4d` targets RDNA4 (`gfx1200`,
|
||
`gfx1201`); `libavx` serves the host backend (AVX-512/AVX2) with no GPU at all; `libref` is the
|
||
slow reference implementation every other library is checked against.
|
||
- **Architecture plugins** — declare one model family at one weight format: its weights, buffers,
|
||
KV groups and the ops it issues each step. The engine picks one by an **exact (architecture id,
|
||
quantisation descriptor) pair, with no fallback**.
|
||
- **Quantiser plugins** — `libquant`'s `rtn`, `gptq`, `awq`, `autoround`, `paroquant`, `m2xfp4`,
|
||
`ggml`, `cast`, `import`; they write a weight's numbers into an encoding's planes.
|
||
- **Tools** — `rad-info`, `rad-convert`, `rad-export`, `rad-kbench`, `rad-tune`, `rad-schemas`,
|
||
`rad-tokcheck`, plus `rad-chatparse`, `rad-avx-check`, `r4d_selftest` in the build tree.
|
||
|
||
## Two phases, one static graph
|
||
|
||
`rad_arch_declare` runs once per rank and names every weight, buffer, KV group and op, in order.
|
||
The core resolves every op against the kernel hierarchy per band, per domain, and completes the
|
||
whole plan before a byte of weights is read. `rad_arch_step` then only *issues* handles obtained at
|
||
declare; it may not allocate and may not synchronise. A model whose declare succeeds is fully known
|
||
statically, which is what makes `--debug-graph`, the buffer liveness plan and `rad-convert`'s
|
||
plan-only check possible.
|
||
|
||
Consequences a plugin author must respect:
|
||
|
||
- An op that never resolves is `RAD_NULL_HANDLE` and only fails if issued. A fusion is a *selection*:
|
||
ask for the fused op, emit the chain when `rad_op_resolved` is false.
|
||
- Declaration order is the buffer planner's liveness. A drafter must be declared after the trunk
|
||
it runs behind, or trunk and drafter buffers silently share bytes.
|
||
- Every op in the graph must have a row in a loaded kernel library. `libref` implements the
|
||
conventional vocabulary in `docs/OPS.md`; device-only ops (most fusions) exist only in `libr4d`.
|
||
|
||
## The `.rad` container (RAD2)
|
||
|
||
A container is one file holding everything the server needs: weights as **canonical encoding
|
||
planes**, the tokeniser (and optionally a second vocab), the chat template, metadata, the recipe it
|
||
was made with, and any draft head. It names no kernel library.
|
||
|
||
- Planes are kernel-independent: codes, scales, zero points, codebooks — row-major, bit-packed
|
||
sub-byte where needed. A kernel rearranges them into its own layout at load (`layout`/`relayout`
|
||
hooks), losslessly, so a kernel library can change layouts without reconverting the model.
|
||
- Encodings are data, not an enum: `RadEncoding` names the scheme (`plain`, `affine`, or a
|
||
quantiser's own), the planes by role, the scale grouping, and a transform (`fwhtN`, `perm`,
|
||
`givens`). That is how one file can hold block-FP8, int4, MXFP4, NVFP4-imported, ParoQuant
|
||
rotations and llama.cpp grids.
|
||
- `rad-info` is the reader: sections (strings, metadata, directory, planes, encodings, vocab,
|
||
profile, data), `--meta`, `--vocab`, `--recipe`, `--plugins`, and `-v` for every tensor with its
|
||
encoding and what made it.
|
||
|
||
See [[Radiance Conversion]] for producing one and [[Radiance Tools]] for inspecting it.
|
||
|
||
## Selection key and version law
|
||
|
||
- A checkpoint's architecture id is its `config.json` `model_type` without underscores; `rad-convert
|
||
--arch ID` overrides it. `rad_arch_quant` supplies the quantisation descriptor (empty for an
|
||
unquantised checkpoint).
|
||
- **No fallback.** A raw official FP8 checkpoint whose `quantization_config.quant_method` is `fp8`
|
||
is not the pair the `qwen4exp_fp8` plugin claims (`(qwen4exp, "")`), so both serve and convert
|
||
refuse it until the descriptor is changed with `--set` or a staged `config.json`. The deployment
|
||
lesson is on [[frank-archon]]; the mechanism is on [[Radiance Conversion]].
|
||
- `RAD_ABI_VERSION` covers every header a plugin compiles against. A plugin reporting a different
|
||
number is refused at load, with both numbers printed. It moves on any shape change, not on a new
|
||
optional export; rebuild when it changes. Release version and ABI are independent: `/version`
|
||
reports the release, `rad-info --plugins` reports what loaded.
|
||
|
||
## Environment
|
||
|
||
| variable | meaning |
|
||
|---|---|
|
||
| `RADIANCE_HOME` | plugin search path: `:`-separated homes, each holding `kernels/`, `architectures/`, `quantizers/`; first wins, same-name files shadow later homes. Compiled-in default is `<prefix>/share/radiance` |
|
||
| `RADIANCE_KERNELS` | default for `--kernels`: ordered kernel libraries, first wins; unnamed libraries are not loaded |
|
||
| `RADIANCE_TUNE` | pin a kernel's tuned axes for a measurement (a measurement aid, not a deployment setting) |
|
||
| `RADIANCE_CALIB_DIR` | a serve run records per-layer input Gram matrices here (the GPTQ calibration tap) |
|
||
| `RADIANCE_CALIB_TOKENS` | tokens after which the tap stops and writes (default 65536) |
|
||
| `RADIANCE_LOG`, `RADIANCE_LOG_STEPS` | log level; per-step token counts |
|
||
| `RADIANCE_DEBUG_RESID` | residual-stream probe per (layer, site) |
|
||
| `RADIANCE_DFLASH_Q_TEMP` | temperature of a DFlash2 coupled walk's candidate softmax |
|
||
|
||
The full list of debug knobs (`RADIANCE_DEBUG_*`, `RADIANCE_DUMP_*`, `RADIANCE_OP_ORACLE`, …) is in
|
||
the source's `getenv` calls; none is a deployment setting.
|
||
|
||
## Where the pages are
|
||
|
||
| Page | Owns |
|
||
|---|---|
|
||
| [[Radiance Serve Options]] | every serve flag, grouped, with defaults and the law behind each knob |
|
||
| [[Radiance HTTP API]] | endpoints, request fields, streaming, tool calls, refusal rules |
|
||
| [[Radiance Conversion]] | `rad-convert`, the recipe language, quantisers, imports, reuse, `rad-export`, KL scoring |
|
||
| [[Radiance Model Families]] | which plugin claims which model, and the conversion command per family |
|
||
| [[Radiance Plugins]] | kernel and quantiser plugins: ABI, rows, hooks, encodings, build and checks |
|
||
| [[Radiance Architecture Plugins]] | declaring a model: weights, buffers, KV groups, ops, step, probe |
|
||
| [[Radiance Tools]] | `rad-info`, `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-tokcheck`, harnesses |
|
||
| [[Radiance Operations]] | the container contract, host prerequisites, readiness, ledgers, restarts |
|
||
| [[frank-archon]] | the fleet's live radiance endpoint: shape, Eras, rollback, measured laws |
|
||
|
||
## Sources
|
||
|
||
- `README.md`, `docs/GUIDE.md` §1–§4, `docs/TOOLS.md`, `docs/PLUGIN.md`, `docs/DOCKER.md`,
|
||
`docs/ARCHITECTURES.md`, `docs/OPS.md`, `spec.md` §2–§4, `abi/*.h`, `core/config.cpp`,
|
||
`core/rad_core.h` at commit `8da043c` (codeberg.org/StillDeadcode/radiance).
|