serverAppConfigs/serverAppConfigs-docs/LLM/Radiance Operations.md

---
type: "howto"
title: "Radiance Operations — running, reading, restarting"
description: "Running a radiance server: the container contract, host prerequisites, readiness, ledgers and safe restarts."
tags:
  - llm
  - radiance
  - operations
  - howto
timestamp: 2026-10-30
---

# Radiance operations

This is the engine-level runbook. The fleet's deployed shape, its unit and its deploy path live on
[[frank-archon]]; the flags themselves on [[Radiance Serve Options]].

## Install

```sh
git clone https://codeberg.org/StillDeadcode/radiance.git && cd radiance
./build.sh                                   # configure, build for the cards present, run fast tests
./build.sh --no-hip                          # host backend only
cmake -S . -B build -G Ninja -DRAD_GPU_TARGETS='gfx1201'   # cards this machine lacks
cmake --install build                        # after -DCMAKE_INSTALL_PREFIX=/opt/radiance at first configure
```

Needs Linux, git, a C++20 compiler, CMake 3.21+, Ninja; for device kernels ROCm 10.1 or 7.2
(`hipcc`, `ROCM_PATH` if not `/opt/rocm`).

- **Name the prefix at the first configure.** The default plugin path `<prefix>/share/radiance` is
  compiled in from that configure; `cmake --install build --prefix DIR` alone gives a tree that looks
  for its plugins elsewhere.
- The prefix gets `bin/`, `include/radiance/`, `lib/cmake/radiance/`, `libexec/radiance/`,
  `share/radiance/` (plugins and recipes) and `share/doc/radiance/`.
- With ROCm 10.1 the build turns VOPD pairing off because its LLVM 24 pairs in heap-address order, so
  two builds of one source can differ. The Docker image's compiler has the fix and keeps pairing
  (faster fp8 prefill GEMMs); declare such a compiler with `-DRAD_HIP_VOPD_REPEATABLE=ON`. ROCm 7.2 is
  unaffected.

Run without Docker:

```sh
export RADIANCE_HOME=$PWD/build/radiance_home   # plugins; an install finds its own
build/bin/radiance --model /srv/models/rad/minicpm5-2b-fp8.rad --port 8000 <flags>
build/bin/radiance --model StillDeadcode/minicpm5-2b-fp8 --port 8000 <flags>
```

`--model` also takes a repo id: the server runs `scripts/hfget.sh` into `--download-dir`, serves the
one `.rad` at the repo's top level, or the directory as a checkpoint when it holds none. A repo with
several containers is refused, naming them. Every start checks the hub; a start that cannot reach it
serves the copy a complete earlier fetch left, and a transfer that fails part-way stops startup.

## Container

The published image is `stilldeadcode/radiance`, kernels built for `gfx1201`. The host needs Docker
and an `amdgpu` driver supporting ROCm 10.1 — **not** ROCm itself.

```sh
docker run -d --name radiance --network host --init --ulimit memlock=-1 \
    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined \
    --group-add "$(getent group render | cut -d: -f3)" \
    --group-add "$(getent group video | cut -d: -f3)" \
    -v /srv/models/rad:/models:ro \
    radiance --model /models/m.rad --tp 2 --host :: --port 8000
```

Each option is load-bearing:

| option | why |
|---|---|
| `--device /dev/kfd --device /dev/dri`, `render`/`video` groups | the cards |
| `--security-opt seccomp=unconfined` | Docker's default profile refuses io_uring (a disk-tier n-gram table's reads) and the NUMA calls the ROCm runtime makes for host memory |
| `--init` | an init process as PID 1 forwards SIGTERM; the engine finishes its step and releases the cards |
| `--ulimit memlock=-1` | a host-side pinned n-gram copy can be tens of GiB, far past Docker's default |
| `--network host` | the server binds the host's addresses directly, IPv4 and IPv6, no NAT |

The image is `FROM scratch` around the stripped install tree: busybox as the only shell, plus `curl`,
`jq` and a CA bundle for `hfget.sh`; **no Python, no package manager**. Tools (`rad-convert`,
`rad-info`, `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-tokcheck`) are on `PATH`; recipes are in
`/opt/radiance/share/radiance/recipes/`. Its entrypoint is `radiance`. It sets
`GPU_MAX_ALLOC_PERCENT=100` and `GPU_MAX_HEAP_SIZE=100` because the HIP runtime's defaults cap one
allocation and the heap below what the placement plan hands out.

Build your own with `docker/build.sh` (`-t 'gfx1200;gfx1201'` for targets, `-j` compile jobs, `-E`
to skip tests, `-s host` to build remotely). The context is `git archive` of the commit, so the image
holds only committed content. The `rootfs` stage fails the build if any ELF cannot be resolved inside
a chroot of the tree.

Compose: one file per model plus `common.yaml`, copied to the serving host with an `.env` setting the
image, models directory (mounted read-only at `/models`), API key, port, user and GPU groups. Every
model listens on the same port, and most use both cards, so **run one at a time**. The models
directory must be on a local filesystem that does direct I/O (ext4, XFS, btrfs); on btrfs with
compression, a container copied into a `chattr +C` directory is stored uncompressed and direct reads
work.

A plugin of your own is mounted and put first on the search path; nothing in the image changes:

```sh
-v /srv/radiance-plugins:/plugins:ro -e RADIANCE_HOME=/plugins:/opt/radiance/share/radiance
```

## Host prerequisites (upstream's two-R9700 checklist)

| item | setting |
|---|---|
| firmware | Above 4G Decoding on (Resizable BAR needs it); CSM off (pure UEFI) |
| kernel command line | `iommu=pt amdgpu.ppfeaturemask=0xfff7ffff amdgpu.ras_enable=0` |
| power/clock | OverDrive bit needed for `ppfeaturemask`; apply a power cap only while no process holds `/dev/kfd` |
| memlock | `--ulimit memlock=-1` (container) / `LimitMEMLOCK=infinity` (systemd) when host memory is pinned |
| two engines, one tmpfs table | `amdgpu.no_system_mem_limit=1`, or the second pin can be refused |
| storage | a local fs with O_DIRECT for the container; a tmpfs sized for the table for `--ngram-placement ram=DIR` |

Why each: `iommu=pt` keeps DMA identity so the cards can reach each other's memory and pinned host
pages; `ppfeaturemask` enables the power controls; `ras_enable=0` turns VRAM ECC off (ECC reserves
about 2 GiB a card). Verify peer access with a rate in `lspci`/the card tables — a missing cell means
no peer access; check `iommu=pt` and the BAR size. The fleet's own boot-line law for Dankclaw
(`iommu=pt`, tracked in the repo) is on [[dankclaw-iommu-translation-off]]; Dankserver's cards are a
different box and its posture is on [[Dankserver System Specs]].

## Readiness: the receipt, then `/health`

Startup states every budget **before anything is allocated**, then loads. `/health` (and
`/v1/health`, `/ping`) answers only once the load completes, so a bare health gate is honest. While it
loads, expect 503 for minutes; do not restart over it. `--live` draws the live view (redirected, one
`greppable key=value` line). `/diagnostics` returns the startup report as JSON — **same-machine-only
without an API key** — with every option and its source and the hardware read again now.

Read the receipt in this order:

1. `elastic` — what the experts and KV cache share after non-expert weights.
2. The split: `experts … / kv cache …`, and `unclaimed` if any.
3. `kv: the pool backs N % of the worst case … M full-length sessions` — the referee for whether the
   demand signal (`--max-model-len` × `--max-num-seqs`) is funded.
4. `activation arena` and what each size lends; `prefill staging`; `hardware queues` and their VRAM.
5. `host_pool` opened/used, `vram_weights`, the PLE or n-gram placement, the vision tower's placement.
6. `device: the N cards reach each other's memory directly` or the failed pairs;
   `expert prefetch on/off`; `chunk geometry`; `speculation:`; `chained launches`; `draft trees`.
7. `card N settled at … free` — memory taken after the budget read comes out of headroom; a warning
   names the `--gpu-headroom-mib` that clears it.

A boot refusal prints the reason and exits nonzero; when it happens inside `podman run`, the
container stops at once and the unit's restart policy re-runs the same refusal. A unit in
`activating (auto-restart)` with a fast nonzero exit is therefore a **preflight/refusal or a boot
refusal**, not an engine fault: read the journal line and the argv, not the GPU.

## Lifecycle and restart

- SIGTERM finishes the current step and releases the cards. In a container, `--init` forwards it;
  `podman stop -t N`/`docker stop` is the graceful path.
- A systemd unit should `exec podman wait` (or run in the foreground) so the script's lifetime is the
  container's: a crash is a nonzero exit and the restart policy is real. Set `TimeoutStopSec` above
  the stop grace, or systemd records `Failed with result timeout` on a healthy stop.
- `Restart=always` rather than `on-failure` for a server whose front end can exit cleanly after a
  worker fault.
- Model load is slow (minutes cold). The deployer's LLM wait budget is on [[Deployment Flow]]; a
  container restart loop during a legitimate load is worse than waiting.

## Two engines on one set of cards

- `ROCR_VISIBLE_DEVICES` picks each engine's cards explicitly.
- `--weight-prefetch off`: a weight read beside a collective puts a queue switch into every decode
  step when another server shares the cards.
- `--ngram-placement ram=DIR`: two engines on separate card sets hold one table in host memory
  (`DIR/ngram-<id>.tab` in a tmpfs, `MAP_SHARED`, pinned). The `<id>` hashes the container path, size,
  mtime and the table's place, so a file never serves another model. The first engine fills it; a
  second verifies samples and only pins it; the last to leave deletes it. The driver counts a shared
  page against every process that pins it, so the pinned total is twice the table.
- The engine prefers discrete cards (all of the first one's architecture), integrated only when
  there is nothing else; on integrated graphics device memory is host memory and the budget leaves an
  eighth of host RAM to the host.

## Logs and diagnostics

- `-v` / `-vv`; `RADIANCE_LOG` sets the level without a flag. `RADIANCE_LOG_STEPS=1` prints every
  step's token count (how a chunk geometry is confirmed in practice).
- The engine writes to stderr. Under a container, the journal holds only the container's own output;
  a rootless `podman logs` covers only the current instance across recreations.
- `/metrics` is Prometheus text: engine steps by kind and their seconds, decode/prefill rates,
  prefill chunks held back, steps held by interleave, draft acceptance and depth, prompt-lookup
  proposals held, KV bytes and the tokens they address, stream misses and bytes, prefix hits and
  evictions, preemptions and queue depth. `/stats`,
  `/graph`, `/sessions` back the dashboard. `scripts/statsdump.py` records `/stats` to JSONL with
  time and gap, for a machine with no scraper (its key comes from `--api-key` or
  `RADIANCE_API_KEY`).
- `RADIANCE_PROFILE_EVERY`, `RADIANCE_DUMP_BUF`/`RADIANCE_DUMP_DIR`, `RADIANCE_OP_ORACLE`(+) and the
  `RADIANCE_DEBUG_*` family are debug instruments; they are not deployment settings.

## Verifying a deploy (fleet law)

`deploy.sh` green is not proof of what is running. A leftover systemd drop-in or a stale container
identity can serve a different model at a different shape while `/health` is 200. After deploying
any recently-benched service:

1. `systemctl show <svc> -p DropInPaths` — must be empty.
2. `tr '\0' ' ' </proc/$(pidof radiance)/cmdline` (or the container's `Config.Cmd`) — the argv you
   meant.
3. The boot ledger — the shape you meant.
4. A behavioural probe — `/v1/models` for the served id, one chat turn, one tool call, and a vision
   probe with a solid colour and an exact expected word.

`/v1/models` is an echo on some runtimes; on radiance it is authoritative for the alias, but the
argv and the ledger are what prove the payload and shape. The three-pin revert discipline for a
payload swap is on [[Model Swap and Rollback]]; the live deployment is [[frank-archon]].

## Security posture

- `--api-key` requires a bearer token or `X-Api-Key` on every route except `/health`, `/v1/health`,
  `/ping` and the dashboard; the key is a command-line argument, so it is readable in
  `/proc/<pid>/cmdline` by any local user of that host. On a single-tenant inference box the network
  boundary is the real control; do not run one where untrusted local users share the host.
- `/diagnostics` is same-machine-only without a key, because it prints the whole configuration.
- `--security-opt seccomp=unconfined` is a genuine widening, required by io_uring and the ROCm
  runtime's NUMA calls; it belongs on a dedicated model host, not a general-purpose one.
- Mount model directories read-only; a container with `/models` writable adds nothing a serve needs.
- A control vector can remove refusal behaviour. Loading one is a safety decision, not a
  performance one; the flag is `--control-vector`, and `--control-vector disabled` is the default.

## Failure signatures at startup

| what you see | what it is |
|---|---|
| unknown/retired flag, or a flag refused by name | a boot refusal — the pin does not have it; change the trio, not the invocation |
| "no plugin claims `<arch>` at `<quant>`; served at: …" | the (id, descriptor) pair has no plugin; fix the descriptor with `--set`/convert |
| "reports ABI N, engine M" | rebuild the plugin against this engine |
| budget refusal / `host_pool` short | the shape does not fit this card/host; lower demand or pin more host RAM |
| `--ngram-placement ram` refuses | memlock short; raise it or use `auto`/`disk` |
| `--kv-cache-dtype fp8_v4`/`k6_v4` refused, plugin named | no kernels for this geometry |
| `--p2p on` refuses, pairs named | the driver has no peer access; use `auto`/`off` |
| first launch of a never-run kernel faults | too little `--gpu-headroom-mib` for its code object |
| unit loop, no container, no engine log | a preflight refusal (TTM/memlock/digest/model) re-run by `Restart=always` |

## Related

- [[Radiance Serve Options]] — every knob and its law.
- [[Radiance HTTP API]] — readiness and the wire.
- [[Radiance Tools]] — how to corroborate a receipt.
- [[frank-archon]] — the fleet's radiance unit, guard chain and Eras.
- [[Model Swap and Rollback]] — the swap discipline this page's verification serves.
- [[Deployment Flow]] — how a committed change reaches the host.