--- type: "howto" title: "Radiance Operations — running, reading, restarting" description: "Running a radiance server: the container contract, host prerequisites, readiness, ledgers and safe restarts." tags: - llm - radiance - operations - howto timestamp: 2026-10-30 --- # Radiance operations This is the engine-level runbook. The fleet's deployed shape, its unit and its deploy path live on [[frank-archon]]; the flags themselves on [[Radiance Serve Options]]. ## Install ```sh git clone https://codeberg.org/StillDeadcode/radiance.git && cd radiance ./build.sh # configure, build for the cards present, run fast tests ./build.sh --no-hip # host backend only cmake -S . -B build -G Ninja -DRAD_GPU_TARGETS='gfx1201' # cards this machine lacks cmake --install build # after -DCMAKE_INSTALL_PREFIX=/opt/radiance at first configure ``` Needs Linux, git, a C++20 compiler, CMake 3.21+, Ninja; for device kernels ROCm 10.1 or 7.2 (`hipcc`, `ROCM_PATH` if not `/opt/rocm`). - **Name the prefix at the first configure.** The default plugin path `/share/radiance` is compiled in from that configure; `cmake --install build --prefix DIR` alone gives a tree that looks for its plugins elsewhere. - The prefix gets `bin/`, `include/radiance/`, `lib/cmake/radiance/`, `libexec/radiance/`, `share/radiance/` (plugins and recipes) and `share/doc/radiance/`. - With ROCm 10.1 the build turns VOPD pairing off because its LLVM 24 pairs in heap-address order, so two builds of one source can differ. The Docker image's compiler has the fix and keeps pairing (faster fp8 prefill GEMMs); declare such a compiler with `-DRAD_HIP_VOPD_REPEATABLE=ON`. ROCm 7.2 is unaffected. Run without Docker: ```sh export RADIANCE_HOME=$PWD/build/radiance_home # plugins; an install finds its own build/bin/radiance --model /srv/models/rad/minicpm5-2b-fp8.rad --port 8000 build/bin/radiance --model StillDeadcode/minicpm5-2b-fp8 --port 8000 ``` `--model` also takes a repo id: the server runs `scripts/hfget.sh` into `--download-dir`, serves the one `.rad` at the repo's top level, or the directory as a checkpoint when it holds none. A repo with several containers is refused, naming them. Every start checks the hub; a start that cannot reach it serves the copy a complete earlier fetch left, and a transfer that fails part-way stops startup. ## Container The published image is `stilldeadcode/radiance`, kernels built for `gfx1201`. The host needs Docker and an `amdgpu` driver supporting ROCm 10.1 — **not** ROCm itself. ```sh docker run -d --name radiance --network host --init --ulimit memlock=-1 \ --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined \ --group-add "$(getent group render | cut -d: -f3)" \ --group-add "$(getent group video | cut -d: -f3)" \ -v /srv/models/rad:/models:ro \ radiance --model /models/m.rad --tp 2 --host :: --port 8000 ``` Each option is load-bearing: | option | why | |---|---| | `--device /dev/kfd --device /dev/dri`, `render`/`video` groups | the cards | | `--security-opt seccomp=unconfined` | Docker's default profile refuses io_uring (a disk-tier n-gram table's reads) and the NUMA calls the ROCm runtime makes for host memory | | `--init` | an init process as PID 1 forwards SIGTERM; the engine finishes its step and releases the cards | | `--ulimit memlock=-1` | a host-side pinned n-gram copy can be tens of GiB, far past Docker's default | | `--network host` | the server binds the host's addresses directly, IPv4 and IPv6, no NAT | The image is `FROM scratch` around the stripped install tree: busybox as the only shell, plus `curl`, `jq` and a CA bundle for `hfget.sh`; **no Python, no package manager**. Tools (`rad-convert`, `rad-info`, `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-tokcheck`) are on `PATH`; recipes are in `/opt/radiance/share/radiance/recipes/`. Its entrypoint is `radiance`. It sets `GPU_MAX_ALLOC_PERCENT=100` and `GPU_MAX_HEAP_SIZE=100` because the HIP runtime's defaults cap one allocation and the heap below what the placement plan hands out. Build your own with `docker/build.sh` (`-t 'gfx1200;gfx1201'` for targets, `-j` compile jobs, `-E` to skip tests, `-s host` to build remotely). The context is `git archive` of the commit, so the image holds only committed content. The `rootfs` stage fails the build if any ELF cannot be resolved inside a chroot of the tree. Compose: one file per model plus `common.yaml`, copied to the serving host with an `.env` setting the image, models directory (mounted read-only at `/models`), API key, port, user and GPU groups. Every model listens on the same port, and most use both cards, so **run one at a time**. The models directory must be on a local filesystem that does direct I/O (ext4, XFS, btrfs); on btrfs with compression, a container copied into a `chattr +C` directory is stored uncompressed and direct reads work. A plugin of your own is mounted and put first on the search path; nothing in the image changes: ```sh -v /srv/radiance-plugins:/plugins:ro -e RADIANCE_HOME=/plugins:/opt/radiance/share/radiance ``` ## Host prerequisites (upstream's two-R9700 checklist) | item | setting | |---|---| | firmware | Above 4G Decoding on (Resizable BAR needs it); CSM off (pure UEFI) | | kernel command line | `iommu=pt amdgpu.ppfeaturemask=0xfff7ffff amdgpu.ras_enable=0` | | power/clock | OverDrive bit needed for `ppfeaturemask`; apply a power cap only while no process holds `/dev/kfd` | | memlock | `--ulimit memlock=-1` (container) / `LimitMEMLOCK=infinity` (systemd) when host memory is pinned | | two engines, one tmpfs table | `amdgpu.no_system_mem_limit=1`, or the second pin can be refused | | storage | a local fs with O_DIRECT for the container; a tmpfs sized for the table for `--ngram-placement ram=DIR` | Why each: `iommu=pt` keeps DMA identity so the cards can reach each other's memory and pinned host pages; `ppfeaturemask` enables the power controls; `ras_enable=0` turns VRAM ECC off (ECC reserves about 2 GiB a card). Verify peer access with a rate in `lspci`/the card tables — a missing cell means no peer access; check `iommu=pt` and the BAR size. The fleet's own boot-line law for Dankclaw (`iommu=pt`, tracked in the repo) is on [[dankclaw-iommu-translation-off]]; Dankserver's cards are a different box and its posture is on [[Dankserver System Specs]]. ## Readiness: the receipt, then `/health` Startup states every budget **before anything is allocated**, then loads. `/health` (and `/v1/health`, `/ping`) answers only once the load completes, so a bare health gate is honest. While it loads, expect 503 for minutes; do not restart over it. `--live` draws the live view (redirected, one `greppable key=value` line). `/diagnostics` returns the startup report as JSON — **same-machine-only without an API key** — with every option and its source and the hardware read again now. Read the receipt in this order: 1. `elastic` — what the experts and KV cache share after non-expert weights. 2. The split: `experts … / kv cache …`, and `unclaimed` if any. 3. `kv: the pool backs N % of the worst case … M full-length sessions` — the referee for whether the demand signal (`--max-model-len` × `--max-num-seqs`) is funded. 4. `activation arena` and what each size lends; `prefill staging`; `hardware queues` and their VRAM. 5. `host_pool` opened/used, `vram_weights`, the PLE or n-gram placement, the vision tower's placement. 6. `device: the N cards reach each other's memory directly` or the failed pairs; `expert prefetch on/off`; `chunk geometry`; `speculation:`; `chained launches`; `draft trees`. 7. `card N settled at … free` — memory taken after the budget read comes out of headroom; a warning names the `--gpu-headroom-mib` that clears it. A boot refusal prints the reason and exits nonzero; when it happens inside `podman run`, the container stops at once and the unit's restart policy re-runs the same refusal. A unit in `activating (auto-restart)` with a fast nonzero exit is therefore a **preflight/refusal or a boot refusal**, not an engine fault: read the journal line and the argv, not the GPU. ## Lifecycle and restart - SIGTERM finishes the current step and releases the cards. In a container, `--init` forwards it; `podman stop -t N`/`docker stop` is the graceful path. - A systemd unit should `exec podman wait` (or run in the foreground) so the script's lifetime is the container's: a crash is a nonzero exit and the restart policy is real. Set `TimeoutStopSec` above the stop grace, or systemd records `Failed with result timeout` on a healthy stop. - `Restart=always` rather than `on-failure` for a server whose front end can exit cleanly after a worker fault. - Model load is slow (minutes cold). The deployer's LLM wait budget is on [[Deployment Flow]]; a container restart loop during a legitimate load is worse than waiting. ## Two engines on one set of cards - `ROCR_VISIBLE_DEVICES` picks each engine's cards explicitly. - `--weight-prefetch off`: a weight read beside a collective puts a queue switch into every decode step when another server shares the cards. - `--ngram-placement ram=DIR`: two engines on separate card sets hold one table in host memory (`DIR/ngram-.tab` in a tmpfs, `MAP_SHARED`, pinned). The `` hashes the container path, size, mtime and the table's place, so a file never serves another model. The first engine fills it; a second verifies samples and only pins it; the last to leave deletes it. The driver counts a shared page against every process that pins it, so the pinned total is twice the table. - The engine prefers discrete cards (all of the first one's architecture), integrated only when there is nothing else; on integrated graphics device memory is host memory and the budget leaves an eighth of host RAM to the host. ## Logs and diagnostics - `-v` / `-vv`; `RADIANCE_LOG` sets the level without a flag. `RADIANCE_LOG_STEPS=1` prints every step's token count (how a chunk geometry is confirmed in practice). - The engine writes to stderr. Under a container, the journal holds only the container's own output; a rootless `podman logs` covers only the current instance across recreations. - `/metrics` is Prometheus text: engine steps by kind and their seconds, decode/prefill rates, prefill chunks held back, steps held by interleave, draft acceptance and depth, prompt-lookup proposals held, KV bytes and the tokens they address, stream misses and bytes, prefix hits and evictions, preemptions and queue depth. `/stats`, `/graph`, `/sessions` back the dashboard. `scripts/statsdump.py` records `/stats` to JSONL with time and gap, for a machine with no scraper. - `RADIANCE_PROFILE_EVERY`, `RADIANCE_DUMP_BUF`/`RADIANCE_DUMP_DIR`, `RADIANCE_OP_ORACLE`(+) and the `RADIANCE_DEBUG_*` family are debug instruments; they are not deployment settings. ## Verifying a deploy (fleet law) `deploy.sh` green is not proof of what is running. A leftover systemd drop-in or a stale container identity can serve a different model at a different shape while `/health` is 200. After deploying any recently-benched service: 1. `systemctl show -p DropInPaths` — must be empty. 2. `tr '\0' ' ' /cmdline` by any local user of that host. On a single-tenant inference box the network boundary is the real control; do not run one where untrusted local users share the host. - `/diagnostics` is same-machine-only without a key, because it prints the whole configuration. - `--security-opt seccomp=unconfined` is a genuine widening, required by io_uring and the ROCm runtime's NUMA calls; it belongs on a dedicated model host, not a general-purpose one. - Mount model directories read-only; a container with `/models` writable adds nothing a serve needs. - A control vector can remove refusal behaviour. Loading one is a safety decision, not a performance one; the flag is `--control-vector`, and `--control-vector disabled` is the default. ## Failure signatures at startup | what you see | what it is | |---|---| | unknown/retired flag, or a flag refused by name | a boot refusal — the pin does not have it; change the trio, not the invocation | | "no plugin claims `` at ``; served at: …" | the (id, descriptor) pair has no plugin; fix the descriptor with `--set`/convert | | "reports ABI N, engine M" | rebuild the plugin against this engine | | budget refusal / `host_pool` short | the shape does not fit this card/host; lower demand or pin more host RAM | | `--ngram-placement ram` refuses | memlock short; raise it or use `auto`/`disk` | | `--kv-cache-dtype fp8_v4`/`k6_v4` refused, plugin named | no kernels for this geometry | | `--p2p on` refuses, pairs named | the driver has no peer access; use `auto`/`off` | | first launch of a never-run kernel faults | too little `--gpu-headroom-mib` for its code object | | unit loop, no container, no engine log | a preflight refusal (TTM/memlock/digest/model) re-run by `Restart=always` | ## Related - [[Radiance Serve Options]] — every knob and its law. - [[Radiance HTTP API]] — readiness and the wire. - [[Radiance Tools]] — how to corroborate a receipt. - [[frank-archon]] — the fleet's radiance unit, guard chain and Eras. - [[Model Swap and Rollback]] — the swap discipline this page's verification serves. - [[Deployment Flow]] — how a committed change reaches the host.