serverAppConfigs/serverAppConfigs-docs/LLM/Radiance HTTP API.md

---
type: "reference"
title: "Radiance HTTP API — endpoints, fields, refusals"
description: "The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules."
tags:
  - llm
  - radiance
  - api
  - reference
timestamp: 2026-10-30
---

# Radiance HTTP API

One server serves one model. It answers OpenAI's chat/completions/Responses/images/audio/video APIs,
Anthropic's Messages API and llama-server's native one, so their clients and SDKs work unchanged.
`model` in a request is accepted and **not enforced**.

## Endpoints

| route | what it serves |
|---|---|
| `POST /v1/chat/completions` | chat: tools, reasoning, streaming, grammars |
| `POST /v1/completions` | raw text completion |
| `POST /v1/embeddings` | an embedding head, else 501 |
| `POST /v1/images/generations`, `/v1/images/edits` | OpenAI images API; needs an image model, else 501 |
| `POST /v1/systemone` | LiquidAI d1 decisions: named yes/no, choice and score questions as probabilities; needs a d1 container, else 501 |
| `POST /v1/audio/transcriptions` | OpenAI speech-to-text; needs a transcribing plugin, else 501 |
| `GET /v1/realtime` (WebSocket) | streaming transcription session |
| `POST /v1/audio/speech` | speech (Breeze-TTS-2, drama-3) or a song (MiniMax-Music3), whole or streamed |
| `GET /v1/audio/speech/stream` (WebSocket) | text streamed in, each sentence's audio out |
| `POST /v1/videos`; `GET /v1/videos[/:id[/content]]`; `DELETE /v1/videos/:id` | video jobs made in the background, downloaded as MP4 |
| `POST /v1/messages`, `/v1/messages/count_tokens` | Anthropic Messages, for Claude Code and the Anthropic SDKs |
| `POST /v1/responses`, `/v1/responses/input_tokens` | OpenAI Responses, for Codex and SDK clients |
| `POST /completion`, `/completions`, `/infill` | llama-server native completion and fill-in-the-middle |
| `GET /v1/models`, `/v1/models/:id` | what is loaded, with `max_model_len`, `root`, `capabilities` |
| `GET /health`, `/v1/health`, `/ping`, `POST /ping` | liveness |
| `GET /load`, `/version`, `/props`, `/server_info`, `/diagnostics` | load; release version; llama-server props; full configuration; the startup report as JSON (`/diagnostics` is same-machine-only without an API key) |
| `POST /apply-template`, `/v1/chat/completions/input_tokens` | the rendered prompt; its token count |
| `GET /metrics` | Prometheus |
| `POST /tokenize`, `/detokenize`; `GET /get_tokenizer_info` | vLLM's shape and llama-server's at once |
| `POST /reset_prefix_cache`; `DELETE /sessions/<label>`, `/sessions?id=<id>` | drop every cached prefix; forget a stored conversation |
| `GET /`, `/dashboard`; `GET /stats`, `/graph`, `/sessions` | the dashboard and the JSON it is drawn from |

Unversioned shorthands exist for chat, chat input tokens, embeddings, images, audio transcriptions,
audio speech, models, responses and completions, for a client given the bare host as base URL.
`/completions` **without** `/v1` is llama-server's native completion. A path nothing serves is a JSON
404.

A text prompt to `/v1/completions`, `/v1/embeddings` or `/tokenize` reads control tokens as tokens
(as vLLM and llama.cpp do), so a client rendering its own template can send `<|im_start|>user\n…`.
The model's declared BOS is prepended unless the prompt already opens with it.

## Chat

```sh
curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":200,"temperature":0.7}'
```

- **No chat template → 501** on this route (the container carries one; `--override-chat-template`
  replaces it). Use `/v1/completions` instead.
- Thinking off has three spellings: `chat_template_kwargs.enable_thinking=false`,
  top-level `enable_thinking=false`, or `reasoning_effort:"none"`.
- Reasoning returns in `choices[].message.reasoning_content`, separate from `content`.
  **Reasoning counts against `max_tokens`**; a reply that runs out inside it has empty `content` and
  `finish_reason:"length"`. `usage.completion_tokens_details.reasoning_tokens` tells that case from a
  model that answered nothing.
- `thinking_token_budget` (aliases `thinking_budget`, `thinking_budget_tokens`,
  `reasoning_budget_tokens`) closes the reasoning block at N tokens and forces the close, so it holds
  under speculation and at any temperature. It needs a one-token end marker; otherwise the response
  has no `completion_tokens_details` and a budget is a 400.
- `continue_final_message` continues a conversation ending in an assistant message instead of
  answering it; `add_generation_prompt` opens a new turn.
- Tool calls: `tools`, `tool_choice` (`auto`, `none`, `required`, named function),
  `parallel_tool_calls`. `required`/named hold the whole reply to calls after the reasoning block and
  cannot combine with `response_format` or `grammar`.
- Structured output: `response_format` (`json_object` or `json_schema`), llama-server's `json_schema`
  shorthand, or a raw GBNF `grammar`. A chat template that drops the grammar makes the request a
  refusal.

## Streaming

```sh
curl -N localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","messages":[{"role":"user","content":"Count to ten."}],
       "max_tokens":100,"stream":true,"stream_options":{"include_usage":true}}'
```

Server-sent events in OpenAI's delta format, terminated by `data: [DONE]`. **Nothing is sent while
the prompt is read**: the first event (the `role: assistant` chunk) goes out with the first generated
token, so a client timing its first event measures time-to-first-token. `return_progress` adds
llama-server's `prompt_progress` chunks during prefill; `timings_per_token` adds `timings` to every
chunk.

## Completions, embeddings, images, audio, video, tokens

- **Completions** — `/v1/completions` takes `prompt` (string or token ids), `echo`, `logprobs`,
  `preserved_tokens`. `/completion` is llama-server's native route; `/infill` is fill-in-the-middle.
- **Embeddings** — `input` is text, a batch, or token ids; `encoding_format` `float`/`base64`;
  `dimensions` truncates to the first N and renormalises; `prompt_name` prepends the model's named
  prompt. Pooling is the model's; the engine does not renormalise.
- **Images** — `/v1/images/generations` returns `b64_json`; `size` defaults to 2048×2048. Edits take
  one picture or up to ten (`image[]`), multipart as OpenAI's SDKs send them, with an optional mask
  (repaint only where transparent). `partial_images` streams previews as SSE.
- **Speech to text** — `/v1/audio/transcriptions`; `/v1/realtime` streams audio in and words back as
  they are spoken. The model's prompt shape comes from its plugin's audio format, not from the server.
- **Speech and music** — `/v1/audio/speech` (whole or streamed) with `instructions` describing a voice
  (Breeze-TTS-2) or speakers written in the input (`<|speaker:1|>`, `[laughs]` for drama-3); a music
  model takes the lyrics as `input` and the description as `instructions`. `/v1/audio/speech/stream`
  streams text in and audio out as each sentence completes.
- **Video** — `/v1/videos` is a job: a video with its stereo soundtrack from a prompt, 5–14 s, up to
  1344×768, made in the background; `GET /v1/videos/:id/content` downloads the MP4.
- **Tokenising** — `POST /tokenize` accepts vLLM's shape (`prompt` or `messages` in, `prompt` out) and
  llama-server's (`content` in/out, `with_pieces`). `POST /detokenize` reverses it.
  `GET /get_tokenizer_info` reports the tokeniser configuration.

## Request fields

Accepted fields, in OpenAI's, vLLM's and llama-server's spellings:

| field | notes |
|---|---|
| `model` | accepted, not enforced |
| `messages` / `prompt` / `input` | chat / completion / embedding |
| `max_tokens`, `max_completion_tokens`, `n_predict` | default is what the context leaves; `-1` is llama-server's "no limit"; clamped to context and to `--max-tokens-cap` |
| `temperature`, `top_p`, `top_k`, `min_p`, `typical_p` | sampler |
| `presence_penalty`, `frequency_penalty`, `repetition_penalty`/`repeat_penalty`, `repeat_last_n` | penalties |
| `xtc_probability`, `xtc_threshold`, `dry_*` | XTC and DRY |
| `seed` | absent means "pick one", not 0 |
| `n` | 1..`--max-n`; prefix cache makes the fan-out nearly free after the first |
| `stop` | string or array, bounded by `--max-stop-strings`/`--max-stop-bytes` |
| `priority` | scheduler hint |
| `ignore_eos` | generate to `max_tokens` whatever the model emits |
| `stream`, `stream_options.include_usage`, `timings_per_token`, `return_progress` | streaming controls |
| `samplers` | llama-server's order; this engine's chain is penalties, DRY, top-k, top-p, min-p, typical, XTC, with temperature first or last; any other list is refused by name |
| `session_id` | labels the conversation on `/sessions`, dashboard and `DELETE` |
| `control_vector` | `true` applies the deployment's vector to this request; absent takes the deployment default; a deployment with no vector refuses `true` |
| `logprobs`, `top_logprobs`, `n_probs` | up to 20 alternatives; not with `echo` |
| `tools`, `tool_choice`, `parallel_tool_calls`, legacy `functions`/`function_call` | chat only |
| `add_generation_prompt`, `chat_template_kwargs`, `continue_final_message` | chat only |
| `reasoning_format`, `reasoning_effort`, `enable_thinking`, `thinking_token_budget`, `preserve_thinking` | chat only; levels are the template's (`capabilities.reasoning_efforts`); `none`/`minimal` turn thinking off |
| `response_format`, `json_schema`, `grammar` | structured output; `grammar` and `response_format` are mutually exclusive |
| `echo`, `preserved_tokens` | `/v1/completions` only |
| `encoding_format`, `dimensions`, `prompt_name`, `image_tokens` | `/v1/embeddings` only |
| `images_config` | image models only: a named processor mode or an object restating served sizes |

## What is refused, ignored, and clamped

The contract: **a 200 means every field that could change the answer was honoured.** Fields fall
into four classes.

- **Inert** — accepted and ignored because they cannot change the answer: OpenAI/OpenRouter
  bookkeeping (`user`, `metadata`, `store`, `service_tier`, `prompt_cache_key`, …) and llama-server's
  own speed/slot knobs (`cache_prompt`, `id_slot`, `n_cache_reuse`, `backend_sampling`,
  `speculative.*`, …). The prefix cache and speculation here are the server's and never change output.
- **Guarded** — known, unimplemented, and able to change the answer: accepted at their off value,
  refused by name otherwise. `logit_bias` (empty), `mirostat` (0), `top_n_sigma` (≤0),
  `dynatemp_range` (≤0), `min_keep` (0), `post_sampling_probs` (false), `best_of` (≤1), `suffix`
  (empty), `lora` (empty — an adapter is chosen by `model`), `modalities`/`audio` (`["text"]`/null),
  `n_indent` (0), `t_max_predict_ms` (≤0), `return_tokens` (false), `grammar_lazy`,
  `grammar_triggers`, `preserved_tokens` (empty on chat).
- **Ignored** — anything else unknown, logged at debug. An unknown field is far more often a client's
  extension than a misunderstood request.
- **`max_tokens` is the exception**: if prompt + `max_tokens` exceeds the context, it is **clamped**
  and served, ending `finish_reason:"length"`, because refusing would produce no tokens instead of
  fewer. A prompt that does not fit on its own is a 400 `context_length_exceeded` naming
  `n_prompt_tokens`, `n_ctx`, and how many tokens must go.

## Anthropic, Responses, llama-server

- `/v1/messages` is translated into a chat request, so chat defaults, refusals and limits apply.
  Streamed (`message_start`, content blocks, `message_delta`, `message_stop`) or not:
  `ANTHROPIC_BASE_URL=http://host:8000 ANTHROPIC_API_KEY=$KEY claude` works. `/v1/messages/count_tokens`
  returns the prompt length.
- `/v1/responses` maps `instructions`/`input` (strings, messages, `function_call`,
  `function_call_output`, `reasoning` items) into the chat request; `max_output_tokens`,
  `reasoning.effort`, `text.format` map to chat fields; undefined fields pass through. The event stream
  carries `output_index`, `content_index`, `sequence_number`, and opens with `response.created` and
  `response.in_progress`. A reply cut at `max_output_tokens` is `incomplete`. Function tools a
  `namespace` groups are named `<namespace>__<name>`; built-in tools and freeform custom tools are
  left out.
- `/completion`, `/completions`, `/infill` are llama-server's own shapes, for llama.vim/llama.vscode.

## Auth, CORS, errors

- `--api-key KEY` requires `Authorization: Bearer KEY` or `X-Api-Key: KEY` on every request except
  `/health`, `/v1/health`, `/ping` and the dashboard page. An empty key is refused at startup.
- CORS headers are sent unless `--no-cors`.
- Errors are JSON. Over-context is 400 `context_length_exceeded`; a full queue is 429 with
  `Retry-After`; a capability the model lacks is 501; a guarded field is 400 naming the field and its
  accepted off value.

## Related

- [[Radiance Serve Options]] — the defaults every omitted field takes.
- [[Radiance Operations]] — `/health` semantics, metrics, dashboard.
- [[frank-archon]] — the harness contract built on this surface.