docs(radiance): reference the engine's options, plugins and conversion

Nine pages under LLM/ document the upstream radiance engine at 8da043c: - engine shape, plugin model, RAD2 container, ABI and environment - every serve flag grouped with defaults and the law behind each knob - the HTTP surface, request fields and refusal classes - rad-convert, the recipe language, quantisers, imports, reuse, rad-export, KL scoring - the (arch id, quant) family table and the recipe of record per model - kernel/quantiser plugin authoring, encodings, build and checks - architecture plugin authoring: declare, step, probe, lane and media rules - rad-info/radbench/radtune and the shell harnesses - the container contract, host prerequisites, readiness and restart triage Also completes the dangling Related line on the frank-archon endpoint page.
2026-10-30 15:01:37 -07:00
1 parent da5eb968f9
commit 3a04e34414
@@ -0,0 +1,198 @@
+---
+type: "reference"
+title: "Radiance HTTP API — endpoints, fields, refusals"
+description: "The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules."
+tags:
+  - llm
+  - radiance
+  - api
+  - reference
+timestamp: 2026-10-30
+---
+
+# Radiance HTTP API
+
+One server serves one model. It answers OpenAI's chat/completions/Responses/images/audio/video APIs,
+Anthropic's Messages API and llama-server's native one, so their clients and SDKs work unchanged.
+`model` in a request is accepted and **not enforced**.
+
+## Endpoints
+
+| route | what it serves |
+|---|---|
+| `POST /v1/chat/completions` | chat: tools, reasoning, streaming, grammars |
+| `POST /v1/completions` | raw text completion |
+| `POST /v1/embeddings` | an embedding head, else 501 |
+| `POST /v1/images/generations`, `/v1/images/edits` | OpenAI images API; needs an image model, else 501 |
+| `POST /v1/systemone` | LiquidAI d1 decisions: named yes/no, choice and score questions as probabilities; needs a d1 container, else 501 |
+| `POST /v1/audio/transcriptions` | OpenAI speech-to-text; needs a transcribing plugin, else 501 |
+| `GET /v1/realtime` (WebSocket) | streaming transcription session |
+| `POST /v1/audio/speech` | speech (Breeze-TTS-2, drama-3) or a song (MiniMax-Music3), whole or streamed |
+| `GET /v1/audio/speech/stream` (WebSocket) | text streamed in, each sentence's audio out |
+| `POST /v1/videos`; `GET /v1/videos[/:id[/content]]`; `DELETE /v1/videos/:id` | video jobs made in the background, downloaded as MP4 |
+| `POST /v1/messages`, `/v1/messages/count_tokens` | Anthropic Messages, for Claude Code and the Anthropic SDKs |
+| `POST /v1/responses`, `/v1/responses/input_tokens` | OpenAI Responses, for Codex and SDK clients |
+| `POST /completion`, `/completions`, `/infill` | llama-server native completion and fill-in-the-middle |
+| `GET /v1/models`, `/v1/models/:id` | what is loaded, with `max_model_len`, `root`, `capabilities` |
+| `GET /health`, `/v1/health`, `/ping`, `POST /ping` | liveness |
+| `GET /load`, `/version`, `/props`, `/server_info`, `/diagnostics` | load; release version; llama-server props; full configuration; the startup report as JSON (`/diagnostics` is same-machine-only without an API key) |
+| `POST /apply-template`, `/v1/chat/completions/input_tokens` | the rendered prompt; its token count |
+| `GET /metrics` | Prometheus |
+| `POST /tokenize`, `/detokenize`; `GET /get_tokenizer_info` | vLLM's shape and llama-server's at once |
+| `POST /reset_prefix_cache`; `DELETE /sessions/<label>`, `/sessions?id=<id>` | drop every cached prefix; forget a stored conversation |
+| `GET /`, `/dashboard`; `GET /stats`, `/graph`, `/sessions` | the dashboard and the JSON it is drawn from |
+
+Unversioned shorthands exist for chat, chat input tokens, embeddings, images, audio transcriptions,
+audio speech, models, responses and completions, for a client given the bare host as base URL.
+`/completions` **without** `/v1` is llama-server's native completion. A path nothing serves is a JSON
+404.
+
+A text prompt to `/v1/completions`, `/v1/embeddings` or `/tokenize` reads control tokens as tokens
+(as vLLM and llama.cpp do), so a client rendering its own template can send `<|im_start|>user\n…`.
+The model's declared BOS is prepended unless the prompt already opens with it.
+
+## Chat
+
+```sh
+curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
+  -d '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":200,"temperature":0.7}'
+```
+
+- **No chat template → 501** on this route (the container carries one; `--override-chat-template`
+  replaces it). Use `/v1/completions` instead.
+- Thinking off has three spellings: `chat_template_kwargs.enable_thinking=false`,
+  top-level `enable_thinking=false`, or `reasoning_effort:"none"`.
+- Reasoning returns in `choices[].message.reasoning_content`, separate from `content`.
+  **Reasoning counts against `max_tokens`**; a reply that runs out inside it has empty `content` and
+  `finish_reason:"length"`. `usage.completion_tokens_details.reasoning_tokens` tells that case from a
+  model that answered nothing.
+- `thinking_token_budget` (aliases `thinking_budget`, `thinking_budget_tokens`,
+  `reasoning_budget_tokens`) closes the reasoning block at N tokens and forces the close, so it holds
+  under speculation and at any temperature. It needs a one-token end marker; otherwise the response
+  has no `completion_tokens_details` and a budget is a 400.
+- `continue_final_message` continues a conversation ending in an assistant message instead of
+  answering it; `add_generation_prompt` opens a new turn.
+- Tool calls: `tools`, `tool_choice` (`auto`, `none`, `required`, named function),
+  `parallel_tool_calls`. `required`/named hold the whole reply to calls after the reasoning block and
+  cannot combine with `response_format` or `grammar`.
+- Structured output: `response_format` (`json_object` or `json_schema`), llama-server's `json_schema`
+  shorthand, or a raw GBNF `grammar`. A chat template that drops the grammar makes the request a
+  refusal.
+
+## Streaming
+
+```sh
+curl -N localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
+  -d '{"model":"m","messages":[{"role":"user","content":"Count to ten."}],
+       "max_tokens":100,"stream":true,"stream_options":{"include_usage":true}}'
+```
+
+Server-sent events in OpenAI's delta format, terminated by `data: [DONE]`. **Nothing is sent while
+the prompt is read**: the first event (the `role: assistant` chunk) goes out with the first generated
+token, so a client timing its first event measures time-to-first-token. `return_progress` adds
+llama-server's `prompt_progress` chunks during prefill; `timings_per_token` adds `timings` to every
+chunk.
+
+## Completions, embeddings, images, audio, video, tokens
+
+- **Completions** — `/v1/completions` takes `prompt` (string or token ids), `echo`, `logprobs`,
+  `preserved_tokens`. `/completion` is llama-server's native route; `/infill` is fill-in-the-middle.
+- **Embeddings** — `input` is text, a batch, or token ids; `encoding_format` `float`/`base64`;
+  `dimensions` truncates to the first N and renormalises; `prompt_name` prepends the model's named
+  prompt. Pooling is the model's; the engine does not renormalise.
+- **Images** — `/v1/images/generations` returns `b64_json`; `size` defaults to 2048×2048. Edits take
+  one picture or up to ten (`image[]`), multipart as OpenAI's SDKs send them, with an optional mask
+  (repaint only where transparent). `partial_images` streams previews as SSE.
+- **Speech to text** — `/v1/audio/transcriptions`; `/v1/realtime` streams audio in and words back as
+  they are spoken. The model's prompt shape comes from its plugin's audio format, not from the server.
+- **Speech and music** — `/v1/audio/speech` (whole or streamed) with `instructions` describing a voice
+  (Breeze-TTS-2) or speakers written in the input (`<|speaker:1|>`, `[laughs]` for drama-3); a music
+  model takes the lyrics as `input` and the description as `instructions`. `/v1/audio/speech/stream`
+  streams text in and audio out as each sentence completes.
+- **Video** — `/v1/videos` is a job: a video with its stereo soundtrack from a prompt, 5–14 s, up to
+  1344×768, made in the background; `GET /v1/videos/:id/content` downloads the MP4.
+- **Tokenising** — `POST /tokenize` accepts vLLM's shape (`prompt` or `messages` in, `prompt` out) and
+  llama-server's (`content` in/out, `with_pieces`). `POST /detokenize` reverses it.
+  `GET /get_tokenizer_info` reports the tokeniser configuration.
+
+## Request fields
+
+Accepted fields, in OpenAI's, vLLM's and llama-server's spellings:
+
+| field | notes |
+|---|---|
+| `model` | accepted, not enforced |
+| `messages` / `prompt` / `input` | chat / completion / embedding |
+| `max_tokens`, `max_completion_tokens`, `n_predict` | default is what the context leaves; `-1` is llama-server's "no limit"; clamped to context and to `--max-tokens-cap` |
+| `temperature`, `top_p`, `top_k`, `min_p`, `typical_p` | sampler |
+| `presence_penalty`, `frequency_penalty`, `repetition_penalty`/`repeat_penalty`, `repeat_last_n` | penalties |
+| `xtc_probability`, `xtc_threshold`, `dry_*` | XTC and DRY |
+| `seed` | absent means "pick one", not 0 |
+| `n` | 1..`--max-n`; prefix cache makes the fan-out nearly free after the first |
+| `stop` | string or array, bounded by `--max-stop-strings`/`--max-stop-bytes` |
+| `priority` | scheduler hint |
+| `ignore_eos` | generate to `max_tokens` whatever the model emits |
+| `stream`, `stream_options.include_usage`, `timings_per_token`, `return_progress` | streaming controls |
+| `samplers` | llama-server's order; this engine's chain is penalties, DRY, top-k, top-p, min-p, typical, XTC, with temperature first or last; any other list is refused by name |
+| `session_id` | labels the conversation on `/sessions`, dashboard and `DELETE` |
+| `control_vector` | `true` applies the deployment's vector to this request; absent takes the deployment default; a deployment with no vector refuses `true` |
+| `logprobs`, `top_logprobs`, `n_probs` | up to 20 alternatives; not with `echo` |
+| `tools`, `tool_choice`, `parallel_tool_calls`, legacy `functions`/`function_call` | chat only |
+| `add_generation_prompt`, `chat_template_kwargs`, `continue_final_message` | chat only |
+| `reasoning_format`, `reasoning_effort`, `enable_thinking`, `thinking_token_budget`, `preserve_thinking` | chat only; levels are the template's (`capabilities.reasoning_efforts`); `none`/`minimal` turn thinking off |
+| `response_format`, `json_schema`, `grammar` | structured output; `grammar` and `response_format` are mutually exclusive |
+| `echo`, `preserved_tokens` | `/v1/completions` only |
+| `encoding_format`, `dimensions`, `prompt_name`, `image_tokens` | `/v1/embeddings` only |
+| `images_config` | image models only: a named processor mode or an object restating served sizes |
+
+## What is refused, ignored, and clamped
+
+The contract: **a 200 means every field that could change the answer was honoured.** Fields fall
+into four classes.
+
+- **Inert** — accepted and ignored because they cannot change the answer: OpenAI/OpenRouter
+  bookkeeping (`user`, `metadata`, `store`, `service_tier`, `prompt_cache_key`, …) and llama-server's
+  own speed/slot knobs (`cache_prompt`, `id_slot`, `n_cache_reuse`, `backend_sampling`,
+  `speculative.*`, …). The prefix cache and speculation here are the server's and never change output.
+- **Guarded** — known, unimplemented, and able to change the answer: accepted at their off value,
+  refused by name otherwise. `logit_bias` (empty), `mirostat` (0), `top_n_sigma` (≤0),
+  `dynatemp_range` (≤0), `min_keep` (0), `post_sampling_probs` (false), `best_of` (≤1), `suffix`
+  (empty), `lora` (empty — an adapter is chosen by `model`), `modalities`/`audio` (`["text"]`/null),
+  `n_indent` (0), `t_max_predict_ms` (≤0), `return_tokens` (false), `grammar_lazy`,
+  `grammar_triggers`, `preserved_tokens` (empty on chat).
+- **Ignored** — anything else unknown, logged at debug. An unknown field is far more often a client's
+  extension than a misunderstood request.
+- **`max_tokens` is the exception**: if prompt + `max_tokens` exceeds the context, it is **clamped**
+  and served, ending `finish_reason:"length"`, because refusing would produce no tokens instead of
+  fewer. A prompt that does not fit on its own is a 400 `context_length_exceeded` naming
+  `n_prompt_tokens`, `n_ctx`, and how many tokens must go.
+
+## Anthropic, Responses, llama-server
+
+- `/v1/messages` is translated into a chat request, so chat defaults, refusals and limits apply.
+  Streamed (`message_start`, content blocks, `message_delta`, `message_stop`) or not:
+  `ANTHROPIC_BASE_URL=http://host:8000 ANTHROPIC_API_KEY=$KEY claude` works. `/v1/messages/count_tokens`
+  returns the prompt length.
+- `/v1/responses` maps `instructions`/`input` (strings, messages, `function_call`,
+  `function_call_output`, `reasoning` items) into the chat request; `max_output_tokens`,
+  `reasoning.effort`, `text.format` map to chat fields; undefined fields pass through. The event stream
+  carries `output_index`, `content_index`, `sequence_number`, and opens with `response.created` and
+  `response.in_progress`. A reply cut at `max_output_tokens` is `incomplete`. Function tools a
+  `namespace` groups are named `<namespace>__<name>`; built-in tools and freeform custom tools are
+  left out.
+- `/completion`, `/completions`, `/infill` are llama-server's own shapes, for llama.vim/llama.vscode.
+
+## Auth, CORS, errors
+
+- `--api-key KEY` requires `Authorization: Bearer KEY` or `X-Api-Key: KEY` on every request except
+  `/health`, `/v1/health`, `/ping` and the dashboard page. An empty key is refused at startup.
+- CORS headers are sent unless `--no-cors`.
+- Errors are JSON. Over-context is 400 `context_length_exceeded`; a full queue is 429 with
+  `Retry-After`; a capability the model lacks is 501; a guarded field is 400 naming the field and its
+  accepted off value.
+
+## Related
+
+- [[Radiance Serve Options]] — the defaults every omitted field takes.
+- [[Radiance Operations]] — `/health` semantics, metrics, dashboard.
+- [[frank-archon]] — the harness contract built on this surface.