serverAppConfigs/serverAppConfigs-docs/LLM/Radiance HTTP API.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	Radiance HTTP API — endpoints, fields, refusals	The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules.	llm radiance api reference	2026-10-30

Radiance HTTP API

One server serves one model. It answers OpenAI's chat/completions/Responses/images/audio/video APIs, Anthropic's Messages API and llama-server's native one, so their clients and SDKs work unchanged. model in a request is accepted and not enforced.

Endpoints

route	what it serves
POST /v1/chat/completions	chat: tools, reasoning, streaming, grammars
POST /v1/completions	raw text completion
POST /v1/embeddings	an embedding head, else 501
POST /v1/images/generations, /v1/images/edits	OpenAI images API; needs an image model, else 501
POST /v1/systemone	LiquidAI d1 decisions: named yes/no, choice and score questions as probabilities; needs a d1 container, else 501
POST /v1/audio/transcriptions	OpenAI speech-to-text; needs a transcribing plugin, else 501
GET /v1/realtime (WebSocket)	streaming transcription session
POST /v1/audio/speech	speech (Breeze-TTS-2, drama-3) or a song (MiniMax-Music3), whole or streamed
GET /v1/audio/speech/stream (WebSocket)	text streamed in, each sentence's audio out
POST /v1/videos; GET /v1/videos[/:id[/content]]; DELETE /v1/videos/:id	video jobs made in the background, downloaded as MP4
POST /v1/messages, /v1/messages/count_tokens	Anthropic Messages, for Claude Code and the Anthropic SDKs
POST /v1/responses, /v1/responses/input_tokens	OpenAI Responses, for Codex and SDK clients
POST /completion, /completions, /infill	llama-server native completion and fill-in-the-middle
GET /v1/models, /v1/models/:id	what is loaded, with max_model_len, root, capabilities
GET /health, /v1/health, /ping, POST /ping	liveness
GET /load, /version, /props, /server_info, /diagnostics	load; release version; llama-server props; full configuration; the startup report as JSON (/diagnostics is same-machine-only without an API key)
POST /apply-template, /v1/chat/completions/input_tokens	the rendered prompt; its token count
GET /metrics	Prometheus
POST /tokenize, /detokenize; GET /get_tokenizer_info	vLLM's shape and llama-server's at once
POST /reset_prefix_cache; DELETE /sessions/<label>, /sessions?id=<id>	drop every cached prefix; forget a stored conversation
GET /, /dashboard; GET /stats, /graph, /sessions	the dashboard and the JSON it is drawn from

Unversioned shorthands exist for chat, chat input tokens, embeddings, images, audio transcriptions, audio speech, models, responses and completions, for a client given the bare host as base URL. /completions without /v1 is llama-server's native completion. A path nothing serves is a JSON 404.

A text prompt to /v1/completions, /v1/embeddings or /tokenize reads control tokens as tokens (as vLLM and llama.cpp do), so a client rendering its own template can send <|im_start|>user\n…. The model's declared BOS is prepended unless the prompt already opens with it.

Chat

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":200,"temperature":0.7}'

• No chat template → 501 on this route (the container carries one; --override-chat-template replaces it). Use /v1/completions instead.
• Thinking off has three spellings: chat_template_kwargs.enable_thinking=false, top-level enable_thinking=false, or reasoning_effort:"none".
• Reasoning returns in choices[].message.reasoning_content, separate from content. Reasoning counts against max_tokens; a reply that runs out inside it has empty content and finish_reason:"length". usage.completion_tokens_details.reasoning_tokens tells that case from a model that answered nothing.
• thinking_token_budget (aliases thinking_budget, thinking_budget_tokens, reasoning_budget_tokens) closes the reasoning block at N tokens and forces the close, so it holds under speculation and at any temperature. It needs a one-token end marker; otherwise the response has no completion_tokens_details and a budget is a 400.
• continue_final_message continues a conversation ending in an assistant message instead of answering it; add_generation_prompt opens a new turn.
• Tool calls: tools, tool_choice (auto, none, required, named function), parallel_tool_calls. required/named hold the whole reply to calls after the reasoning block and cannot combine with response_format or grammar.
• Structured output: response_format (json_object or json_schema), llama-server's json_schema shorthand, or a raw GBNF grammar. A chat template that drops the grammar makes the request a refusal.

Streaming

curl -N localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"m","messages":[{"role":"user","content":"Count to ten."}],
       "max_tokens":100,"stream":true,"stream_options":{"include_usage":true}}'

Server-sent events in OpenAI's delta format, terminated by data: [DONE]. Nothing is sent while the prompt is read: the first event (the role: assistant chunk) goes out with the first generated token, so a client timing its first event measures time-to-first-token. return_progress adds llama-server's prompt_progress chunks during prefill; timings_per_token adds timings to every chunk.

Completions, embeddings, images, audio, video, tokens

• Completions — /v1/completions takes prompt (string or token ids), echo, logprobs, preserved_tokens. /completion is llama-server's native route; /infill is fill-in-the-middle.
• Embeddings — input is text, a batch, or token ids; encoding_format float/base64; dimensions truncates to the first N and renormalises; prompt_name prepends the model's named prompt. Pooling is the model's; the engine does not renormalise.
• Images — /v1/images/generations returns b64_json; size defaults to 2048×2048. Edits take one picture or up to ten (image[]), multipart as OpenAI's SDKs send them, with an optional mask (repaint only where transparent). partial_images streams previews as SSE.
• Speech to text — /v1/audio/transcriptions; /v1/realtime streams audio in and words back as they are spoken. The model's prompt shape comes from its plugin's audio format, not from the server.
• Speech and music — /v1/audio/speech (whole or streamed) with instructions describing a voice (Breeze-TTS-2) or speakers written in the input (<|speaker:1|>, [laughs] for drama-3); a music model takes the lyrics as input and the description as instructions. /v1/audio/speech/stream streams text in and audio out as each sentence completes.
• Video — /v1/videos is a job: a video with its stereo soundtrack from a prompt, 5–14 s, up to 1344×768, made in the background; GET /v1/videos/:id/content downloads the MP4.
• Tokenising — POST /tokenize accepts vLLM's shape (prompt or messages in, prompt out) and llama-server's (content in/out, with_pieces). POST /detokenize reverses it. GET /get_tokenizer_info reports the tokeniser configuration.

Request fields

Accepted fields, in OpenAI's, vLLM's and llama-server's spellings:

field	notes
model	accepted, not enforced
messages / prompt / input	chat / completion / embedding
max_tokens, max_completion_tokens, n_predict	default is what the context leaves; -1 is llama-server's "no limit"; clamped to context and to --max-tokens-cap
temperature, top_p, top_k, min_p, typical_p	sampler
presence_penalty, frequency_penalty, repetition_penalty/repeat_penalty, repeat_last_n	penalties
xtc_probability, xtc_threshold, dry_*	XTC and DRY
seed	absent means "pick one", not 0
n	1..--max-n; prefix cache makes the fan-out nearly free after the first
stop	string or array, bounded by --max-stop-strings/--max-stop-bytes
priority	scheduler hint
ignore_eos	generate to max_tokens whatever the model emits
stream, stream_options.include_usage, timings_per_token, return_progress	streaming controls
samplers	llama-server's order; this engine's chain is penalties, DRY, top-k, top-p, min-p, typical, XTC, with temperature first or last; any other list is refused by name
session_id	labels the conversation on /sessions, dashboard and DELETE
control_vector	true applies the deployment's vector to this request; absent takes the deployment default; a deployment with no vector refuses true
logprobs, top_logprobs, n_probs	up to 20 alternatives; not with echo
tools, tool_choice, parallel_tool_calls, legacy functions/function_call	chat only
add_generation_prompt, chat_template_kwargs, continue_final_message	chat only
reasoning_format, reasoning_effort, enable_thinking, thinking_token_budget, preserve_thinking	chat only; levels are the template's (capabilities.reasoning_efforts); none/minimal turn thinking off
response_format, json_schema, grammar	structured output; grammar and response_format are mutually exclusive
echo, preserved_tokens	/v1/completions only
encoding_format, dimensions, prompt_name, image_tokens	/v1/embeddings only
images_config	image models only: a named processor mode or an object restating served sizes

What is refused, ignored, and clamped

The contract: a 200 means every field that could change the answer was honoured. Fields fall into four classes.

• Inert — accepted and ignored because they cannot change the answer: OpenAI/OpenRouter bookkeeping (user, metadata, store, service_tier, prompt_cache_key, …) and llama-server's own speed/slot knobs (cache_prompt, id_slot, n_cache_reuse, backend_sampling, speculative.*, …). The prefix cache and speculation here are the server's and never change output.
• Guarded — known, unimplemented, and able to change the answer: accepted at their off value, refused by name otherwise. logit_bias (empty), mirostat (0), top_n_sigma (≤0), dynatemp_range (≤0), min_keep (0), post_sampling_probs (false), best_of (≤1), suffix (empty), lora (empty — an adapter is chosen by model), modalities/audio (["text"]/null), n_indent (0), t_max_predict_ms (≤0), return_tokens (false), grammar_lazy, grammar_triggers, preserved_tokens (empty on chat).
• Ignored — anything else unknown, logged at debug. An unknown field is far more often a client's extension than a misunderstood request.
• max_tokens is the exception: if prompt + max_tokens exceeds the context, it is clamped and served, ending finish_reason:"length", because refusing would produce no tokens instead of fewer. A prompt that does not fit on its own is a 400 context_length_exceeded naming n_prompt_tokens, n_ctx, and how many tokens must go.

Anthropic, Responses, llama-server

• /v1/messages is translated into a chat request, so chat defaults, refusals and limits apply. Streamed (message_start, content blocks, message_delta, message_stop) or not: ANTHROPIC_BASE_URL=http://host:8000 ANTHROPIC_API_KEY=$KEY claude works. /v1/messages/count_tokens returns the prompt length.
• /v1/responses maps instructions/input (strings, messages, function_call, function_call_output, reasoning items) into the chat request; max_output_tokens, reasoning.effort, text.format map to chat fields; undefined fields pass through. The event stream carries output_index, content_index, sequence_number, and opens with response.created and response.in_progress. A reply cut at max_output_tokens is incomplete. Function tools a namespace groups are named <namespace>__<name>; built-in tools and freeform custom tools are left out.
• /completion, /completions, /infill are llama-server's own shapes, for llama.vim/llama.vscode.

Auth, CORS, errors

• --api-key KEY requires Authorization: Bearer KEY or X-Api-Key: KEY on every request except /health, /v1/health, /ping and the dashboard page. An empty key is refused at startup.
• CORS headers are sent unless --no-cors.
• Errors are JSON. Over-context is 400 context_length_exceeded; a full queue is 429 with Retry-After; a capability the model lacks is 501; a guarded field is 400 naming the field and its accepted off value.

Related

• Radiance Serve Options — the defaults every omitted field takes.
• Radiance Operations — /health semantics, metrics, dashboard.
• frank-archon — the harness contract built on this surface.