docs(sightsound): correct unknown-model-tag contract to live-probed behavior
Prior docs claimed unknown tags silently serve llama3.2:1b. Live probes on the
live endpoint disprove it: /v1/chat/completions answers HTTP 200 with a flat
{"error":"Failed to load <tag> model!"} body and no choices; /v1/completions
answers a nested OpenAI-shaped server_error (code 500) under HTTP 200. Valid
tags route per-request to the named model (response echoes the requested tag);
no restart needed to switch between registry tags. Corrected in the runbook,
the AGENTS.md runtime section, the serve-script header comment, and the
historical FastFlowLM incident note.
This commit is contained in:
1 parent
150412bbca
commit
9b9f0f9051
4 files changed
+22
-9
No files matched your search
@@ -290,6 +290,11 @@ When a request names a tag that is not in the FLM registry, FLM does not return
|
||||
- FLM then loads the 1B model in place of the requested model.
|
||||
- The files now exist in `~/.config/flm/models/Llama-3.2-1B-NPU2` (1.3 GB) on dankclaw. A future unknown-tag request loads the 1B model silently, with no download and no error.
|
||||
|
||||
**Correction (live-probed 2026-08-30):** the fallback no longer fires on the
|
||||
current serve path — an unknown tag now answers HTTP 200 with a flat
|
||||
`{"error":"Failed to load <tag> model!"}` body and nothing is loaded or served;
|
||||
`/v1/completions` answers a nested `server_error` (code 500) under HTTP 200.
|
||||
|
||||
Keep client model IDs as FLM registry tags. Never send a service alias. Optional cleanup: move `~/.config/flm/models/Llama-3.2-1B-NPU2` (an incident artifact) to `.AGENTTRASH` so a stray tag re-downloads loudly instead of serving the wrong model.
|
||||
|
||||
### Request compatibility: the `developer` role (found 2026-08-16)
|
||||
|
||||
@@ -47,9 +47,14 @@ swap is therefore an env-file edit, not a script edit.
|
||||
- The endpoint serves the **whole local registry** (36 tags at first deploy) on
|
||||
one port; `/v1/models` lists them all. Chat completions route per request by
|
||||
the exact FLM registry tag in the `model` field — never a service alias.
|
||||
- **Unknown model tags do NOT 404**: FLM logs an error and silently serves
|
||||
`llama3.2:1b` instead. Client configs must carry the exact registry tag or
|
||||
they will silently get the 1B fallback.
|
||||
- **Unknown model tags do NOT 404** (live-probed): `/v1/chat/completions`
|
||||
answers HTTP 200 with a flat error body
|
||||
`{"error":"Failed to load <tag> model!"}` and no `choices`;
|
||||
`/v1/completions` answers an OpenAI-shaped nested error
|
||||
(`type: server_error`, `code: 500`), also under HTTP 200. A valid tag
|
||||
routes that request to that model (response echoes the requested tag); no
|
||||
server restart is needed to switch between registry tags. Status-code-only
|
||||
clients silently get an empty response — check for the `error` key.
|
||||
- Multimodal content parts accepted by `/v1/chat/completions`:
|
||||
`{"type":"image_url","image_url":{"url":"data:image/png;base64,..."}}` and
|
||||
`{"type":"input_audio","input_audio":{"data":"<base64 WAV, no data: prefix>","format":"wav"}}`.
|
||||
@@ -100,7 +105,7 @@ fault. Do not set `iommu=off` on this host again; it silently kills the NPU.
|
||||
|
||||
The pi client model config (`/home/dank/.pi/agent/models.json` on Dankware)
|
||||
registers this endpoint under the Dankclaw provider with the **exact** FLM
|
||||
registry tag as the model `id` (see above: unknown tags silently fall back to
|
||||
the 1B), `baseUrl` `http://192.168.0.143:52626/v1`, and a per-request context
|
||||
registry tag as the model `id` (unknown tags answer a 200-wrapped error, not
|
||||
a silent 1B fallback), `baseUrl` `http://192.168.0.143:52626/v1`, and a per-request context
|
||||
window equal to the serving ctx-len. Health checks for this service must use
|
||||
`GET /v1/version` (there is no `/health` route).
|
||||
Reference in new issue
Block a user