diff --git a/docs/FastFlowLM NPU Inference.md b/docs/FastFlowLM NPU Inference.md index 8bc8a7d..ec59d3c 100644 --- a/docs/FastFlowLM NPU Inference.md +++ b/docs/FastFlowLM NPU Inference.md @@ -273,6 +273,16 @@ When a request names a tag that is not in the FLM registry, FLM does not return Keep client model IDs as FLM registry tags. Never send a service alias. Optional cleanup: move `~/.config/flm/models/Llama-3.2-1B-NPU2` (an incident artifact) to `.AGENTTRASH` so a stray tag re-downloads loudly instead of serving the wrong model. +### Request compatibility: the `developer` role (found 2026-08-16) + +pi sessions failed against this endpoint with `Stream ended without finish_reason` while curl worked. Root cause, verified by field bisection: + +- The FLM Qwen chat template accepts only the roles `system`, `user`, and `assistant`. Any other role raises `Unexpected message role.` in the template. +- FLM reports the error differently per mode. Non-streaming: HTTP 500 with the template traceback. Streaming: HTTP 200, and the error JSON is the first body. A streaming client sees a stream with no chunks and no `finish_reason`. +- pi sends `role: "developer"` when `model.reasoning && compat.supportsDeveloperRole` (pi source, `openai-completions.js`). The default `supportsDeveloperRole` is true for standard OpenAI-compatible providers, so a reasoning model without a `compat` block always triggers the bug. +- Fix: `"compat": { "supportsDeveloperRole": false }` on the model entry in `~/.pi/agent/models.json`. pi then sends `role: "system"`. Verified with a headless pi session: full round trip, `finish_reason: stop`. +- FLM ignores unknown body fields. `reasoning_effort`, `store`, `max_completion_tokens`, and `tools: []` all arrive in pi requests and pass through without error. + ## Risks and Open Questions - **Ubuntu 26.04 is unofficial.** The distro list stops at 25.10. Validation passes, but breakage would fall on the "generic Linux" path.