dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: FLM rejects developer role — root cause of pi 'Stream ended without finish_reason'

Browse Source 
Streaming requests with role=developer get HTTP 200 + error JSON as
first body; pi sees a chunkless stream. Fix: compat.supportsDeveloperRole
false on the model entry. Verified by field bisection + headless pi run.
This commit is contained in:
Frank
2026-08-16 20:54:13 -07:00
parent 9ec5678ef0
commit f5a78db017
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FastFlowLM NPU Inference.md
+10
View File
@@ -273,6 +273,16 @@ When a request names a tag that is not in the FLM registry, FLM does not return
Keep client model IDs as FLM registry tags. Never send a service alias. Optional cleanup: move `~/.config/flm/models/Llama-3.2-1B-NPU2` (an incident artifact) to `.AGENTTRASH` so a stray tag re-downloads loudly instead of serving the wrong model.
### Request compatibility: the `developer` role (found 2026-08-16)
pi sessions failed against this endpoint with `Stream ended without finish_reason` while curl worked. Root cause, verified by field bisection:
- The FLM Qwen chat template accepts only the roles `system`, `user`, and `assistant`. Any other role raises `Unexpected message role.` in the template.
- FLM reports the error differently per mode. Non-streaming: HTTP 500 with the template traceback. Streaming: HTTP 200, and the error JSON is the first body. A streaming client sees a stream with no chunks and no `finish_reason`.
- pi sends `role: "developer"` when `model.reasoning && compat.supportsDeveloperRole` (pi source, `openai-completions.js`). The default `supportsDeveloperRole` is true for standard OpenAI-compatible providers, so a reasoning model without a `compat` block always triggers the bug.
- Fix: `"compat": { "supportsDeveloperRole": false }` on the model entry in `~/.pi/agent/models.json`. pi then sends `role: "system"`. Verified with a headless pi session: full round trip, `finish_reason: stop`.
- FLM ignores unknown body fields. `reasoning_effort`, `store`, `max_completion_tokens`, and `tools: []` all arrive in pi requests and pass through without error.
## Risks and Open Questions
- **Ubuntu 26.04 is unofficial.** The distro list stops at 25.10. Validation passes, but breakage would fall on the "generic Linux" path.