frank-default-large: full 262144 context, 1 slot
User request 2026-08-19 (second pass): parallel slots degraded performance. PARALLEL=1, CTX_SIZE=262144 (native max, -c equals the per-slot context with one slot). Concurrent requests queue on the single slot; the server never preempts a running generation. pi models.json contextWindow/maxTokens back to 262144 (outside repo). Docs updated to match.
This commit is contained in:
1 parent
8e4d63b68d
commit
048e7eaaba
6 files changed
+22
-20
No files matched your search
@@ -131,9 +131,10 @@ Downloaded with the HF token to
|
||||
|
||||
### Launch profile
|
||||
|
||||
- 200K context per slot (user request 2026-08-19; native max 262144),
|
||||
2 slots, f16 KV (~24 GiB total for both slots), context checkpoints
|
||||
(16), prompt cache
|
||||
- 262144 context (native max), 1 slot (user request 2026-08-19; parallel
|
||||
slots degraded performance), f16 KV (~16 GiB), context checkpoints (16),
|
||||
prompt cache. Concurrent requests queue on the single slot; the server
|
||||
never preempts a running generation
|
||||
- Native MTP draft, depth 4, device ROCm0
|
||||
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
|
||||
puts `<think>` reasoning in `message.reasoning_content`)
|
||||
|
||||
Reference in new issue
Block a user