frank-default-large: full 262144 context, 1 slot

User request 2026-08-19 (second pass): parallel slots degraded
performance. PARALLEL=1, CTX_SIZE=262144 (native max, -c equals the
per-slot context with one slot). Concurrent requests queue on the
single slot; the server never preempts a running generation.
pi models.json contextWindow/maxTokens back to 262144 (outside repo).
Docs updated to match.
This commit is contained in:
Frank committed 2026-08-19 11:17:31 -07:00
1 parent 8e4d63b68d
commit 048e7eaaba
6 files changed
+22 -20

No files matched your search

+4 -3
View File
@@ -131,9 +131,10 @@ Downloaded with the HF token to
### Launch profile
- 200K context per slot (user request 2026-08-19; native max 262144),
2 slots, f16 KV (~24 GiB total for both slots), context checkpoints
(16), prompt cache
- 262144 context (native max), 1 slot (user request 2026-08-19; parallel
slots degraded performance), f16 KV (~16 GiB), context checkpoints (16),
prompt cache. Concurrent requests queue on the single slot; the server
never preempts a running generation
- Native MTP draft, depth 4, device ROCm0
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
puts `<think>` reasoning in `message.reasoning_content`)