frank-default-small: lower context by 32k per stream for cache headroom
3 slots x 229376 (262144 - 32768) = 688128 total context. Updates the .env, models.json (contextWindow/maxTokens 229376 per slot), and the index/deployment/ornith runbook references.
This commit is contained in:
1 parent
4462aa992c
commit
7ba1acdbc5
4 files changed
+8
-7
No files matched your search
+1
-1
@@ -20,7 +20,7 @@ graph LR
|
||||
B[upstream llama.cpp b10121
|
||||
frank-default-small, port 7770]
|
||||
C[Ornith-1.5-35B-A3B-CRACK Q8_0
|
||||
native MTP, MoE experts offloaded via override, 786432 ctx]
|
||||
native MTP, MoE experts offloaded via override, 688128 ctx]
|
||||
A --> B --> C
|
||||
end
|
||||
|
||||
|
||||
Reference in new issue
Block a user