frank-large: roll back to gufo Qwen3.8-27B Q4_K_XL, one session x 262144 f16 KV
frank-small: llama.cpp CUDA (master, sm_86) on MiMo Q6_K GGUF, 2 x 262144, asymmetric KV (f16 K + q8_0 V, flash-attn pinned for the V quant)
This commit is contained in:
1 parent
e452a1afbf
commit
8551bb006b
11 files changed
+729
-816
No files matched your search
@@ -84,7 +84,8 @@ def stream_completion(url: str, model: str, prompt: str):
|
||||
d = ch.get("delta") or {}
|
||||
# reasoning AND content deltas both arrive token by token;
|
||||
# count either so greedy text length drives the rate.
|
||||
if d.get("content") or d.get("reasoning"):
|
||||
# (llama.cpp names the thinking field reasoning_content.)
|
||||
if d.get("content") or d.get("reasoning") or d.get("reasoning_content"):
|
||||
now = time.perf_counter()
|
||||
if first is None:
|
||||
first = now
|
||||
|
||||
Reference in new issue
Block a user