frank-large: roll back to gufo Qwen3.8-27B Q4_K_XL, one session x 262144 f16 KV

frank-small: llama.cpp CUDA (master, sm_86) on MiMo Q6_K GGUF, 2 x 262144,
asymmetric KV (f16 K + q8_0 V, flash-attn pinned for the V quant)
This commit is contained in:
Frank committed 2026-10-01 22:40:59 -07:00
1 parent e452a1afbf
commit 8551bb006b
11 files changed
+729 -816

No files matched your search

+2 -1
View File
@@ -84,7 +84,8 @@ def stream_completion(url: str, model: str, prompt: str):
d = ch.get("delta") or {}
# reasoning AND content deltas both arrive token by token;
# count either so greedy text length drives the rate.
if d.get("content") or d.get("reasoning"):
# (llama.cpp names the thinking field reasoning_content.)
if d.get("content") or d.get("reasoning") or d.get("reasoning_content"):
now = time.perf_counter()
if first is None:
first = now