Commit Graph
6 Commits
Author SHA1 Message Date
Frank f8f845d86b autoresearch: harness setup
Benchmark entrypoint: bash autoresearch.sh
Goal: Make frank-primary faster: raise decode tok/s on the tcclaviger vLLM endpoint (dankserver :7770, Qwen3.8-Flash-Next-Uncensored-FP8, 2x R9700, automatic tiered-expert offload). Context size may be altered (MAXLEN/SEQS/KV_BYTES and friends); quantization may NOT be altered (the 131-shard FP8 payload and the digest-pinned image are frozen).
2026-10-02 07:50:50 -07:00
Frank 8551bb006b frank-large: roll back to gufo Qwen3.8-27B Q4_K_XL, one session x 262144 f16 KV
frank-small: llama.cpp CUDA (master, sm_86) on MiMo Q6_K GGUF, 2 x 262144,
asymmetric KV (f16 K + q8_0 V, flash-attn pinned for the V quant)
2026-10-01 22:40:59 -07:00
Frank e452a1afbf harness: tolerate missing container in ASI capture; llama cmdline fallback 2026-10-01 22:11:34 -07:00
Frank 01ba41cab4 harness: llama-server lifecycle in pre-clean and health guard 2026-10-01 21:58:51 -07:00
Frank cde91c1a00 llama.cpp runtime arm: serve-script runtime switch, harness containerless support, master build 2026-10-01 21:56:30 -07:00
Frank 5df6ef3166 autoresearch: harness setup
Benchmark entrypoint: bash autoresearch.sh
Goal: Maximize frank-large decode throughput on the Strix Halo iGPU (gfx1151) via quantization >=q6/fp6, engine flags, spec-decode shape, or runtime swap — context limits unchanged, KV stays bf16.
2026-10-01 20:41:51 -07:00