6 Commits

Include renames

Author	SHA1	Message	Date
Frank	334b6af2ae	fix(frank-default-large): enable flash attention by pairing V cache to q8_0 ... Root cause of ~1 t/s long-context decode: llama.cpp build has GGML_CUDA_FA_ALL_QUANTS=OFF, and FA requires K type == V type. Mixed cache q8_0/f16 silently disabled FA, collapsing decode at 200k ctx. Set V to q8_0 (halves V memory too) and retune threads 12/1 for 16 Zen5 cores.	2026-08-12 02:09:05 -07:00
Frank	668083fe76	perf: frank-default-large ubatch 512 (compute graph scales with ubatch; 602100 ctx KV leaves ~16GB, ubatch 2048 OOM)	2026-08-11 17:19:57 -07:00
Frank	62cc2b850a	perf: frank-default-large batch capped at 2048/2048 — 3x200700 ctx KV ~50GB leaves no room for batch 12288 (OOM)	2026-08-11 17:11:04 -07:00
Frank	bf64c0d9c2	perf: frank-default-large batch 12288 / ubatch 4096	2026-08-11 16:36:39 -07:00
Frank	25e90bbbf7	deploy: frank-default-large final config — ROCm + draft-dflash, 3x200700 ctx (bench-fastest)	2026-08-11 16:27:15 -07:00
Frank	aa2f4bfe39	feat: frank-default-large → llama.cpp Qwen3.6-27B Q8_K_P + DFlash draft + ngram-mod (bench pending)	2026-08-11 13:47:37 -07:00