Frank 334b6af2ae fix(frank-default-large): enable flash attention by pairing V cache to q8_0 ...

Root cause of ~1 t/s long-context decode: llama.cpp build has
GGML_CUDA_FA_ALL_QUANTS=OFF, and FA requires K type == V type.
Mixed cache q8_0/f16 silently disabled FA, collapsing decode at
200k ctx. Set V to q8_0 (halves V memory too) and retune
threads 12/1 for 16 Zen5 cores.

2026-08-12 02:09:05 -07:00

..

bench-frank-large.sh

feat: frank-default-large → llama.cpp Qwen3.6-27B Q8_K_P + DFlash draft + ngram-mod (bench pending)

2026-08-11 13:47:37 -07:00

bench-pipeline-64k.sh

bench: 4k pipeline harness w/ early crash check; parametrized spec-type

2026-08-11 16:14:44 -07:00

deploy.sh

docs+deploy: Dankserver now llama.cpp frank-default-small (HauhauCS Q6_K_P + DFlash)

2026-08-11 01:18:29 -07:00

frank-default-large.env

fix(frank-default-large): enable flash attention by pairing V cache to q8_0

2026-08-12 02:09:05 -07:00

frank-default-large.service

feat: frank-default-large → llama.cpp Qwen3.6-27B Q8_K_P + DFlash draft + ngram-mod (bench pending)

2026-08-11 13:47:37 -07:00

frank-default-small.env

Lower dankserver compute threads 8→7 to leave CPU headroom

2026-08-11 13:36:36 -07:00

frank-default-small.service

feat: frank-default-small = HauhauCS Qwen3.6-35B-A3B uncensored Q6_K_P + DFlash on dankserver

2026-08-11 00:57:04 -07:00

llama-server.service

sync scripts from remotes: ROCmFPX llama-server.service (dankclaw), dankserver vLLM serve/validate; drop failed-attempt scripts

2026-08-10 19:48:46 -07:00

serve-ling3-vllm.sh

feat: raise Ling-3 GPU memory utilization to 0.95

2026-08-10 23:25:02 -07:00

serve-llama-frank-large.sh

fix(frank-default-large): enable flash attention by pairing V cache to q8_0

2026-08-12 02:09:05 -07:00

serve-llama-frank-small.sh

chore: frank-default-small — ngram-mod, drop DFlash draft

2026-08-11 02:29:32 -07:00

serve-vllm-qwen36-dflash.sh

sync scripts from remotes: ROCmFPX llama-server.service (dankclaw), dankserver vLLM serve/validate; drop failed-attempt scripts

2026-08-10 19:48:46 -07:00

validate-vllm.sh

sync scripts from remotes: ROCmFPX llama-server.service (dankclaw), dankserver vLLM serve/validate; drop failed-attempt scripts

2026-08-10 19:48:46 -07:00