docs(kv): add bit-density KV cache quant table across 3 runtimes

- Place ik's Trellis (IQ*_KT), IQK (IQ*_K), MXFP4, and dedicated fast KV type
  Q8_KV in a single high->low bits-per-value table with ROCmFPX TurboQuant KV.
- Mark mainline-provided types (*), KV-native vs weight-family-reused-for-KV.
- Hadamard annotated as a transform on top of a base quant, not a row.
- bpw read from exact ggml block structs (QK_K=256, fp16 scale): type_size x 8 / QK,
  verified against the struct comment "6.5625 bits per weight" and the trait table.
  Notes where legacy nominal figures (4.0/4.25) diverge.
- AGENTS.md: add benchmark directive (use llama-bench / vllm bench; ask before
  rolling your own).
This commit is contained in:
Frank committed 2026-08-25 20:03:13 -07:00
1 parent 4308f62004
commit 2277a2e9ac
2 files changed
+93 -1

No files matched your search

+3
View File
@@ -174,6 +174,9 @@ These servers serve real users (about 3 people). Running a bench while the
service is up starves the GPU and corrupts results. Restart the service when
the bench finishes.
ALWAYS bench using llama-bench for llama.cpp or vllm bench for vllm runtimes.
DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE OFFICIAL BENCHES. ASK THE USER BEFORE DEVIATING FROM THIS DIRECTIVE.
1. Stop the service before any benchmark:
`sudo systemctl stop frank-default-small` (Dankserver) or
`sudo systemctl stop frank-default-large` (Dankclaw).