10 lines
1.1 KiB
Markdown
10 lines
1.1 KiB
Markdown
## LLM Runtime KV Cache (frank-archon)
|
||
|
||
**KV cache is unquantized (fp8 native)** — radiance engine 1.0.13 on frank-primary (frank-archon) runs with fp8 KV cache pool of 3.46 GiB. A prior quantized state was reversed; the unquantized configuration is the current production state.
|
||
|
||
**Per-slot context ceiling** — `frank-archon` and `frank-dragoon` share a 262,144 position per-slot ceiling (`--ctx-size 401408 --parallel 2`), distinct from the 27B-era 200,192 limit. The harness clamps `max_tokens` to `min(45056, contextWindow − est − 4096)` and the 400-zone law under-counts; the 400 constant is the floor.
|
||
|
||
**MTP head placement** — `frank-dragoon` halogen-flash-server 0.16.2 embeds the MTP head inside the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched). `frank-zealot` llama.cpp has no MTP tensors in the Q6_K GGUF file.
|
||
|
||
**Per-slot vs pool context** — `frank-zealot` exposes `200704` as the per-slot window (configured `--ctx-size 401408 --parallel 2`), not the 262,144 native model size or the 409,600 aggregate pool. The harness must compact before llama.cpp accepts longer prompts.
|