Select hybrid expert placement from benchmark

docs/LLM/Dankserver System Specs.md

@@ -60,12 +60,12 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
 
 ### Active Services
 
-The active service on Dankserver is `frank-default-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q6_K) on port 7770, with expert tensors from blocks 0 through 32 held on CPU via `--override-tensor`. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
+The active service on Dankserver is `frank-default-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q6_K) on port 7770, with expert tensors selected by `--override-tensor`; full-attention blocks 27 and 31 plus output-adjacent blocks 33–40 remain on CUDA. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
 
 ## Hardware Relevance
 
-- 24 GB VRAM hosts the Q6_K non-expert tensors and expert tensors from blocks 33 through 40. The provisional context pool contains 491520 f16 cells across three 163840-token slots. The configuration will be rebalanced after measuring the newly available headroom. Verify the actual headroom after each model restart with `nvidia-smi` and the service journal.
-- 62 GB system RAM holds approximately 19.67 GiB of CPU expert tensors with room to spare.
+- 24 GB VRAM hosts the Q6_K non-expert tensors and the measured hybrid expert placement. The context pool contains 491520 f16 cells across three 163840-token slots. Verify actual headroom after each model restart with `nvidia-smi` and the service journal.
+- 62 GB system RAM holds the remaining CPU-resident expert tensors with room to spare.
 - 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
 - Root disk has approximately 140 GB free. Store the GGUF models under `/home/dank/models` (about 32 GB total).
 

docs/LLM/dealign-ornith-crack.md

@@ -45,21 +45,23 @@ The non-expert tensors total 2.02 GiB. The layer totals are 660.96 MiB for
 ordinary SSM blocks, 655.81 MiB for full-attention blocks, and 662.40 MiB for
 block 40. The expert portion is 630 MiB in every block.
 
-The provisional service keeps all non-expert tensors on CUDA. It also keeps
-expert tensors from blocks 33 through 40 on CUDA. Those blocks sit next to the
-output and include the bundled MTP head. Expert tensors from blocks 0 through
-32 move to CPU through this explicit override:
+The measured hybrid configuration keeps all non-expert tensors on CUDA. It
+also keeps expert tensors from full-attention blocks 27 and 31, plus the
+output-adjacent blocks 33 through 40, on CUDA. Block 40 includes the bundled
+MTP head. Expert tensors from all other blocks move to CPU through this
+explicit override:
 
 ```
-OVERRIDE_EXPERT='blk\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
+OVERRIDE_EXPERT='blk\.(0|1|2|3|4|5|6|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|28|29|30|32)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
 OVERRIDE_BUFT=CPU
 ```
 
-The previous five-block configuration placed 3.69 GiB of expert weights on
-CUDA and 22.13 GiB on CPU. This provisional eight-block configuration adds
-approximately 1.89 GiB of expert weights to CUDA while reducing the context
-pool by 98304 cells. The final placement will be selected after measuring this
-new headroom and discussing layer priorities.
+The measured benchmark compared three policies with a 16K-token prompt:
+full-attention blocks plus block 40, contiguous top blocks 33–40, and this
+hybrid. The hybrid was fastest at approximately 33.6 generated tokens per
+second, versus 33.1 for the contiguous-top policy and 30.3 for the
+full-attention-only policy. Its first-request prompt throughput was about
+2711 tokens per second.
 
 To list the buffer type names on the host, run a bogus override:
 
@@ -89,7 +91,7 @@ llama-server \
   --mmproj mmproj-Ornith-1.5-35B-A3B-f16.gguf \
   --chat-template-file chat-template.jinja \
   --spec-type draft-mtp \
-  --override-tensor "blk\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
+  --override-tensor "blk\\.(0|1|2|3|4|5|6|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|28|29|30|32)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
   -ngl all -c 491520 -np 3 \
   --cache-type-k f16 --cache-type-v f16 --no-mmap \
   --temp 1.0 --top-p 0.95 --top-k 20
@@ -102,8 +104,9 @@ llama-server \
 2. The provisional context pool contains 491520 cells. Three slots receive
    163840 cells each, which is 96K below the model context per stream.
 3. `--n-cpu-moe` remains supported. This service currently prefers the
-   explicit tensor override because it preserves the final eight expert
-   blocks on CUDA pending the next layer-priority decision.
+   explicit tensor override because the 16K benchmark found this hybrid
+   placement fastest. It preserves the final eight blocks plus full-attention
+   blocks 27 and 31 on CUDA.
 4. The service uses the Qwen-Sharp template from
    `peculiar-ragdoll/Qwen-Sharp-Chat-Templates` through
    `scripts/chat-template.jinja`.

docs/LLM/deployment.md

@@ -124,10 +124,11 @@ curl -sf http://127.0.0.1:7770/health
 The active service on Dankserver is upstream llama.cpp (b10121) serving
 `Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf` under the alias `frank-default-small`
 on port 7770. It runs the stock `/home/dank/llama.cpp/build/bin/llama-server`
-binary with expert tensors from blocks 0 through 32 pinned to CPU by an
-explicit `--override-tensor` regex. Expert tensors from blocks 33 through 40
-stay on CUDA. The service uses 7 CPU threads, native `draft-mtp`
-speculative decoding, f16 KV, context 491520 (3 slots of 163840 each), batch
+binary with expert tensors selected by an explicit `--override-tensor`
+regex. Expert tensors from full-attention blocks 27 and 31 and the
+output-adjacent blocks 33 through 40 stay on CUDA; other expert tensors are
+CPU-resident. The service uses 7 CPU threads, native `draft-mtp` speculative
+decoding, f16 KV, context 491520 (3 slots of 163840 each), batch
 4096, ubatch 2048, and `--no-mmap`. The unit is
 `scripts/frank-default-small.service`. It launches through
 `scripts/serve-frank-small.sh`, which reads tunables from