Tune small model expert placement

docs/LLM/dealign-ornith-crack.md

@@ -46,16 +46,16 @@ ordinary SSM blocks, 655.81 MiB for full-attention blocks, and 662.40 MiB for
 block 40. The expert portion is 630 MiB in every block.
 
 The service keeps all non-expert tensors on CUDA. It also keeps expert tensors
-from blocks 37 through 40 on CUDA. Those blocks sit next to the output and
-include the bundled MTP head. Expert tensors from blocks 0 through 36 move to
+from blocks 36 through 40 on CUDA. Those blocks sit next to the output and
+include the bundled MTP head. Expert tensors from blocks 0 through 35 move to
 CPU through this explicit override:
 
 ```
-OVERRIDE_EXPERT='blk\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35|36)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
+OVERRIDE_EXPERT='blk\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
 OVERRIDE_BUFT=CPU
 ```
 
-This places 3.08 GiB of expert weights on CUDA and 22.76 GiB on CPU. The
+This places 3.69 GiB of expert weights on CUDA and 22.13 GiB on CPU. The
 selection leaves the output-adjacent blocks and MTP path on CUDA while the
 f16 context cache uses the remaining GPU budget.
 
@@ -87,7 +87,7 @@ llama-server \
   --mmproj mmproj-Ornith-1.5-35B-A3B-f16.gguf \
   --chat-template-file chat-template.jinja \
   --spec-type draft-mtp \
-  --override-tensor "blk\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35|36)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
+  --override-tensor "blk\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
   -ngl all -c 589824 -np 3 \
   --cache-type-k f16 --cache-type-v f16 --no-mmap \
   --temp 1.0 --top-p 0.95 --top-k 20
@@ -100,7 +100,7 @@ llama-server \
 2. The context pool contains 589824 cells. Three slots receive 196608 cells
    each, which is 64K below the model context per stream.
 3. `--n-cpu-moe` remains supported. This service prefers the explicit tensor
-   override because it preserves the final four expert blocks on CUDA.
+   override because it preserves the final five expert blocks on CUDA.
 4. The service uses the Qwen-Sharp template from
    `peculiar-ragdoll/Qwen-Sharp-Chat-Templates` through
    `scripts/chat-template.jinja`.