serverAppConfigs/docs/LLM/dealign-ornith-crack.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	dealign Ornith-1.5-35B-A3B CRACK on Dankserver	dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q8_0) served by upstream llama.cpp on the RTX 3090. MoE model with the MTP head bundled; expert weights offloaded to CPU with an explicit --override-tensor. frank-default-small.	llamacpp cuda dankserver moe ornith mtp	2026-08-24

dealign Ornith-1.5-35B-A3B CRACK on Dankserver

Active 2026-08-24. frank-default-small serves this model. It replaced the HauhauCS Qwen3.6-35B-A3B Q6_K_P service on the same upstream llama.cpp binary (b10121).

frank-default-small (port 7770) serves Ornith-1.5-35B-A3B-CRACK-Q8_0.gguf from dealignai on Hugging Face. It is a MoE model (256 experts, 8 active) with a hybrid GatedDeltaNet (SSM) + attention architecture, abliterated (uncensored). The MTP speculative head is bundled in the GGUF, so draft-mtp auto-detects it from the target model.

Model Files

File	Size
Ornith-1.5-35B-A3B-CRACK-Q8_0.gguf	37.2 GB
mmproj-Ornith-1.5-35B-A3B-f16.gguf (vision)	0.90 GB

Store the files under /home/dank/models/Ornith-1.5-35B-A3B-CRACK/.

Expert Offload — --override-tensor, not --n-cpu-moe

The service pins the MoE expert weights to CPU with an explicit tensor override rather than the layer-counter --n-cpu-moe. The override uses the same internal regex that --cpu-moe / --n-cpu-moe expand:

OVERRIDE_EXPERT=\.ffn_(up|down|gate|gate_up)_(ch|)exps
OVERRIDE_BUFT=CPU

This is passed as --override-tensor "$OVERRIDE_EXPERT=$OVERRIDE_BUFT". Only the FFN expert tensors move to CPU; every other tensor stays on the 24 GB card up to --gpu-layers. To list the buffer type names on the host, run a bogus override and read the error:

llama-server --model <ANY> -c 64 -ot foo=BADTYPE

On Dankserver the available buffer types are CPU and CUDA0.

Speculative Decoding

MoE model with a bundled MTP head -> native draft-mtp. The head is auto-detected from the target (b10121 creates the MTP draft context against the model file). ngram under MTP hurts on this family; MTP alone is the proven best. --spec-draft-n-max 4 is the knee.

--spec-type draft-mtp
--spec-draft-n-max 4

Launch Excerpt

llama-server \
  -m Ornith-1.5-35B-A3B-CRACK-Q8_0.gguf \
  --mmproj mmproj-Ornith-1.5-35B-A3B-f16.gguf \
  --chat-template-file chat-template.jinja \
  --spec-type draft-mtp \
  --override-tensor "\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
  -ngl all -c 688128 -np 3 \
  --cache-type-k f16 --cache-type-v f16 --no-mmap \
  --temp 1.0 --top-p 0.95 --top-k 20

Known Issues

1. The 37.2 GB Q8_0 file is large for the 62 GB RAM box; lower PARALLEL or switch quant if VRAM or RAM is tight.
2. --n-cpu-moe remains supported; this service deliberately prefers the explicit tensor override so only the expert tensors reach CPU.

See Also

• deployment — deploy and update process
• Dankserver System Specs — the box
• frank-redo-2026-08-17 — fleet topology history