Frank
02883d3ed2
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
66 lines
1.9 KiB
Markdown
66 lines
1.9 KiB
Markdown
---
|
|
type: reference
|
|
title: Ornith Model Specifications
|
|
description: Specifications for Ornith-1.0-9B dense and Ornith-1.0-35B MoE models — context length, quantization options, memory requirements.
|
|
tags:
|
|
- ornith
|
|
- models
|
|
- llama.cpp
|
|
- GGUF
|
|
- unverified
|
|
verified: 2026-08-28
|
|
---
|
|
|
|
# Model Specifications
|
|
|
|
## Ornith-1.0-9B-GGUF
|
|
|
|
- **Source:** https://huggingface.co/ornith-ai/Ornith-1.0-9B-GGUF
|
|
- **Architecture:** Dense ~9B model
|
|
- **Context Length:** 262,144 tokens (256K)
|
|
- **Quantization Options:**
|
|
- Q4_K_M: 5.3 GB (recommended for most systems)
|
|
- Q5_K_M: 6.47 GB
|
|
- Q6_K: 7.36 GB
|
|
- Q8_0: 9.53 GB
|
|
- bf16: 17.9 GB
|
|
|
|
## Ornith-1.0-35B-GGUF (MoE)
|
|
|
|
- **Source:** https://huggingface.co/ornith-ai/Ornith-1.0-35B-GGUF
|
|
- **Architecture:** 35B MoE (Mixture of Experts)
|
|
- **Context Length:** 262,144 tokens (256K)
|
|
- **Quantization Options:**
|
|
- Q4_K_M: 20 GB (recommended)
|
|
- Other quantizations available
|
|
|
|
**Note:** No fp4 quantization exists for these models. Q4_K_M is the lightest quantization.
|
|
|
|
## Comparison: 9B vs 35B MoE
|
|
|
|
| Aspect | 9B Dense | 35B MoE |
|
|
|--------|----------|---------|
|
|
| **Memory (Q4)** | 5.3 GB | 20 GB |
|
|
| **VRAM for Full GPU** | 8 GB | 24-40 GB |
|
|
| **Speed (GPU)** | Faster | Slower but better quality |
|
|
| **Quality** | Good | Excellent |
|
|
| **Context** | 256K | 256K |
|
|
| **Best For** | Quick tasks, limited RAM | Complex reasoning, best quality |
|
|
|
|
## Requirements
|
|
|
|
### Hardware
|
|
- **RAM:** Minimum 16GB for 9B model, 32GB+ for 35B model
|
|
- **GPU** (optional but recommended):
|
|
- 9B Q4: 6-8GB VRAM for full GPU offload
|
|
- 35B Q4: 24GB+ VRAM for full GPU offload
|
|
|
|
### Software
|
|
- llama.cpp built with CUDA support (for GPU acceleration)
|
|
- Git (for cloning llama.cpp repo)
|
|
|
|
## Related
|
|
|
|
* [Installation](/tools/ornith-llama-cpp-setup/installation.md) — How to set up the server
|
|
* [GPU Offloading](/tools/ornith-llama-cpp-setup/gpu-offloading.md) — Layer allocation by VRAM
|