docs: rebuild KB in OKF v0.1 with live-verified facts
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
---
|
||||
type: reference
|
||||
title: Performance Benchmarks
|
||||
description: Token throughput benchmarks for Ornith 9B and 35B models with GPU vs CPU inference.
|
||||
tags:
|
||||
- ornith
|
||||
- performance
|
||||
- benchmarks
|
||||
- unverified
|
||||
verified: 2026-08-28
|
||||
---
|
||||
|
||||
# Performance
|
||||
# Performance
|
||||
|
||||
## 9B Model:
|
||||
|
||||
- **Prompt Processing:** ~10-15 tokens/sec (CPU), ~50-100 tokens/sec (GPU)
|
||||
- **Generation:** ~8-12 tokens/sec (CPU), ~40-80 tokens/sec (GPU)
|
||||
|
||||
## 35B MoE Model:
|
||||
|
||||
- **Prompt Processing:** ~5-8 tokens/sec (CPU+GPU mix), ~30-50 tokens/sec (full GPU)
|
||||
- **Generation:** ~4-8 tokens/sec (CPU+GPU mix), ~20-40 tokens/sec (full GPU)
|
||||
|
||||
**Note:** MoE models can be slower due to expert routing overhead, but provide better quality.
|
||||
|
||||
## Tuning Tips
|
||||
|
||||
1. **Increase GPU layers** (`-ngl`) for better speed
|
||||
2. **Enable Flash Attention:** `--flash-attn on` (if supported)
|
||||
3. **Use a smaller quantization** (Q4_K_M instead of Q8_0)
|
||||
4. **Reduce context window size** if memory-bound
|
||||
5. **KV Cache Quantization:** `--cache-type-k q8_0` to reduce KV cache memory
|
||||
|
||||
## Related
|
||||
|
||||
* [GPU Offloading](/tools/ornith-llama-cpp-setup/gpu-offloading.md) — Optimize GPU layers for your hardware
|
||||
* [Troubleshooting](/tools/ornith-llama-cpp-setup/troubleshooting.md) — Fix performance issues
|
||||
Reference in New Issue
Block a user