docs: rebuild KB in OKF v0.1 with live-verified facts

Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
2026-08-28 07:57:13 -07:00
parent 552d1ef8bf
commit 02883d3ed2
116 changed files with 13008 additions and 0 deletions
@@ -0,0 +1,46 @@
+---
+type: reference
+title: GPU Layer Offloading Guidelines
+description: How many GPU layers to offload based on your VRAM size for both Ornith 9B and 35B models.
+tags:
+  - unverified
+  - ornith
+  - gpu
+  - offloading
+  - llama.cpp
+timestamp: 2026-07-18
+verified: 2026-08-28
+---
+
+# GPU Layer Offloading
+
+## For 9B Q4_K_M (5.63 GB):
+
+- **8GB VRAM:** `-ngl 35-40` (fits most layers)
+- **12GB VRAM:** `-ngl 99` (all layers)
+- **16GB+ VRAM:** `-ngl 99` (all layers + room for large KV cache)
+
+## For 35B Q4_K_M (~20 GB):
+
+- **8GB VRAM:** `-ngl 10-14` (partial offload)
+- **12GB VRAM:** `-ngl 18-22` (partial offload)
+- **16GB VRAM:** `-ngl 28-32` (partial offload)
+- **24GB VRAM:** `-ngl 40-48` (majority of layers)
+- **40GB+ VRAM:** `-ngl 99` (all layers)
+
+**Note:** The 35B model is a MoE architecture, so actual memory usage depends on which experts are activated.
+
+## Context Window Considerations
+
+- **9B model:** Supports up to 262,144 tokens
+- **35B model:** Supports up to 262,144 tokens
+- **KV Cache Memory:** Larger context windows require more VRAM/RAM for the key-value cache
+  - 131K context: ~8-16 GB depending on model and quantization
+  - 256K context: ~16-32 GB depending on model and quantization
+
+**Recommendation:** Start with 131K context if you have limited VRAM, increase to 256K if you have headroom.
+
+## Related
+
+* [Installation](/tools/ornith-llama-cpp-setup/installation.md) — How to set the -ngl flag
+* [Model Specs](/tools/ornith-llama-cpp-setup/model-specs.md) — Memory requirements