docs: rebuild KB in OKF v0.1 with live-verified facts
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
This commit is contained in:
1 parent
552d1ef8bf
commit
02883d3ed2
116 files changed
+13008
No files matched your search
@@ -0,0 +1,97 @@
|
||||
---
|
||||
type: howto
|
||||
title: Installing Ornith + llama.cpp
|
||||
description: Step-by-step setup — clone and build llama.cpp, download models, start the server, verify it works.
|
||||
tags:
|
||||
- ornith
|
||||
- installation
|
||||
- llama.cpp
|
||||
- unverified
|
||||
verified: 2026-08-28
|
||||
---
|
||||
|
||||
# Installation
|
||||
|
||||
## 1. Clone and Build llama.cpp
|
||||
|
||||
```bash
|
||||
cd ~/GitPackages
|
||||
git clone https://github.com/ggerganov/llama.cpp
|
||||
cd llama.cpp
|
||||
cmake -B build -DLLAMA_CUDA=ON
|
||||
cmake --build build --config Release
|
||||
```
|
||||
|
||||
## 2. Download the Model
|
||||
|
||||
```bash
|
||||
mkdir -p ~/models/ornith
|
||||
cd ~/models/ornith
|
||||
|
||||
# For 9B model (faster, less memory)
|
||||
wget https://huggingface.co/ornith-ai/Ornith-1.0-9B-GGUF/resolve/main/ornith-1.0-9b-Q4_K_M.gguf
|
||||
|
||||
# OR for 35B MoE model (better performance, more memory)
|
||||
wget https://huggingface.co/ornith-ai/Ornith-1.0-35B-GGUF/resolve/main/ornith-1.0-35b-Q4_K_M.gguf
|
||||
```
|
||||
|
||||
## 3. Start the Server
|
||||
|
||||
**For 9B model with full GPU offload (8GB+ VRAM):**
|
||||
```bash
|
||||
cd ~/GitPackages/llama.cpp
|
||||
./build/bin/llama-server \
|
||||
-m ~/models/ornith/ornith-1.0-9b-Q4_K_M.gguf \
|
||||
-c 262144 \
|
||||
-ngl 99 \
|
||||
--host 0.0.0.0 \
|
||||
--port 8080
|
||||
```
|
||||
|
||||
**For 35B MoE model with partial GPU offload (8GB VRAM example):**
|
||||
```bash
|
||||
cd ~/GitPackages/llama.cpp
|
||||
./build/bin/llama-server \
|
||||
-m ~/models/ornith/ornith-1.0-35b-Q4_K_M.gguf \
|
||||
-c 131072 \
|
||||
-ngl 12 \
|
||||
--host 0.0.0.0 \
|
||||
--port 8080
|
||||
```
|
||||
|
||||
**Key Parameters:**
|
||||
- `-m`: Path to GGUF model file
|
||||
- `-c` / `--ctx-size`: Context window size (131072 or 262144)
|
||||
- `-ngl` / `--n-gpu-layers`: Number of layers to offload to GPU
|
||||
- 99 = all layers
|
||||
- Lower values = more CPU inference
|
||||
- `--host`: Bind address (0.0.0.0 for all interfaces, 127.0.0.1 for local only)
|
||||
- `--port`: Server port (default 8080)
|
||||
|
||||
## 4. Verify Server is Running
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/health
|
||||
# Expected: {"status":"ok"}
|
||||
```
|
||||
|
||||
## 5. Test the Model
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "frank-default-local",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, what is 2+2?"}
|
||||
],
|
||||
"max_tokens": 150,
|
||||
"temperature": 0.7
|
||||
}' | jq .
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
* [Model Specs](/tools/ornith-llama-cpp-setup/model-specs.md) — Model options and quantization
|
||||
* [GPU Offloading](/tools/ornith-llama-cpp-setup/gpu-offloading.md) — Tuning GPU layer counts
|
||||
* [Pi Configuration](/tools/ornith-llama-cpp-setup/pi-configuration.md) — Adding to models.json
|
||||
Reference in new issue
Block a user