Files

T

Frank 02883d3ed2 docs: rebuild KB in OKF v0.1 with live-verified facts

Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center
knowledge base: concept files promoted to path identity, index listings
stripped of frontmatter, all wikilinks/relative links resolved, every
checkable claim verified against the live fleet (Dankserver = Ryzen 7
7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged.
kb-lint exits 0.

2026-08-28 07:57:13 -07:00

2.4 KiB

Raw Blame History

type, title, description, tags, verified

type

title

description

Installation

1. Clone and Build llama.cpp

cd ~/GitPackages
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
cmake -B build -DLLAMA_CUDA=ON
cmake --build build --config Release

2. Download the Model

mkdir -p ~/models/ornith
cd ~/models/ornith

# For 9B model (faster, less memory)
wget https://huggingface.co/ornith-ai/Ornith-1.0-9B-GGUF/resolve/main/ornith-1.0-9b-Q4_K_M.gguf

# OR for 35B MoE model (better performance, more memory)
wget https://huggingface.co/ornith-ai/Ornith-1.0-35B-GGUF/resolve/main/ornith-1.0-35b-Q4_K_M.gguf

3. Start the Server

For 9B model with full GPU offload (8GB+ VRAM):

cd ~/GitPackages/llama.cpp
./build/bin/llama-server \
  -m ~/models/ornith/ornith-1.0-9b-Q4_K_M.gguf \
  -c 262144 \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

For 35B MoE model with partial GPU offload (8GB VRAM example):

cd ~/GitPackages/llama.cpp
./build/bin/llama-server \
  -m ~/models/ornith/ornith-1.0-35b-Q4_K_M.gguf \
  -c 131072 \
  -ngl 12 \
  --host 0.0.0.0 \
  --port 8080

Key Parameters:

-m: Path to GGUF model file
-c / --ctx-size: Context window size (131072 or 262144)
-ngl / --n-gpu-layers: Number of layers to offload to GPU
- 99 = all layers
- Lower values = more CPU inference
--host: Bind address (0.0.0.0 for all interfaces, 127.0.0.1 for local only)
--port: Server port (default 8080)

4. Verify Server is Running

curl http://localhost:8080/health
# Expected: {"status":"ok"}

5. Test the Model

curl -s http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "frank-default-local",
    "messages": [
      {"role": "user", "content": "Hello, what is 2+2?"}
    ],
    "max_tokens": 150,
    "temperature": 0.7
  }' | jq .

Model Specs — Model options and quantization
GPU Offloading — Tuning GPU layer counts
Pi Configuration — Adding to models.json

2.4 KiB Raw Blame History