Frank
02883d3ed2
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
72 lines
1.6 KiB
Markdown
72 lines
1.6 KiB
Markdown
---
|
|
type: howto
|
|
title: Troubleshooting Ornith + llama.cpp
|
|
description: Common errors — CUDA OOM, segfaults, model not responding, poor performance — and their fixes.
|
|
tags:
|
|
- unverified
|
|
- ornith
|
|
- troubleshooting
|
|
- llama.cpp
|
|
timestamp: 2026-07-18
|
|
verified: 2026-08-28
|
|
---
|
|
|
|
# Troubleshooting
|
|
|
|
## CUDA Out of Memory
|
|
|
|
```
|
|
Error: cudaMalloc failed: out of memory
|
|
```
|
|
|
|
**Solution:** Reduce `-ngl` value to offload fewer layers to GPU.
|
|
|
|
## Segmentation Fault
|
|
|
|
```
|
|
Segmentation fault (core dumped)
|
|
```
|
|
|
|
**Solution:**
|
|
1. Reduce context size (`-c` parameter)
|
|
2. Reduce GPU layers (`-ngl`)
|
|
3. Check if model file is corrupted
|
|
|
|
## Model Not Responding
|
|
|
|
```
|
|
curl: (7) Failed to connect to localhost port 8080
|
|
```
|
|
|
|
**Solution:** Check if server is running:
|
|
```bash
|
|
ps aux | grep llama-server
|
|
tail -f ~/logs/llama-server.log
|
|
```
|
|
|
|
## Poor Performance
|
|
|
|
**Solutions:**
|
|
1. Increase GPU layers (`-ngl`)
|
|
2. Enable Flash Attention: `--flash-attn on` (if supported)
|
|
3. Use a smaller quantization (Q4_K_M instead of Q8_0)
|
|
4. Reduce context window size
|
|
|
|
## Security Considerations
|
|
|
|
- **Local Server:** By default, binds to localhost (127.0.0.1)
|
|
- **Network Exposure:** If binding to 0.0.0.0, consider:
|
|
- Adding authentication (`--api-key`)
|
|
- Using a firewall to restrict access
|
|
- Running behind a reverse proxy with TLS
|
|
|
|
```bash
|
|
# Add API key authentication
|
|
./build/bin/llama-server ... --api-key your-secret-key
|
|
```
|
|
|
|
## Related
|
|
|
|
* [Installation](/tools/ornith-llama-cpp-setup/installation.md) — The commands that might be causing issues
|
|
* [GPU Offloading](/tools/ornith-llama-cpp-setup/gpu-offloading.md) — GPU configuration
|