Frank
c3f30bdbd0
Purge live-state claims that ~/.pi is our harness; keep general pi documentation in omp-docs/ and dated historical records. - README.md: single config source; Legacy Notes reworded - AGENTS.md: drop 'like ~/.pi did' framing - APPEND_SYSTEM.md: okf-spec -> omp-docs/references/okf-spec.md - dotfiles/zsh/.zshrc: systemprompt alias -> APPEND_SYSTEM.md - managed skill llm-fleet-model-swap + model_config_sync skill: single-source models.yml; validation script YAML-only - memories (MEMORY.md, memory_summary.md, learned.md): single source - omp-docs: .okf root -> ~/.omp/omp-docs; de-live pi-era claims in pi/index.md, pi-web-access.md, engineering/agents-files.md
2.2 KiB
2.2 KiB
System Topology & Hardware
- dankclaw: AMD Strix Halo APU (gfx1151) + RTX 3090 in M.2 dock (PCIe x4, ~29 GB/s sequential). Combined GPU budget ~88 GiB. OS: Ubuntu 24.04.
- dankserver: ASUS TUF GAMING B650-E WIFI. Dual AMD R9700 GPUs.
- RCCL Constraint: Chipset-attached PCIe lanes fail RCCL atomics (broadcast/all-reduce/send-recv). Fix: Move card #2 to CPU-attached M.2_2 slot via PCIe x4 riser to restore atomic-op routing.
LLM Model Portfolio
- frank-large (halogen): Qwen3.8-27B-ROCmFPX-Q8_0. Runs on Strix Halo (gfx1151).
- KV Pool: 524288 positions (~14.4 GiB). Admission is pool-bound, not slot-bound (caps at 3 simultaneous full-length requests).
- Behavior: Defaults to greedy. Stalls (full-CPU, no disk activity, no output) under co-tenancy with pinned trunk are documented behavior, not crashes.
- Constraints: maxTokens <= 65536. First request after restart may be silent ~45-60s while first-touching KV pool.
- frank-small: Qwen3.8-27B GSQ-RCO IQ3_S. Runs on RTX 3090 (M.2 dock). Defaults to sampling (temp 1.0, top_k 20, top_p 0.95).
- ik_llamacpp: mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF (i1-Q4_K_M).
- Backend: CUDA (3090) + Vulkan (Strix Halo) split.
- Performance: ~3.8-4.0 t/s decode, ~192 t/s prefill.
- Bottleneck: PCIe x4 launch/sync overhead; CUDA graphs currently abort on hc ops (ggml_repeat_4d).
Software Infrastructure
- Podman: Rootless deployment using Quadlet units for boot persistence. Note:
restart: unless-stoppedsurvives crashes, but not explicit stops (kernel power-off). - Authentik: Anubis guard chain. Marreta app connected via ProxyProvider (fixed null flow crash).
- AGENTS.md: Structured as (1) Active-topology table, (2) Runbook pointers, (3) Key Lessons Learned.
Durable Lessons
- ROCm: Use layer-split-not-tensor; mmap/PLE constraints apply; HIP env collisions (
TEMP/HOST). - Memory:
freecommand lies about available memory (doesn't account for pinned/reserved weights). - PCIe: M.2 x4 dock is ~29 GB/s; do not assume x16 bandwidth.
- Skill Discovery:
ompscans~/.omp/agent/skills/only. New skills surface on the next session.