## System Topology & Hardware - **dankclaw**: AMD Strix Halo APU (gfx1151) + RTX 3090 in M.2 dock (PCIe x4, ~29 GB/s sequential). Combined GPU budget ~88 GiB. OS: Ubuntu 24.04. - **dankserver**: ASUS TUF GAMING B650-E WIFI. Dual AMD R9700 GPUs. - **RCCL Constraint**: Chipset-attached PCIe lanes fail RCCL atomics (broadcast/all-reduce/send-recv). Fix: Move card #2 to CPU-attached M.2_2 slot via PCIe x4 riser to restore atomic-op routing. ## LLM Model Portfolio - **frank-large (halogen)**: Qwen3.8-27B-ROCmFPX-Q8_0. Runs on Strix Halo (gfx1151). - *KV Pool*: 524288 positions (~14.4 GiB). Admission is pool-bound, not slot-bound (caps at 3 simultaneous full-length requests). - *Behavior*: Defaults to greedy. Stalls (full-CPU, no disk activity, no output) under co-tenancy with pinned trunk are documented behavior, not crashes. - *Constraints*: maxTokens <= 65536. First request after restart may be silent ~45-60s while first-touching KV pool. - **frank-small**: Qwen3.8-27B GSQ-RCO IQ3_S. Runs on RTX 3090 (M.2 dock). Defaults to sampling (temp 1.0, top_k 20, top_p 0.95). - **ik_llamacpp**: mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-GGUF (i1-Q4_K_M). - *Backend*: CUDA (3090) + Vulkan (Strix Halo) split. - *Performance*: ~3.8-4.0 t/s decode, ~192 t/s prefill. - *Bottleneck*: PCIe x4 launch/sync overhead; CUDA graphs currently abort on hc ops (ggml_repeat_4d). ## Software Infrastructure - **Podman**: Rootless deployment using Quadlet units for boot persistence. Note: `restart: unless-stopped` survives crashes, but not explicit stops (kernel power-off). - **Authentik**: Anubis guard chain. Marreta app connected via ProxyProvider (fixed null flow crash). - **AGENTS.md**: Structured as (1) Active-topology table, (2) Runbook pointers, (3) Key Lessons Learned. ## Durable Lessons - **ROCm**: Use layer-split-not-tensor; mmap/PLE constraints apply; HIP env collisions (`TEMP`/`HOST`). - **Memory**: `free` command lies about available memory (doesn't account for pinned/reserved weights). - **PCIe**: M.2 x4 dock is ~29 GB/s; do not assume x16 bandwidth. - **Skill Discovery**: `omp` scans `~/.omp/agent/skills/` only. New skills surface on the next session.