Frank
5b08896e4e
--home-blank-.omp--: MEMORY.md +140 / learned.md / memory_summary.md / raw_memories.md +1002 rewritten by auto-learn after the model rename, frank-medium split-balance and bottleneck studies. Skills renamed to single-topic buckets: dotfile-git-workflow -> ghostty-text-aa + niri-display-rotation + noctalia-theme, handy-service-mgmt -> handy-flm-inference, inference-runtime -> llama-cpp-parallelism. --home-blank-Projects-serverAppConfig--: 38 new rollout summaries, raw_memories.md/MEMORY.md refreshed, skills re-bucketed (bench-protocol, llm-fleet-model-swap, rootless-quadlet-log-retrieval, orchestration-quadlet). Per-worktree bucket (wt-20261006-122204) is now gitignored, not committed.
109 lines
5.8 KiB
Markdown
109 lines
5.8 KiB
Markdown
# Frank Fleet Memory Consolidation
|
|
|
|
## 1. Fleet Topology & Identity
|
|
|
|
### Hosts
|
|
- **dankclaw** (192.168.0.143): Strix Halo APU (gfx1151, ~114 GiB GTT) + M.2 docked RTX 3090 (24 GiB, PCIe x4). Hosts `frank-large` (Halogen) and `frank-small` (Llama.cpp).
|
|
- **dankserver** (192.168.0.235): ASUS B650-E, 2x AMD R9700 (gfx1201, 32 GiB each), 4x DIMMs DDR5. Hosts `frank-primary` (VLLM/Radiance) and `frank-medium` (Llama.cpp).
|
|
- **frankbook-pro** (192.168.0.112): M-series Mac mini (~30 GiB RAM). Role unclear/secondary.
|
|
|
|
### Services (Live as of 2026-10-05)
|
|
- **frank-large** (:7770): Halogen-flash-server v2, Qwen3.8-27B Uncensored (Q6_0/Q8_0/ROCmFP4), MTP/DFlash2 drafter, HALOGEN_CTX=262144, KV pool 524288 pos.
|
|
- **frank-small** (:7771): Llama.cpp, MiMo-V2.6-9B (Q6_K/Q5_K_M) or Qwen3.8-9B (IQ3_S), 2x200704 f16-KV.
|
|
- **frank-primary** (:7770): tcclaviger VLLM fork, Qwen3.8-Flash-Next FP8, TP=2, SEQS=3, MNBT=2048, MAXLEN=200704, Arena ~17.77 GiB.
|
|
- **frank-medium** (:7770): Llama.cpp, Qwen3.8-27B UD-Q8_K_XL, Tensor Split 35:30, CACHE_RAM 16384 MiB.
|
|
- **Authentik**: `ridley` shape template, Django ORM creation, Group-based ACLs.
|
|
- **ComfyUI**: Strix Halo iGPU, FP8/Q6 quantization, Ghostty image protocol.
|
|
|
|
## 2. Hardware & Memory Constraints
|
|
|
|
### RAM Stability (dankserver)
|
|
- **Critical**: 4x DIMMs at 4800 MT/s caused silent corruption (NaN/garbage) at ~40k tokens in `frank-primary`. **Fix**: Drop to 4600 MT/s.
|
|
- **Lesson**: Consumer non-ECC RAM raises no MCE for silent flips. Short in-band checks miss intermittent corruption.
|
|
- **Shape E**: 3.15e9 B KV pool, 17.77 GiB Arena, UTIL 0.982.
|
|
|
|
### GPU & PCIe
|
|
- **dankclaw 3090**: PCIe x4 dock (~29 GB/s). Decode bottlenecked by graph splits + PCIe sync. 30 t/s unreachable without mainline PR #28901.
|
|
- **dankserver R9700**: Chipset-attached PCIe (Gen4 x1) fails RCCL atomics. **Fix**: M.2→PCIe x4 riser for 2nd card.
|
|
- **Strix Halo**: 64 GiB Unified, RADV Vulkan. HALOGEN_CTX must be 262144 (native Qwen3.8-Flash-Next).
|
|
|
|
### KV & Context
|
|
- **frank-large**: `maxTokens` 57344 (harness) + HALOGEN_CTX 262144. 400 error if `prompt + max > ctx`.
|
|
- **frank-medium**: `CACHE_RAM` 16384 MiB required for 196608 ctx DeltaNet state. 8192 MiB caused refusals.
|
|
- **frank-small**: 2x200704 f16-KV slots.
|
|
|
|
## 3. Runtime Environments
|
|
|
|
### Halogen (dankclaw)
|
|
- **Container**: `halogen-flash-server v2` (Swift1.5 Abliterated).
|
|
- **Env**: `HALOGEN_CTX=262144`, `HALOGEN_KV_POOL_POSITIONS=524288`.
|
|
- **Pitfalls**: `HALOGEN_FLASH_PIN_TRUNK=1` causes multi-minute CPU stalls under co-tenancy. `HALOGEN_FLASH_PIN_TRUNK=0` as last resort.
|
|
- **Boot**: First request may be silent ~45-60s (first-touch KV pool).
|
|
|
|
### Llama.cpp (dankclaw/dankserver)
|
|
- **Forks**: `ik_llama.cpp` (CUDA+Vulkan), `rocmfpx` (ROCmFP4).
|
|
- **Flags**: `--cache-ram 16384` (medium), `--expert-precise-memory` (primary), `--max-model-len 200704`.
|
|
- **MTP**: Qwen3.8-Flash-Next MTP head needs full per-layer loading (Flash-Next specific).
|
|
|
|
### VLLM (dankserver)
|
|
- **Fork**: tcclaviger RDNA4 vLLM.
|
|
- **Seqs**: `SEQS=3` mandatory (SEQS=2 fails `torch.fx` ConstraintViolationError).
|
|
- **MTP**: Speculative decoding disabled/optional.
|
|
|
|
## 4. Operational Hygiene
|
|
|
|
### Bench Protocol
|
|
1. `sudo systemctl stop <service>`
|
|
2. Run benchmark
|
|
3. `sudo systemctl start <service>`
|
|
4. `curl -s <port>/health` → 200
|
|
5. **Trap**: `trap EXIT 'systemctl start <service>'; curl -s <port>/health` for self-restoring harnesses.
|
|
|
|
### Quadlet Persistence
|
|
- **Rootless Podman**: `restart: unless-stopped` only survives crashes. Bare containers vanish on reboot.
|
|
- **Pattern**: Git-tracked `.container` files in `~/.config/containers/systemd/`.
|
|
- **Historical**: Caddy lost after reboot (Aug 26); 5 observability services lost (Sep 11).
|
|
|
|
### Git & Deploy
|
|
- **Repo**: `/home/blank/Projects/serverAppConfig`.
|
|
- **Gate**: `deploy.sh` rejects dirty trees (untracked/modified files).
|
|
- **Pattern**: `git stash` → deploy → `git stash pop`.
|
|
|
|
## 5. Troubleshooting Knowledge Base
|
|
|
|
### max_tokens 400 Overflow
|
|
- **Symptom**: `prompt is X tokens and the context is Y, leaving room for Z`. `X+max > Y`.
|
|
- **Fix**: Adjust `maxTokens` in `models.yml` (e.g., 57344) or `HALOGEN_CTX` (262144).
|
|
- **Note**: OMP snapshots at process start; existing sessions need restart (`omp -r`).
|
|
|
|
### Orphaned `llama-server` (frank-medium)
|
|
- **Symptom**: Unit stopped, GPUs busy.
|
|
- **Diag**: `ps -eo pid,ppid,args | grep llama`, `rocm-smi --showmeminfo`.
|
|
- **Fix**: `kill <pid>`, `systemctl start`, verify geometry.
|
|
|
|
### Context Bloat
|
|
- **Source**: 175 managed-skills descriptions (~38k nonMessageTokens).
|
|
- **Fix**: Distill to vault reference pages, move dirs to `~/.AGENTTRASH/`, purge on approval.
|
|
|
|
### Authentik User Creation
|
|
- **Method**: `ak shell` on `dankclaw`, Django ORM (`User.objects.create`...), `ak shell` sets password, assign groups (`Non-Admin Users`, `jellyfin_users`).
|
|
- **ACLs**: Group membership is sole gate.
|
|
|
|
### ComfyUI (iGPU)
|
|
- **Target**: Strix Halo iGPU (3090 occupied).
|
|
- **Quant**: FP8 or Q6.
|
|
- **Display**: Ghostty `\e_G` base64 PNG, or `chafa` helper.
|
|
|
|
## 6. Skills & Playbooks
|
|
|
|
- **llm-fleet-model-swap**: Check HF model card, download sha256, commit → push → deploy.sh → health.
|
|
- **halogen-flash-server-memory**: Memory model, two failure modes (stall, 400), co-tenancy.
|
|
- **frank-primary-ram-stability**: DDR5 4800 vs 4600 MT/s, chunk boundaries, prefix cache eviction.
|
|
- **frank-medium-bench-protocol**: Restore half optional? No. Self-restoring harness recommended.
|
|
- **authentik-user-creation**: Django ORM workflow, group-based access.
|
|
- **orchestration-quadlet**: Rootless Podman boot persistence.
|
|
- **context-bloat-distillation**: Skill directory to vault migration.
|
|
- **comfyui-qwen-workflow**: iGPU image generation, ghostty display.
|
|
- **expert-arena-definition**: `arena = budget - non-expert - kv_reserve - graphs - overhead`.
|
|
- **skill-routing-pitfall**: `html-change-report` for git commits, not creative writing.
|