chore: track live agent config (config, models, append-system, managed-skills, memories, plugin lock)

2026-08-28 07:56:47 -07:00
parent 772607434a
commit 552d1ef8bf
23 changed files with 640 additions and 0 deletions
@@ -0,0 +1,72 @@
+# Raw Memories
+
+## 01a042cc-e1c5-7482-87a2-14deadb82202
+updated_at: 1787833610
+## frank-default-large: Qwen3.8-Flash-Next tuning on Dankclaw (Strix Halo gfx1151)
+
+### Model & Architecture
+- kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params (51.2B PLE n-gram + 125.8B MoE/attention)
+- 56B MoE: 512 experts (10 used), 48 layers (12 attn + 36 SSM, full_attention_interval=4), 2 KV heads x 256 dim
+- QSA indexer: 4 heads, top_k=2048; hyper-connections: count=4, low_rank=320
+- PLE layer 1: ngram_size=3, heads_per_ngram=8
+- Quant: Q4_0_ROCMFP4_STRIX (file_type=105), 113.47 GiB single GGUF
+- PLE table: 50.66 GiB Q8_0, streamed from page cache (NEVER in GPU VRAM)
+- GPU resident: ~63-69 GiB on Strix Halo unified memory pool (116 GiB SVM-backed)
+
+### Runtime
+- ROCmFPX llama.cpp fork at /home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server
+- PR #27742 (qwen4exp architecture) merged
+- Build: -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie
+
+### Critical Constraints (DO NOT CHANGE)
+- FIT=off: auto-fit segfaults on this model's hyper-connection/indexer config
+- -ngl 999 places all layers on SVM-backed GPU pool
+- NEVER name env vars TEMP, HOST, or GPU_MAX_HEAP_SIZE — ROCm HIP runtime reads bare env vars (2026-08-13 TEMP incident)
+- llama.cpp reads --host/--port/--temp as CLI flags only, never env vars
+- Text-only: no mmproj, no MTP draft head
+
+### Current Tunings (env file: frank-default-large.env)
+- BATCH_SIZE=4096, UBATCH_SIZE=4096
+- NUM_THREADS=16, THREAD_BATCH=32
+- CTX_SIZE=524288, PARALLEL=2
+- CACHE_TYPE_K=f16, CACHE_TYPE_V=f16
+- CTX_CHECKPOINTS=16, CACHE_RAM=8192
+- HSA_OVERRIDE_GFX_VERSION=11.5.1
+- Port 7770 (LISTEN_PORT)
+
+### Benchmark Status (interrupted)
+- Goal: tune batch/ubatch for optimal token prefill
+- Next planned step: threads-batch sweep
+- Benchmark servers targeted ports 7772 and 7773
+- Session interrupted during health-check polling loop; servers returned no output via curl on those ports (likely still loading 113 GiB model or not yet started)
+- The repeated poll pattern (5+ identical checks, ~5s apart) suggests the assistant was waiting for model load completion before starting benchmarks
+
+### Workflow Pattern
+- SSH into dankclaw from controller; check health via `curl -s http://127.0.0.1:PORT/health`
+- Model load on unified memory is extremely slow (113 GiB mmap); benchmark scripts must account for long startup
+- Use frank-default-small (the serving model for this agent) to orchestrate benchmarks against frank-default-large
+
+## 01a042a4-7e85-7728-802d-c8442896f6e6
+updated_at: 1787827055
+## frank-default-large model swap (2026-09-02)
+
+**Host:** Dankclaw — AMD Radeon 8060S (Strix Halo), gfx1151 iGPU, 64 GiB unified memory. Runtime: charlie12345/ROCmFPX llama.cpp build-kairic v81. Port 7770.
+
+**Swap:** STRIX_LEAN-GGUF (56B MoE / 35.76B PLE, 3 shards, 98.5 GiB) → Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B total / 51.2B PLE, single file, 113.47 GiB on disk).
+
+**File path:** `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf`
+
+**Key architectural fact:** PLE (Projection Layer Embedding) table streams from page cache — never enters GPU VRAM regardless of tier. This is why a 114 GiB file fits in 64 GiB unified memory: only ~63-69 GiB is GPU-resident (same as LEAN variant). The PLE table occupies host RAM/page-cache.
+
+**Performance:** Load time ~55 seconds. Generation speed and prompt throughput within noise of STRIX_LEAN at every context depth. 200/200 tests pass.
+
+**Config files modified:**
+- `scripts/frank-default-large.env` — MODEL_DIR, MODEL_FILE, header comments
+- `scripts/serve-frank-large.sh` — header comments, default values
+- `docs/LLM/kingjones-strix-rocmfp4.md` — new 195-line runbook
+- `docs/LLM/index.md` — active model reference updated
+- `docs/index.md` — top-level index links new doc
+
+**Git:** Branch `frank-large-strix-model` pushed to origin (Gitea on Dankclaw).
+
+**Prior history on this slot (for context):** CIRU PromptForge v3 → Kairic Edge IU4 (reverted) → kingjones Qwen3.8-27B ROCmFP4 STRIX MTP (Q6 AGENT, standalone MTP draft, Turbo4 V cache, 262K ctx/1 slot) → Q8_0 AGENT experiment (failed, set_rows bug) → STRIX MTP restored → now Flash-Next full 177B.