# Raw Memories

## 01a042cc-e1c5-7482-87a2-14deadb82202
updated_at: 1787833610
## frank-default-large: Qwen3.8-Flash-Next tuning on Dankclaw (Strix Halo gfx1151)

### Model & Architecture
- kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params (51.2B PLE n-gram + 125.8B MoE/attention)
- 56B MoE: 512 experts (10 used), 48 layers (12 attn + 36 SSM, full_attention_interval=4), 2 KV heads x 256 dim
- QSA indexer: 4 heads, top_k=2048; hyper-connections: count=4, low_rank=320
- PLE layer 1: ngram_size=3, heads_per_ngram=8
- Quant: Q4_0_ROCMFP4_STRIX (file_type=105), 113.47 GiB single GGUF
- PLE table: 50.66 GiB Q8_0, streamed from page cache (NEVER in GPU VRAM)
- GPU resident: ~63-69 GiB on Strix Halo unified memory pool (116 GiB SVM-backed)

### Runtime
- ROCmFPX llama.cpp fork at /home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server
- PR #27742 (qwen4exp architecture) merged
- Build: -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie

### Critical Constraints (DO NOT CHANGE)
- FIT=off: auto-fit segfaults on this model's hyper-connection/indexer config
- -ngl 999 places all layers on SVM-backed GPU pool
- NEVER name env vars TEMP, HOST, or GPU_MAX_HEAP_SIZE — ROCm HIP runtime reads bare env vars (2026-08-13 TEMP incident)
- llama.cpp reads --host/--port/--temp as CLI flags only, never env vars
- Text-only: no mmproj, no MTP draft head

### Current Tunings (env file: frank-default-large.env)
- BATCH_SIZE=4096, UBATCH_SIZE=4096
- NUM_THREADS=16, THREAD_BATCH=32
- CTX_SIZE=524288, PARALLEL=2
- CACHE_TYPE_K=f16, CACHE_TYPE_V=f16
- CTX_CHECKPOINTS=16, CACHE_RAM=8192
- HSA_OVERRIDE_GFX_VERSION=11.5.1
- Port 7770 (LISTEN_PORT)

### Benchmark Status (interrupted)
- Goal: tune batch/ubatch for optimal token prefill
- Next planned step: threads-batch sweep
- Benchmark servers targeted ports 7772 and 7773
- Session interrupted during health-check polling loop; servers returned no output via curl on those ports (likely still loading 113 GiB model or not yet started)
- The repeated poll pattern (5+ identical checks, ~5s apart) suggests the assistant was waiting for model load completion before starting benchmarks

### Workflow Pattern
- SSH into dankclaw from controller; check health via `curl -s http://127.0.0.1:PORT/health`
- Model load on unified memory is extremely slow (113 GiB mmap); benchmark scripts must account for long startup
- Use frank-default-small (the serving model for this agent) to orchestrate benchmarks against frank-default-large

## 01a042a4-7e85-7728-802d-c8442896f6e6
updated_at: 1787827055
## frank-default-large model swap (2026-09-02)

**Host:** Dankclaw — AMD Radeon 8060S (Strix Halo), gfx1151 iGPU, 64 GiB unified memory. Runtime: charlie12345/ROCmFPX llama.cpp build-kairic v81. Port 7770.

**Swap:** STRIX_LEAN-GGUF (56B MoE / 35.76B PLE, 3 shards, 98.5 GiB) → Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B total / 51.2B PLE, single file, 113.47 GiB on disk).

**File path:** `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf`

**Key architectural fact:** PLE (Projection Layer Embedding) table streams from page cache — never enters GPU VRAM regardless of tier. This is why a 114 GiB file fits in 64 GiB unified memory: only ~63-69 GiB is GPU-resident (same as LEAN variant). The PLE table occupies host RAM/page-cache.

**Performance:** Load time ~55 seconds. Generation speed and prompt throughput within noise of STRIX_LEAN at every context depth. 200/200 tests pass.

**Config files modified:**
- `scripts/frank-default-large.env` — MODEL_DIR, MODEL_FILE, header comments
- `scripts/serve-frank-large.sh` — header comments, default values
- `docs/LLM/kingjones-strix-rocmfp4.md` — new 195-line runbook
- `docs/LLM/index.md` — active model reference updated
- `docs/index.md` — top-level index links new doc

**Git:** Branch `frank-large-strix-model` pushed to origin (Gitea on Dankclaw).

**Prior history on this slot (for context):** CIRU PromptForge v3 → Kairic Edge IU4 (reverted) → kingjones Qwen3.8-27B ROCmFP4 STRIX MTP (Q6 AGENT, standalone MTP draft, Turbo4 V cache, 262K ctx/1 slot) → Q8_0 AGENT experiment (failed, set_rows bug) → STRIX MTP restored → now Flash-Next full 177B.