command-center/agent/memories/--home-dank-Projects-serverAppConfigs--/MEMORY.md

frank-default-large Model & Runtime Reference

Host & Environment

• Host: Dankclaw (AMD Radeon 8060S Strix Halo, gfx1151 iGPU)
• Memory: 64 GiB unified memory + page cache/SVM (~116 GiB total pool)
• Runtime: charlie12345/ROCmFPX llama.cpp fork (llama-server)
• Build Flags: -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie

Model Architecture

• Model: Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B total params)
• Composition: 51.2B PLE (n-gram) + 125.8B MoE/Attention (56B active MoE: 512 experts, 10 used; 48 layers: 12 attn + 36 SSM)
• Quant: Q4_0_ROCMFP4_STRIX (file_type=105), ~113.47 GiB single GGUF
• VRAM Behavior: PLE table streams exclusively from page cache/host RAM. GPU resident memory remains ~63-69 GiB regardless of total file size.
• Indexing/Connections: QSA indexer (4 heads, top_k=2048); Hyper-connections (count=4, low_rank=320); PLE layer 1 (ngram_size=3, heads_per_ngram=8)

Critical Constraints (DO NOT OVERRIDE)

• FIT=off: Auto-fit crashes on this architecture's hyper-connection/indexer config.
• Context Limit: -ngl 999 required to force SVM backing for all layers.
• Env Vars: NEVER set TEMP, HOST, or GPU_MAX_HEAP_SIZE. ROCm HIP runtime consumes bare env vars. llama.cpp CLI flags (--host, --port, --temp) are not read from env.
• Capabilities: Text-only. No mmproj, no MTP draft head.

Configuration & Tunings (scripts/frank-default-large.env)

BATCH_SIZE=4096
UBATCH_SIZE=4096
NUM_THREADS=16
THREAD_BATCH=32
CTX_SIZE=524288
PARALLEL=2
CACHE_TYPE_K=f16
CACHE_TYPE_V=f16
CTX_CHECKPOINTS=16
CACHE_RAM=8192
HSA_OVERRIDE_GFX_VERSION=11.5.1
LISTEN_PORT=7770

Operational Workflow & Benchmarking

• Startup: Model load takes ~55s+. GPU pool handles heavy I/O via mmap.
• Health Checks: Use curl -s http://127.0.0.1:7770/health (or target benchmark ports). Always poll in loops (5+ checks, ~5s apart) before proceeding.
• Benchmark Orchestration: SSH from controller. Use frank-default-small to drive benchmark scripts against frank-default-large.
• Current Status: Batch/ubatch tuning interrupted during health-polling for benchmark servers (ports 7772/7773). Next phase: threads-batch sweep.
• Prior History: CIRU PromptForge v3 → Kairic Edge IU4 → Qwen3.8-27B ROCmFP4 STRIX MTP → Q8_0 AGENT (failed) → STRIX MTP → Qwen3.8-Flash-Next 177B.

File & Git References

• Model Path: /home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
• Configs: scripts/frank-default-large.env, scripts/serve-frank-large.sh
• Docs: docs/LLM/kingjones-strix-rocmfp4.md, docs/LLM/index.md, docs/index.md
• Git Branch: frank-large-strix-model (origin: Dankclaw Gitea)