# frank-default-large operations

Use this when changing or troubleshooting the large model service on Dankclaw.

Current durable model/runtime:
- Host: Dankclaw, Strix Halo gfx1151 unified memory.
- Alias: frank-default-large.
- Endpoint: http://192.168.0.143:7770/v1, health on port 7770.
- Repo files: scripts/serve-frank-large.sh and scripts/frank-default-large.env under /home/dank/Projects/serverAppConfigs.
- Runtime: /home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server.
- Model: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF.
- Local GGUF: /home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf.

Hard constraints:
- Keep --fit off unless the qwen4exp hyper-connection auto-fit segfault is proven fixed.
- Keep PLE table mmap/page-cache streamed. Do not add --no-mmap and do not force the PLE table into GPU memory.
- Avoid bare env vars TEMP, HOST, and GPU_MAX_HEAP_SIZE. HIP/ROCm can read them unexpectedly.
- Do not enable MTP/speculative decoding casually. Prior MTP heads failed because blk.0.hc_attn_norm.weight was missing. Latest durable state has SPEC_TYPE=none.
- Stock llama.cpp cannot load this model; qwen4exp architecture support and ROCmFP4 tensor support must coexist in the same runtime tree.
- If flash-attn graph build fails on QSA mask, port scripts/patches/qwen4exp-qsa-f32-mask.patch into the active runtime tree and rebuild.

Before edits:
1. cd /home/dank/Projects/serverAppConfigs
2. git status --short
3. Inspect current scripts/frank-default-large.env and scripts/serve-frank-large.sh; do not rely on old memory for exact tunables.

After edits:
1. Run repo tests appropriate to the change.
2. Deploy/restart using the current repo workflow.
3. Verify health: curl -s http://192.168.0.143:7770/health or via localhost on Dankclaw.
4. Watch logs during model load; 113+ GiB mmap/model load can take significant time.

Benchmarking notes:
- Separate prefill/TTFT from decode tok/s.
- For decode sweeps, an 8K prompt and 256 generated tokens is sufficient unless testing long-run thermal/stability.
- Prior decode sweep found THREAD_BATCH=16 best, but current repo env always wins.