docs: link STRIX runbook from top-level index

This commit is contained in:
Frank committed 2026-08-27 03:01:27 -07:00
1 parent 0cf1f02bb2
commit 263fa99c16
4 files changed
+225 -24

No files matched your search

+11 -11
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN as
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX as
# frank-default-large via the charlie12345/ROCmFPX llama.cpp fork with
# PR #27742 (qwen4exp architecture) merged, on Dankclaw (Strix Halo
# gfx1151, unified memory).
@@ -9,17 +9,17 @@
# with -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON
# -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DLLAMA_BUILD_WEBUI=OFF.
#
# Model: 56B MoE (512 experts, 10 used), 48 layers (12 attention + 36
# SSM/Mamba, full_attention_interval=4), 2 KV heads × 256 dim, QSA
# Model: 177B params (56B MoE + 51.2B PLE n-gram table). 512 experts,
# 10 used, 48 layers (12 attention + 36 SSM/Mamba,
# full_attention_interval=4), 2 KV heads x 256 dim, QSA
# indexer (4 heads, top_k=2048), hyper-connections (count=4, low_rank=320),
# PLE (layer 1). Q4_0_ROCMFP4_STRIX_LEAN quant, 98.5 GiB, 3-part split.
#
# Context: 131072 (128k) × 1 slot. f16 KV cache. Model is loaded with
# mmap (default) — the 98.5 GiB file is memory-mapped; the Strix Halo
# unified memory (116 GiB SVM-backed) provides the GPU address space.
# PLE (layer 1). Q4_0_ROCMFP4_STRIX quant, 113.47 GiB, single file.
# PLE table (50.66 GiB Q8_0) streamed from page cache; never in GPU VRAM.
# GPU resident: ~63-69 GiB. Context: up to 131072.
# Loaded with mmap (default) into the Strix Halo unified memory pool.
# --fit off is set because the auto-fit code segfaults on this model's
# hyper-connection/indexer configuration; -ngl 999 places all layers on
# the GPU (SVM-backed), which the unified memory pool handles.
# the GPU (SVM-backed).
#
# The Qwen-Sharp chat template (scripts/chat-template.jinja) is applied at
# runtime so the model emits reasoning in DeepSeek format; the reasoning
@@ -37,8 +37,8 @@ fi
# --- defaults (used when the env file does not set them) -------------------
: "${LLAMA_BIN:=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server}"
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN}"
: "${MODEL_FILE:=Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-00001-of-00003.gguf}"
:: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
:: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
: "${CHAT_TEMPLATE_KWARGS:={}}"
: "${ALIAS:=frank-default-large}"