docs: link STRIX runbook from top-level index

This commit is contained in:
Frank committed 2026-08-27 03:01:27 -07:00
1 parent 0cf1f02bb2
commit 263fa99c16
4 files changed
+225 -24

No files matched your search

+17 -13
View File
@@ -1,5 +1,5 @@
# Tunables for frank-default-large (kingjones777 Qwen3.8-Flash-Next
# ROCmFP4 STRIX_LEAN, qwen4exp hybrid MoE, Dankclaw, Strix Halo gfx1151).
# ROCmFP4 STRIX, qwen4exp hybrid MoE, Dankclaw, Strix Halo gfx1151).
#
# Runtime: ROCmFPX llama.cpp fork with PR #27742 (qwen4exp architecture)
# merged and built at /home/dank/GitPackages/ROCmFPX/build-qwen38.
@@ -7,16 +7,19 @@
# -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON
# -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie.
# Replaced the kingjones Qwen3.8-27B Q6_0 AGENT deployment on 2026-09-01.
# Replaced the STRIX_LEAN deployment on 2026-08-27.
#
# Model: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-GGUF.
# 56B MoE (512 experts, 10 used), 48 layers (12 attention + 36 SSM,
# full_attention_interval=4), 2 KV heads × 256 dim, embedding 2560,
# QSA indexer (4 heads, key_length 128, top_k 2048), hyper-connections
# Model: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF.
# 177B params (51.2B PLE n-gram table + 125.8B total MoE/attention).
# Qwen3.8-Flash-Next base: 56B MoE (512 experts, 10 used), 48 layers
# (12 attention + 36 SSM, full_attention_interval=4), 2 KV heads x 256 dim,
# embedding 2560, QSA indexer (4 heads, top_k=2048), hyper-connections
# (count=4, low_rank=320), PLE (layer 1, ngram_size=3, heads_per_ngram=8).
# Q4_0_ROCMFP4_STRIX_LEAN quant, 98.5 GiB, 3-part GGUF split.
# Q4_0_ROCMFP4_STRIX quant (file_type=105), 113.47 GiB, single GGUF file.
# PLE table: 50.66 GiB Q8_0, streamed from page cache (never in GPU VRAM).
# Text-only (no mmproj, no MTP draft head).
#
# Context: 262144 (256k) × 2 slots. f16 KV cache (~6.4 GiB per slot,
# Context: 262144 (256k) x 2 slots. f16 KV cache (~6.4 GiB per slot,
# ~12.8 GiB total). --fit off because the auto-fit code segfaults on this
# model's hyper-connection/indexer config; -ngl 999 places all layers on
# the SVM-backed GPU pool (116 GiB unified).
@@ -30,9 +33,9 @@
LLAMA_BIN=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server
# Model files from
# https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-GGUF
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN
MODEL_FILE=Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-00001-of-00003.gguf
# https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
MODEL_FILE=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
# Qwen-Sharp chat template (tracked in the repo, staged on the host). The
# Sharp template drives thinking on by default (medium effort); the server
@@ -46,9 +49,10 @@ ALIAS=frank-default-large
BIND_ADDR=0.0.0.0
LISTEN_PORT=7770
# Serving: 262144 context (256k) × 2 slots. f16 KV, context checkpoints,
# prompt cache. The model is 98.5 GiB loaded via mmap into the Strix Halo
# unified memory pool (116 GiB SVM-backed). --fit off avoids the auto-fit
# Serving: f16 KV, context checkpoints, prompt cache. The model is 113.47
# GiB loaded via mmap into the Strix Halo unified memory pool (116 GiB
# SVM-backed). PLE table (50.66 GiB Q8_0) is streamed from page cache; never
# enters GPU VRAM. GPU resident: ~63-69 GiB. --fit off avoids the auto-fit
# segfault on this model's qwen4exp hyper-connection/indexer configuration.
CTX_SIZE=524288
PARALLEL=2