docs: link STRIX runbook from top-level index
This commit is contained in:
1 parent
0cf1f02bb2
commit
263fa99c16
4 files changed
+225
-24
No files matched your search
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN as
|
||||
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX as
|
||||
# frank-default-large via the charlie12345/ROCmFPX llama.cpp fork with
|
||||
# PR #27742 (qwen4exp architecture) merged, on Dankclaw (Strix Halo
|
||||
# gfx1151, unified memory).
|
||||
@@ -9,17 +9,17 @@
|
||||
# with -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON
|
||||
# -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DLLAMA_BUILD_WEBUI=OFF.
|
||||
#
|
||||
# Model: 56B MoE (512 experts, 10 used), 48 layers (12 attention + 36
|
||||
# SSM/Mamba, full_attention_interval=4), 2 KV heads × 256 dim, QSA
|
||||
# Model: 177B params (56B MoE + 51.2B PLE n-gram table). 512 experts,
|
||||
# 10 used, 48 layers (12 attention + 36 SSM/Mamba,
|
||||
# full_attention_interval=4), 2 KV heads x 256 dim, QSA
|
||||
# indexer (4 heads, top_k=2048), hyper-connections (count=4, low_rank=320),
|
||||
# PLE (layer 1). Q4_0_ROCMFP4_STRIX_LEAN quant, 98.5 GiB, 3-part split.
|
||||
#
|
||||
# Context: 131072 (128k) × 1 slot. f16 KV cache. Model is loaded with
|
||||
# mmap (default) — the 98.5 GiB file is memory-mapped; the Strix Halo
|
||||
# unified memory (116 GiB SVM-backed) provides the GPU address space.
|
||||
# PLE (layer 1). Q4_0_ROCMFP4_STRIX quant, 113.47 GiB, single file.
|
||||
# PLE table (50.66 GiB Q8_0) streamed from page cache; never in GPU VRAM.
|
||||
# GPU resident: ~63-69 GiB. Context: up to 131072.
|
||||
# Loaded with mmap (default) into the Strix Halo unified memory pool.
|
||||
# --fit off is set because the auto-fit code segfaults on this model's
|
||||
# hyper-connection/indexer configuration; -ngl 999 places all layers on
|
||||
# the GPU (SVM-backed), which the unified memory pool handles.
|
||||
# the GPU (SVM-backed).
|
||||
#
|
||||
# The Qwen-Sharp chat template (scripts/chat-template.jinja) is applied at
|
||||
# runtime so the model emits reasoning in DeepSeek format; the reasoning
|
||||
@@ -37,8 +37,8 @@ fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${LLAMA_BIN:=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server}"
|
||||
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN}"
|
||||
: "${MODEL_FILE:=Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-00001-of-00003.gguf}"
|
||||
:: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX}"
|
||||
:: "${MODEL_FILE:=Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf}"
|
||||
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
||||
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
||||
: "${ALIAS:=frank-default-large}"
|
||||
|
||||
Reference in new issue
Block a user