Frank
b6ce3afb1f
Operator call 2026-09-19: switch the iGPU endpoint from halogen-flash-server to jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the CIRU v4.4.1 runtime with the QSA window hotfix (host build against the private ROCm 10 SDK), one stream of the full 262144 trained window. The service leaves the quadlet shape for a root unit (the same shape as frank-small and frank-medium: a private-SDK host binary has no image to install and no generated unit to dodge); the halogen quadlet is archived, the deploy handler retires it on the host, gates on 7770, and its stray-run sweep matches the model path so it cannot take frank-small down.
45 lines
2.0 KiB
Bash
45 lines
2.0 KiB
Bash
# frank-large — CIRU v4.4.1 (QSA window hotfix) serving
|
|
# jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the Strix Halo iGPU (gfx1151).
|
|
#
|
|
# These are the launcher tunables read by the runtime's
|
|
# scripts/ciru/run-orca-server.sh (see the serve script). The 36-variable
|
|
# kernel bundle and the PLE pager defaults live in the runtime's own
|
|
# profiles/strix-halo-production.env, which the launcher sources AFTER this
|
|
# file; every profile line is a ${VAR:-default}, so a value set here wins.
|
|
#
|
|
# DO NOT name a new tunable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP runtime
|
|
# reads those bare (the 2026-08-13 TEMP incident). HOST is the one exception
|
|
# because the launcher itself consumes it for --host; its value doubles as a
|
|
# JIT-cache directory name, which is why 0.0.0.0 is safe.
|
|
|
|
# The assembled payload (scripts/download-frank-large-orca.sh): the launcher
|
|
# refuses to start without the main GGUF, the MTP head and the PLE sidecar.
|
|
MODEL_DIR=/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca
|
|
|
|
# Fleet endpoint: the same port and alias the halogen predecessor served, so
|
|
# no client, probe or models.yml entry changes address.
|
|
HOST=0.0.0.0
|
|
PORT=7770
|
|
|
|
# One stream, full context (operator call, 2026-09-19). MTP requires exactly
|
|
# one slot in this runtime, so the drafter's constraint and the operator's
|
|
# ask are the same number. 262144 is the model's trained window; the GGUF
|
|
# carries no rope scaling.
|
|
CONTEXT_SIZE=262144
|
|
PARALLEL_SLOTS=1
|
|
|
|
# MTP head: on. Depth 4 is the Orca variant's own default (mtp_depth_default
|
|
# in the launcher); the sidecar is the Q8_0 MTP file from the model repo.
|
|
ENABLE_MTP=1
|
|
|
|
# Text-only for now: the F16 projector is on disk at
|
|
# vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj; flip to 1 to enable it. The
|
|
# fleet's image path is frank-small either way, so models.yml stays
|
|
# input: [text] until this changes.
|
|
ENABLE_VISION=0
|
|
|
|
# PLE pager: the 52.4 GiB payload is mmap'd and paged by the engine's own
|
|
# workers; this is the hot window kept resident. The vendor's production
|
|
# profile default is 4096.
|
|
PLE_CACHE_MIB=4096
|