dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-primary: cutover state to Qwen3.8-Flash-Next Q8_0 (6 shards, PLE pinned, MTP4, sharp template)

Browse Source 
- env: STATE_FILE -> r9v-state/q8xl; add dev-overlay adapter mount
  (quantized inject shard dequant, fork 363d606), FULL_MUTABLE_CACHE=0
  (80-slot tiered vs the overlay's uncensored 160-slot injection), and
  the qwen-sharp-v22.4.0.jinja chat template via the launcher's new
  R9V_CHAT_TEMPLATE_FILE/KWARGS passthrough (fork 30d4a32)
- serve/deploy: shard gates extended to SHARD4/5/6 (unset-safe, the
  uncensored rollback state carries none)
- seams: STATE_FILE/LLM_PRIMARY_STATE -> q8xl
This commit is contained in:
Frank
2026-09-27 22:25:08 -07:00
parent 928fd81dbd
commit b06895d725
4 changed files with 61 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+13 -7
View File
@@ -1185,10 +1185,12 @@ deploy_llm_small() {
# deploy_llm_primary — frank-primary is the R9V radiance-vLLM fork (docker-
# pinned prebuilt, gfx1201, driven through the docker->podman shim) serving
# Qwen3.8-Flash-Next-Uncensored IQ4_XS with the MTP4 drafter on the two
# R9700s as a ROOT unit whose serve script execs the fork's launcher through
# rootless podman as dank. Same port and alias as the llama.cpp endpoint it
# replaced (2026-09-27 cutover); the harness models.yml id is unchanged.
# the unsloth Qwen3.8-Flash-Next Q8_0 payload (6 GGUF shards) with the MTP4
# drafter on the two R9700s as a ROOT unit whose serve script execs the
# fork's launcher through rootless podman as dank. Same port and alias as
# the endpoints it replaced (the 2026-09-27 cutover from uncensored
# IQ4_XS); the harness models.yml id is unchanged. The uncensored state dir
# (/home/dank/r9v-state/uncensored) stays on disk as the rollback target.
#
# Gates: the on-host R9V setup state must exist and be ready (it carries the
# pinned image digest and every payload path), the pinned runtime image must
@@ -1196,7 +1198,7 @@ deploy_llm_small() {
# PLE payload must match its pinned byte count, and every GGUF shard + the
# mmproj must exist (per-file sha256 verification was setup's job at fetch
# time, never re-hashed here).
LLM_PRIMARY_STATE=/home/dank/r9v-state/uncensored/setup.json
LLM_PRIMARY_STATE=/home/dank/r9v-state/q8xl/setup.json
LLM_PRIMARY_CONTAINER=frank-primary-r9v
deploy_llm_primary() {
@@ -1216,7 +1218,9 @@ config = state.get("config") or {}
if state.get("ready") is not True:
sys.exit("R9V setup state is not ready")
for key in ("R9V_IMAGE", "R9V_MODEL_DIR", "R9V_PLE_PATH", "R9V_PLE_EXPECTED_BYTES",
"R9V_TARGET_REL", "R9V_TARGET_SHARD2_REL", "R9V_TARGET_SHARD3_REL", "R9V_MMPROJ_REL"):
"R9V_TARGET_REL", "R9V_TARGET_SHARD2_REL", "R9V_TARGET_SHARD3_REL",
"R9V_TARGET_SHARD4_REL", "R9V_TARGET_SHARD5_REL", "R9V_TARGET_SHARD6_REL",
"R9V_MMPROJ_REL"):
print(f"{key}={shlex.quote(config.get(key, ''))}")
PY
)" || return 1
@@ -1239,7 +1243,9 @@ PY
fi
fi
local rel
for rel in "$R9V_TARGET_REL" "$R9V_TARGET_SHARD2_REL" "$R9V_TARGET_SHARD3_REL" "$R9V_MMPROJ_REL"; do
for rel in "$R9V_TARGET_REL" "${R9V_TARGET_SHARD2_REL:-}" "${R9V_TARGET_SHARD3_REL:-}" \
"${R9V_TARGET_SHARD4_REL:-}" "${R9V_TARGET_SHARD5_REL:-}" "${R9V_TARGET_SHARD6_REL:-}" \
"$R9V_MMPROJ_REL"; do
[[ -z "$rel" ]] && continue
if [[ ! -f "$R9V_MODEL_DIR/$rel" ]]; then
echo "ERROR: payload file missing: $R9V_MODEL_DIR/$rel" >&2