From 11638442e7eee4b66edba5a0c38f06647c1b5133 Mon Sep 17 00:00:00 2001 From: Frank Date: Tue, 1 Sep 2026 02:53:20 -0700 Subject: [PATCH] Revert frank-default-small to Huihui FP8: Thomson-1.0-Small is BF16 ~66GB, exceeds Dankserver 62GB RAM (thrash/OOM); needs a quantized checkpoint --- scripts/frank-default-small.env | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/frank-default-small.env b/scripts/frank-default-small.env index 9998952..e0dc179 100644 --- a/scripts/frank-default-small.env +++ b/scripts/frank-default-small.env @@ -32,8 +32,8 @@ # The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP # incident). This box is NVIDIA/CUDA, but the rule keeps the env portable. -# Model directory (safetensors checkpoint, ~35 GB, BF16 qwen3_5_moe MoE). -MODEL_DIR=/home/dank/models/thomson-1.0-small +# Model directory (safetensors checkpoint, ~35 GB, block-fp8). +MODEL_DIR=/home/dank/models/huihui-qwen36-35b-a3b-abliterated-fp8 # FreeToken venv (uv-managed CPython 3.12, ft CLI, flashinfer, torch+cu130). FT_VENV=/home/dank/freetoken/.venv