dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Revert frank-default-small to Huihui FP8: Thomson-1.0-Small is BF16 ~66GB, exceeds Dankserver 62GB RAM (thrash/OOM); needs a quantized checkpoint

Browse Source
This commit is contained in:
Frank
2026-09-01 02:53:20 -07:00
parent 415f64964a
commit 11638442e7
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-small.env
+2 -2
View File
@@ -32,8 +32,8 @@
# The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
# incident). This box is NVIDIA/CUDA, but the rule keeps the env portable.
# Model directory (safetensors checkpoint, ~35 GB, BF16 qwen3_5_moe MoE).
MODEL_DIR=/home/dank/models/thomson-1.0-small
# Model directory (safetensors checkpoint, ~35 GB, block-fp8).
MODEL_DIR=/home/dank/models/huihui-qwen36-35b-a3b-abliterated-fp8
# FreeToken venv (uv-managed CPython 3.12, ft CLI, flashinfer, torch+cu130).
FT_VENV=/home/dank/freetoken/.venv