Frank
34ab6b4d0c
Ornith-1.5-35B-A3B-FP8 failed to load on FreeToken v0.1.2: its compressed-tensors checkpoint uses a nonstandard expert tensor layout (FreeToken: 'Missing MoE expert source layers: gate_up'). Retarget to the HauhauCS Aggressive FP8 finetune of the SAME Qwen3.6-35B-A3B arch Huihui used, which carries compressed-tensors BLOCK-fp8 (128x128) with the standard per-expert naming (mlp.experts.N.gate_proj/up_proj/down_proj + .weight_scale) that FreeToken's MoE loader reads. - MODEL_DIR -> /home/dank/models/hauhaucs-qwen36-35b-a3b-aggressive-fp8 - use the checkpoint's own chat_template.jinja (drop the Sharp override; forcing a template the finetune wasn't trained on risks format drift) - pool plan unchanged (same ~35 GB FP8 mass / qwen3_5_moe arch)
16 lines
469 B
Desktop File
16 lines
469 B
Desktop File
[Unit]
|
|
Description=FreeToken Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-FP8 qwen3_5_moe hybrid MoE (dankserver, frank-default-small)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-default-small.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh
|
|
Restart=on-failure
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target |