frank-default-small: retarget to lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8

Ornith-1.5-35B-A3B-FP8 failed to load on FreeToken v0.1.2: its
compressed-tensors checkpoint uses a nonstandard expert tensor layout
(FreeToken: 'Missing MoE expert source layers: gate_up').  Retarget to
the HauhauCS Aggressive FP8 finetune of the SAME Qwen3.6-35B-A3B arch
Huihui used, which carries compressed-tensors BLOCK-fp8 (128x128) with
the standard per-expert naming (mlp.experts.N.gate_proj/up_proj/down_proj
+ .weight_scale) that FreeToken's MoE loader reads.

- MODEL_DIR -> /home/dank/models/hauhaucs-qwen36-35b-a3b-aggressive-fp8
- use the checkpoint's own chat_template.jinja (drop the Sharp override;
  forcing a template the finetune wasn't trained on risks format drift)
- pool plan unchanged (same ~35 GB FP8 mass / qwen3_5_moe arch)

scripts/frank-default-small.env

@@ -1,5 +1,5 @@
-# Tunables for frank-default-small (ornith-ai/Ornith-1.5-35B-A3B-FP8,
+# Tunables for frank-default-small (lamianlbe/Qwen3.6-35B-A3B-Uncensored-
-# qwen3_5_moe hybrid MoE, Dankserver, RTX 3090).
+# HauhauCS-Aggressive-fp8, qwen3_5_moe hybrid MoE, Dankserver, RTX 3090).
 #
 # Runtime: FreeToken v0.1.2 (edge-native MoE serving engine).
 # Installed via uv into /home/dank/freetoken/.venv (CPython 3.12).
@@ -8,30 +8,33 @@
 # layers page KV (~20 KiB/token); the GDN layers use constant-size
 # per-sequence state (mamba slot pool).
 #
-# Model: ornith-ai/Ornith-1.5-35B-A3B-FP8 (HuggingFace safetensors,
+# Model: lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8
-# compressed-tensors FP8, ~35 GB on disk).  Ornith-1.5 is a
+# (HuggingFace safetensors, ~37.5 GB on disk).  HauhauCS "Aggressive"
-# self-improvement RL-trained coding/agentic model built on Qwen3.5 +
+# uncensored finetune of Qwen3.6-35B-A3B — same qwen3_5_moe arch as the
-# Gemma4; ~35B total / ~3B active (A3B), same qwen3_5_moe arch and the
+# Huihui FP8 it replaces (40 layers: 10 full-attention + 30 GDN, 256
-# same ~35 GB FP8 weight mass as the Huihui FP8 it replaces, so the
+# experts / 8 used, 2 KV heads × 128 dim, native 262144 context) and the
-# FreeToken memory/pool plan carries unchanged.  Native 262144 context.
+# same ~35 GB FP8 weight mass, so the FreeToken memory/pool plan carries
-# Reasoning model: each assistant turn opens with a <think> block,
+# unchanged.  Quantized to compressed-tensors BLOCK-fp8 (e4m3, dynamic
-# surfaced separately via the qwen3 reasoning parser (REASONING=on).
+# group-128 activations, 128×128 weight blocks) with the standard
-# The RTX 3090 has no FP8 tensor cores, so FP8 runs via upcast (as
+# per-expert tensor layout (mlp.experts.{N}.gate_proj/up_proj/down_proj
-# Huihui did).  Replaced batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-
+# + .weight_scale) that FreeToken's MoE loader reads — the layout the
-# FP8 on 2026-09-01.
+# Ornith compressed-tensors checkpoint lacked.  Also ships a BF16 visual
+# encoder + MTP module (both in the quant "ignore" list; text-only
+# serving ignores them).  The RTX 3090 has no FP8 tensor cores, so FP8
+# runs via upcast.  Replaced batsclamp/Huihui-...-FP8 on 2026-09-01.
 #
-# RISK: this is compressed-tensors PER-CHANNEL FP8 (weights strategy
+# NOTE (loader risk): compressed-tensors block-fp8 is not the modelopt
-# "channel"), the one FP8 layout the runbook flags as a FreeToken v0.1.2
+# block-fp8 the runbook lists as ✅ — it shares the block layout and the
-# loader bug.  Load is being verified live; 37 GB fits RAM so a failure is
+# per-expert naming but carries a different quant_method.  Load is
-# a clean crash-loop, not a swap-thrash.  The BF16 sibling
+# verified live; 37.5 GB fits RAM so a failure is a clean exit, never a
-# (ornith-ai/Ornith-1.5-35B-A3B, ~70 GB) does NOT fit this 62 GB box —
+# swap-thrash.
-# never fall back to it (that is the 2026-09-01 Thomson outage mode).
 #
-# Chat template: the checkpoint's stock Qwen template is WRONG for this
+# Chat template: the checkpoint ships its own chat_template.jinja (the
-# model (its own model card says so and ships a modified
+# format this finetune was trained on); FreeToken reads it from the model
-# chat_template.jinja for the agentic benchmarks).  The fleet standard is
+# directory — no override.  The fleet Qwen-Sharp template is deliberately
-# the Qwen-Sharp template (scripts/chat-template.jinja, repo-tracked and
+# NOT forced here: applying a template the model was not trained on risks
-# shared with frank-default-large) — see CHAT_TEMPLATE_FILE below.
+# format drift on an uncensored finetune.  To force Sharp, set
+# CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja.
 #
 # Pool sizing: GPU budget (~17.6 GiB after weights + fixed caches) is
 # split MoE-first.  MOE_CACHE_SIZE pins resident expert slots; NUM_TOKENS
@@ -44,14 +47,8 @@
 # The ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
 # incident).  This box is NVIDIA/CUDA, but the rule keeps the env portable.
 
-# Model directory (safetensors checkpoint, ~35 GB, compressed-tensors FP8).
+# Model directory (safetensors checkpoint, ~37.5 GB, compressed-tensors block-fp8).
-MODEL_DIR=/home/dank/models/ornith-1.5-35b-a3b-fp8
+MODEL_DIR=/home/dank/models/hauhaucs-qwen36-35b-a3b-aggressive-fp8
-
-# Chat template: the fleet Qwen-Sharp template (repo-tracked), shared with
-# frank-default-large.  The checkpoint's stock template mis-serializes
-# reasoning for this model; the Sharp template renders the <think> trail and
-# the Qwen3.5-family tool-call shape correctly.
-CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
 
 # FreeToken venv (uv-managed CPython 3.12, ft CLI, flashinfer, torch+cu130).
 FT_VENV=/home/dank/freetoken/.venv

scripts/frank-default-small.service

@@ -1,5 +1,5 @@
 [Unit]
-Description=FreeToken Ornith-1.5-35B-A3B-FP8 qwen3_5_moe hybrid MoE (dankserver, frank-default-small)
+Description=FreeToken Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-FP8 qwen3_5_moe hybrid MoE (dankserver, frank-default-small)
 After=network-online.target
 Wants=network-online.target
 

scripts/serve-frank-small.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
-# Serve ornith-ai/Ornith-1.5-35B-A3B-FP8 as frank-default-small via
+# Serve lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8 as
-# FreeToken on Dankserver (RTX 3090, 62 GB RAM).
+# frank-default-small via FreeToken on Dankserver (RTX 3090, 62 GB RAM).
 #
 # Runtime: FreeToken v0.1.2 (edge-native MoE serving engine).
 #   - Experts live in host RAM; an LRU cache of expert slots is resident
@@ -10,14 +10,15 @@
 #     hold constant-size per-sequence state (mamba slots).
 #   - OpenAI-compatible API (/v1/chat/completions, /v1/models).
 #
-# Model: ornith-ai/Ornith-1.5-35B-A3B-FP8 — Ornith-1.5 (self-improvement
+# Model: lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8 —
-# RL-trained coding/agentic model over Qwen3.5 + Gemma4), 35B total / 3B
+# HauhauCS "Aggressive" uncensored finetune of Qwen3.6-35B-A3B in
-# active, in compressed-tensors per-channel FP8 (float-quantized, dynamic
+# compressed-tensors block-fp8 (e4m3, dynamic group-128 activations,
-# token activations).  Same qwen3_5_moe arch and ~35 GB FP8 mass as the
+# 128×128 weight blocks) with the standard per-expert tensor layout
-# Huihui FP8 it replaces: 40 layers (10 full-attention + 30 GDN), 256
+# (mlp.experts.{N}.gate_proj/up_proj/down_proj + .weight_scale).  Same
-# experts / 8 used, 2 KV heads × 128 dim, native 262144 context.  FreeToken
+# qwen3_5_moe arch and ~35 GB FP8 mass as the Huihui FP8 it replaces: 40
-# loads a Qwen-Sharp chat template over it (see CHAT_TEMPLATE_FILE in the
+# layers (10 full-attention + 30 GDN), 256 experts / 8 used, 2 KV heads ×
-# env file); the checkpoint's own stock template mis-serializes reasoning.
+# 128 dim, native 262144 context.  FreeToken reads the checkpoint's own
+# chat_template.jinja from the model directory (no template override).
 #
 # Pool sizing: the GPU budget is split MoE-first between expert residency
 # and KV pages.  --moe-cache-size pins the number of resident expert
@@ -39,7 +40,7 @@ if [[ -f "$ENV_FILE" ]]; then
 fi
 
 # --- defaults (used when the env file does not set them) -----------------
-: "${MODEL_DIR:=/home/dank/models/ornith-1.5-35b-a3b-fp8}"
+: "${MODEL_DIR:=/home/dank/models/hauhaucs-qwen36-35b-a3b-aggressive-fp8}"
 : "${ALIAS:=frank-default-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7770}"