frank-small: SGLang MiMo-V2.6-Distill-Qwen-9B INT8 (qwen3_5 hybrid, RTX 3090)

Replaces llama.cpp Gemma4-12B-QAT (payload + binary retained as rollback).
SGLang in a self-contained python3.12 venv with an extracted CUDA 13.0
nvcc toolchain for the sgl-kernel/flashinfer JIT.

Shape: fp8 KV, pinned 606,208-token pool / 4 x 151,552 slots, bf16 mamba
states pinned at 32 slots (the engine autofills the ssm pool to a fixed
~5.1 GB regardless of dtype), eager prefill (no headroom for graph
capture), no spec decode (the checkpoint ships no mtp.* tensors despite
mtp_num_hidden_layers=1 — NEXTN drafts with an uninitialised 4.24 GiB
ghost at 0% acceptance).

Verified: 83.9 tok/s decode, 170k single prompt, 4 concurrent 54k
requests, vision (in-checkpoint tower), reasoning_content field.

shared/homepage/services.json

@@ -303,8 +303,8 @@
     "section": "AI",
     "details": {
       "role": "Fast utility model",
-      "spec": "Gemma4-12B QAT (Q4_K_M, 7.4 GiB) — 1M-token pool across four conversations; fastest at images.",
-      "inputs": ["text", "image", "audio"],
+      "spec": "MiMo-V2.6-Distill-Qwen-9B (INT8 W8A16, 12 GiB) on SGLang — 606k-token pool across four 151k-token conversations; 84 tok/s decode; sees images.",
+      "inputs": ["text", "image"],
       "use": "Agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence."
     },
     "url": "",