docs: rename frank-default-large/small/sightsound to frank-large/small/npu across vault
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
This commit is contained in:
1 parent
5a083eee6c
commit
8212d4451a
68 files changed
+1053
-1207
No files matched your search
@@ -1,20 +1,20 @@
|
||||
---
|
||||
type: configuration
|
||||
title: FreeToken frank-default-small Runbook
|
||||
description: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-default-small on Dankserver (RTX 3090).
|
||||
title: FreeToken frank-small Runbook
|
||||
description: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-small on Dankserver (RTX 3090).
|
||||
tags:
|
||||
- freetoken
|
||||
- dankserver
|
||||
- frank-default-small
|
||||
- frank-small
|
||||
- qwen3.6
|
||||
- moe
|
||||
- fp8
|
||||
timestamp: 2026-08-30
|
||||
---
|
||||
|
||||
# FreeToken frank-default-small Runbook
|
||||
# FreeToken frank-small Runbook
|
||||
|
||||
The `frank-default-small` service on Dankserver is served by
|
||||
The `frank-small` service on Dankserver is served by
|
||||
**FreeToken v0.1.2** — an edge-native MoE serving engine that keeps
|
||||
expert weights in host RAM with an LRU GPU cache. Model history on this
|
||||
runtime: official `Qwen/Qwen3.6-35B-A3B-FP8` (since 2026-09-02) →
|
||||
@@ -130,8 +130,8 @@ modelopt block-fp8 remains the only FP8 path that loads.
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `scripts/serve-frank-small.sh` | Serve script (launches `ft serve`) |
|
||||
| `scripts/frank-default-small.env` | Tunables (model path, pool sizes) |
|
||||
| `scripts/frank-default-small.service` | Systemd unit |
|
||||
| `scripts/frank-small.env` | Tunables (model path, pool sizes) |
|
||||
| `scripts/frank-small.service` | Systemd unit |
|
||||
|
||||
## Limitations
|
||||
|
||||
@@ -150,7 +150,7 @@ modelopt block-fp8 remains the only FP8 path that loads.
|
||||
- **`model` field is required** in `/v1/chat/completions` bodies —
|
||||
FreeToken rejects requests without it with a 422 (`Field required`),
|
||||
unlike llama.cpp's server which defaults to the loaded model. Send
|
||||
`"model": "frank-default-small"`.
|
||||
`"model": "frank-small"`.
|
||||
- Load on this swap took well under the old llama.cpp numbers:
|
||||
weights → experts (parallel build) → CUDA graph capture → Uvicorn
|
||||
in ~2 minutes; the 2026-08-30 swap verified ~40 tok/s decode on a
|
||||
|
||||
Reference in new issue
Block a user