rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
This commit is contained in:
1 parent
c71a3066c0
commit
87721ab0fe
87 files changed
+460
-460
No files matched your search
@@ -1,27 +1,27 @@
|
||||
---
|
||||
type: configuration
|
||||
title: FreeToken frank-small Runbook
|
||||
description: RETIRED 2026-09-15 - FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-small on Dankserver (RTX 3090 era).
|
||||
title: FreeToken frank-secondary Runbook
|
||||
description: RETIRED 2026-09-15 - FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-secondary on Dankserver (RTX 3090 era).
|
||||
tags:
|
||||
- freetoken
|
||||
- dankserver
|
||||
- frank-small
|
||||
- frank-secondary
|
||||
- qwen3.6
|
||||
- moe
|
||||
- fp8
|
||||
timestamp: 2026-08-30
|
||||
---
|
||||
|
||||
# FreeToken frank-small Runbook
|
||||
# FreeToken frank-secondary Runbook
|
||||
|
||||
> **RETIRED 2026-09-15.** The RTX 3090 left Dankserver and the successor
|
||||
> [[frank-medium-rocmfpx]] (charlie12345/ROCmFPX llama.cpp on 2x Radeon AI
|
||||
> PRO R9700) took the :7770 endpoint and the port. This page is kept as the
|
||||
> retirement record of the FreeToken/CUDA era; the unit was archived to
|
||||
> `~/.AGENTTRASH/20260915-frank-small-retire/host-units/` and the repo files
|
||||
> `~/.AGENTTRASH/20260915-frank-secondary-retire/host-units/` and the repo files
|
||||
> to `scripts/.AGENTTRASH/`.
|
||||
|
||||
The `frank-small` service on Dankserver is served by
|
||||
The `frank-secondary` service on Dankserver is served by
|
||||
**FreeToken v0.1.2** — an edge-native MoE serving engine that keeps
|
||||
expert weights in host RAM with an LRU GPU cache. Model history on this
|
||||
runtime: official `Qwen/Qwen3.6-35B-A3B-FP8` (since 2026-09-02) →
|
||||
@@ -134,8 +134,8 @@ modelopt block-fp8 remains the only FP8 path that loads.
|
||||
|
||||
## Sampling
|
||||
|
||||
**frank-small samples by default, and our config never said so.**
|
||||
`scripts/serve-frank-small.sh` passes no sampling arguments at all — only
|
||||
**frank-secondary samples by default, and our config never said so.**
|
||||
`scripts/serve-frank-secondary.sh` passes no sampling arguments at all — only
|
||||
`--model`, `--served-model-name`, `--host`, `--port`, `--moe-cache-size`,
|
||||
`--num-tokens` and `--max-running-requests`. The behaviour comes from the
|
||||
runtime's own default:
|
||||
@@ -160,7 +160,7 @@ would be byte-identical.
|
||||
|
||||
### This is the opposite of frank-large
|
||||
|
||||
| | frank-small (FreeToken) | frank-large (halogen) |
|
||||
| | frank-secondary (FreeToken) | frank-large (halogen) |
|
||||
|---|---|---|
|
||||
| Silent client (no `temperature`) | **samples** at temp 1.0 / top_k 20 / top_p 0.95 | **greedy** — no server-side sampler defaults |
|
||||
| Deterministic across runs | No | Yes |
|
||||
@@ -168,7 +168,7 @@ would be byte-identical.
|
||||
|
||||
A client that talks to both endpoints and sends no sampler gets
|
||||
non-deterministic answers from :7770 on Dankserver and reproducible ones from
|
||||
:7770 on Dankclaw. If a workflow needs determinism from frank-small it must
|
||||
:7770 on Dankclaw. If a workflow needs determinism from frank-secondary it must
|
||||
send `temperature: 0` explicitly, or the service must be relaunched with
|
||||
`--sampling-defaults none` (framework defaults only).
|
||||
|
||||
@@ -188,9 +188,9 @@ send `temperature: 0` explicitly, or the service must be relaunched with
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `scripts/serve-frank-small.sh` | Serve script (launches `ft serve`) |
|
||||
| `scripts/frank-small.env` | Tunables (model path, pool sizes) |
|
||||
| `scripts/frank-small.service` | Systemd unit |
|
||||
| `scripts/serve-frank-secondary.sh` | Serve script (launches `ft serve`) |
|
||||
| `scripts/frank-secondary.env` | Tunables (model path, pool sizes) |
|
||||
| `scripts/frank-secondary.service` | Systemd unit |
|
||||
|
||||
## Limitations
|
||||
|
||||
@@ -209,7 +209,7 @@ send `temperature: 0` explicitly, or the service must be relaunched with
|
||||
- **`model` field is required** in `/v1/chat/completions` bodies —
|
||||
FreeToken rejects requests without it with a 422 (`Field required`),
|
||||
unlike llama.cpp's server which defaults to the loaded model. Send
|
||||
`"model": "frank-small"`.
|
||||
`"model": "frank-secondary"`.
|
||||
- Load on this swap took well under the old llama.cpp numbers:
|
||||
weights → experts (parallel build) → CUDA graph capture → Uvicorn
|
||||
in ~2 minutes; the 2026-08-30 swap verified ~40 tok/s decode on a
|
||||
|
||||
Reference in new issue
Block a user