rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)

Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
This commit is contained in:
Frank committed 2026-10-06 15:37:13 -07:00
1 parent c71a3066c0
commit 87721ab0fe
87 files changed
+460 -460

No files matched your search

@@ -1,27 +1,27 @@
---
type: configuration
title: FreeToken frank-small Runbook
description: RETIRED 2026-09-15 - FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-small on Dankserver (RTX 3090 era).
title: FreeToken frank-secondary Runbook
description: RETIRED 2026-09-15 - FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 as frank-secondary on Dankserver (RTX 3090 era).
tags:
- freetoken
- dankserver
- frank-small
- frank-secondary
- qwen3.6
- moe
- fp8
timestamp: 2026-08-30
---
# FreeToken frank-small Runbook
# FreeToken frank-secondary Runbook
> **RETIRED 2026-09-15.** The RTX 3090 left Dankserver and the successor
> [[frank-medium-rocmfpx]] (charlie12345/ROCmFPX llama.cpp on 2x Radeon AI
> PRO R9700) took the :7770 endpoint and the port. This page is kept as the
> retirement record of the FreeToken/CUDA era; the unit was archived to
> `~/.AGENTTRASH/20260915-frank-small-retire/host-units/` and the repo files
> `~/.AGENTTRASH/20260915-frank-secondary-retire/host-units/` and the repo files
> to `scripts/.AGENTTRASH/`.
The `frank-small` service on Dankserver is served by
The `frank-secondary` service on Dankserver is served by
**FreeToken v0.1.2** — an edge-native MoE serving engine that keeps
expert weights in host RAM with an LRU GPU cache. Model history on this
runtime: official `Qwen/Qwen3.6-35B-A3B-FP8` (since 2026-09-02) →
@@ -134,8 +134,8 @@ modelopt block-fp8 remains the only FP8 path that loads.
## Sampling
**frank-small samples by default, and our config never said so.**
`scripts/serve-frank-small.sh` passes no sampling arguments at all — only
**frank-secondary samples by default, and our config never said so.**
`scripts/serve-frank-secondary.sh` passes no sampling arguments at all — only
`--model`, `--served-model-name`, `--host`, `--port`, `--moe-cache-size`,
`--num-tokens` and `--max-running-requests`. The behaviour comes from the
runtime's own default:
@@ -160,7 +160,7 @@ would be byte-identical.
### This is the opposite of frank-large
| | frank-small (FreeToken) | frank-large (halogen) |
| | frank-secondary (FreeToken) | frank-large (halogen) |
|---|---|---|
| Silent client (no `temperature`) | **samples** at temp 1.0 / top_k 20 / top_p 0.95 | **greedy** — no server-side sampler defaults |
| Deterministic across runs | No | Yes |
@@ -168,7 +168,7 @@ would be byte-identical.
A client that talks to both endpoints and sends no sampler gets
non-deterministic answers from :7770 on Dankserver and reproducible ones from
:7770 on Dankclaw. If a workflow needs determinism from frank-small it must
:7770 on Dankclaw. If a workflow needs determinism from frank-secondary it must
send `temperature: 0` explicitly, or the service must be relaunched with
`--sampling-defaults none` (framework defaults only).
@@ -188,9 +188,9 @@ send `temperature: 0` explicitly, or the service must be relaunched with
| File | Purpose |
|------|---------|
| `scripts/serve-frank-small.sh` | Serve script (launches `ft serve`) |
| `scripts/frank-small.env` | Tunables (model path, pool sizes) |
| `scripts/frank-small.service` | Systemd unit |
| `scripts/serve-frank-secondary.sh` | Serve script (launches `ft serve`) |
| `scripts/frank-secondary.env` | Tunables (model path, pool sizes) |
| `scripts/frank-secondary.service` | Systemd unit |
## Limitations
@@ -209,7 +209,7 @@ send `temperature: 0` explicitly, or the service must be relaunched with
- **`model` field is required** in `/v1/chat/completions` bodies —
FreeToken rejects requests without it with a 422 (`Field required`),
unlike llama.cpp's server which defaults to the loaded model. Send
`"model": "frank-small"`.
`"model": "frank-secondary"`.
- Load on this swap took well under the old llama.cpp numbers:
weights → experts (parallel build) → CUDA graph capture → Uvicorn
in ~2 minutes; the 2026-08-30 swap verified ~40 tok/s decode on a