rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
This commit is contained in:
1 parent
c71a3066c0
commit
87721ab0fe
87 files changed
+460
-460
No files matched your search
@@ -35,7 +35,7 @@ timestamp: 2026-10-01
|
||||
(`scripts/frank-large.service`, `User=dank`, `Restart=always`,
|
||||
`ExecStop=podman stop -t 120`, `TimeoutStopSec=180` > the stop grace).
|
||||
- **Model:** `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` in **full BF16**
|
||||
(17.53 GiB, 4 shards) — the same checkpoint frank-small serves as an INT8
|
||||
(17.53 GiB, 4 shards) — the same checkpoint frank-secondary serves as an INT8
|
||||
W8A16 AutoRound conversion on the 3090, here unquantized.
|
||||
- **Draft:** `Qwen/Qwen3.5-9B` (18 GiB on disk; only the 15 `mtp.*` tensors
|
||||
are loaded into the draft pass).
|
||||
@@ -82,7 +82,7 @@ Qwen3.5-9B that preserves the hidden-state geometry — identical
|
||||
per-position acceptance is the direct measurement that the distillation
|
||||
preserved more than geometry. MiMo itself ships **ZERO `mtp.*` tensors**
|
||||
(verified in both the BF16 base and the INT8 conversion) — which is exactly
|
||||
why frank-small's SGLang NEXTN draft is a 0.00-acceptance ghost: SGLang
|
||||
why frank-secondary's SGLang NEXTN draft is a 0.00-acceptance ghost: SGLang
|
||||
reads the head from the TARGET checkpoint, vLLM reads the draft from a
|
||||
separate path. That difference is the whole trick:
|
||||
|
||||
@@ -183,4 +183,4 @@ unit/env/serve trio and redeploying returns the 27B endpoint.
|
||||
- Engine image: https://hub.docker.com/r/kyuz0/vllm-therock-gfx1151
|
||||
- Serve script: `scripts/serve-frank-large.sh`; tunables: `scripts/frank-large.env`
|
||||
- Predecessor: [[frank-large-gufo-qwen38-27b]]; sibling on the same arch:
|
||||
[[frank-small-sglang-mimo-v26]]
|
||||
[[frank-secondary-sglang-mimo-v26]]
|
||||
Reference in new issue
Block a user