rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
This commit is contained in:
1 parent
c71a3066c0
commit
87721ab0fe
87 files changed
+460
-460
No files matched your search
@@ -80,7 +80,7 @@ co-tenants.
|
||||
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
|
||||
"move layers to the card" converts reclaimable cache into device memory and
|
||||
frees nothing for co-tenants. A served model's footprint ≠ weight size
|
||||
(frank-small: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
|
||||
(frank-secondary: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
|
||||
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
|
||||
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
|
||||
- **Context lengths: derive, never remember.** `llama-bench` computes
|
||||
@@ -157,7 +157,7 @@ in the timings. Context cap for full-model residency on that shape stayed
|
||||
## Related
|
||||
|
||||
- [[GGUF Compatibility]] — what a payload is before it is priced
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
|
||||
- [[frank-secondary-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
|
||||
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
|
||||
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
|
||||
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
|
||||
Reference in new issue
Block a user