rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
This commit is contained in:
@@ -372,15 +372,15 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — the ACTIVE
|
||||
frank-small runbook since 2026-10-05 (evening; **UNCECERSED RCO-IQ3_XXS
|
||||
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-131072]] — the ACTIVE
|
||||
frank-secondary runbook since 2026-10-05 (evening; **UNCECERSED RCO-IQ3_XXS
|
||||
with MTP back on since 2026-10-06 evening**): llama.cpp master (CUDA
|
||||
sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS
|
||||
(the MTP head in the file is drafted) on the RTX 3090 — TWO
|
||||
131,072-token conversations at Frank's 128k cap, the tightest fit in
|
||||
the endpoint's history (489 MiB measured free), q8_0 K + q8_0 V, vision
|
||||
through the publisher's own projector, ~63 t/s recipe decode with MTP
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
|
||||
- [[frank-secondary-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
|
||||
superseded by swift15): llama.cpp master serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** — TWO conversations at
|
||||
200,704 tokens each with fully unquantized f16 KV and mmproj vision;
|
||||
@@ -388,32 +388,32 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
weight-bandwidth), batch 4096 / ubatch 2048 (+4.6 % prefill, measured
|
||||
2026-10-05), 2,279 MiB free, and the ≥256 px image-probe trap (tiny
|
||||
images answer "video")
|
||||
- [[frank-small-llama-mimo-q5km]] — RETIRED 2026-10-02 → 10-03 (rollback
|
||||
- [[frank-secondary-llama-mimo-q5km]] — RETIRED 2026-10-02 → 10-03 (rollback
|
||||
target, payload on disk; owns the measured boot-pair table): same runtime
|
||||
and shape one rung lower — 104.8 tok/s, 3,571 MiB true free
|
||||
- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; that same
|
||||
- [[frank-secondary-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; that same
|
||||
Q6_K rung returned ACTIVE 2026-10-03 at the new shape, see
|
||||
[[frank-small-llama-mimo-q6k-2x200704]]): the first llama.cpp + GGUF shape
|
||||
[[frank-secondary-llama-mimo-q6k-2x200704]]): the first llama.cpp + GGUF shape
|
||||
for this endpoint, Q6_K at 2 x 262,144 / f16-K + q8_0-V, 20.9 GiB measured
|
||||
- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (rollback target):
|
||||
- [[frank-secondary-sglang-mimo-v26]] — RETIRED 2026-10-01 (rollback target):
|
||||
SGLang in a self-contained python3.12 venv serving
|
||||
letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5
|
||||
hybrid, INT8 W8A16, in-checkpoint vision, NO MTP weights shipped) on
|
||||
Dankclaw's RTX 3090, port 7771 — fp8 KV, pinned 606,208-token pool,
|
||||
2 x 262,144 slots, bf16 mamba states, eager prefill
|
||||
- [[frank-small-gemma4-12b]] — RETIRED 2026-10-01 (rollback target):
|
||||
- [[frank-secondary-gemma4-12b]] — RETIRED 2026-10-01 (rollback target):
|
||||
mainline llama.cpp serving
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M
|
||||
(MTP assistant sidecar + ngram-mod, q8_0 KV) on
|
||||
Dankclaw's RTX 3090, port 7771
|
||||
- [[frank-small-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
|
||||
- [[frank-secondary-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
|
||||
rollback target): vLLM serving google/gemma-4-12B-it-qat-w4a16-ct (w4a16
|
||||
Marlin) with the bf16 MTP assistant on the RTX 3090 — bf16 KV, 289,647-
|
||||
token pool, 2 concurrent full-context sessions, native text+image+audio
|
||||
- [[frank-small-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target):
|
||||
the prior frank-small runbook (2026-09-16 → 09-17), ISTA-DASLab
|
||||
- [[frank-secondary-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target):
|
||||
the prior frank-secondary runbook (2026-09-16 → 09-17), ISTA-DASLab
|
||||
Qwen3.8-27B-GSQ-RCO IQ3_S with own in-file MTP head + BF16 projector
|
||||
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for the old frank-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
|
||||
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for the old frank-secondary (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
|
||||
- [[freetoken-frank-small]] — RETIRED 2026-09-15: the FreeToken/CUDA frank-small runbook, retired with the RTX 3090
|
||||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
|
||||
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
|
||||
@@ -433,7 +433,7 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
|
||||
- [[frank-large-benchmark]] — backend/spec benchmark behind the old head
|
||||
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
|
||||
- [[frank-small-benchmark]] — the small-model benchmark suite
|
||||
- [[frank-secondary-benchmark]] — the small-model benchmark suite
|
||||
- [[frank-medium-batch-ubatch]] — the 2026-09-16 batch/ubatch sweep on the 2x
|
||||
R9700 layer split: prefill PEAKS at ubatch 512 and falls on both sides, the
|
||||
logical batch buys nothing measurable, decode never moves, and the ubatch's
|
||||
|
||||
Reference in New Issue
Block a user