rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)

Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
This commit is contained in:
Frank committed 2026-10-06 15:37:13 -07:00
1 parent c71a3066c0
commit 87721ab0fe
87 files changed
+460 -460

No files matched your search

+20 -20
View File
@@ -16,7 +16,7 @@ digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
ctx, vision ON)** and
**llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
**llama.cpp CUDA (frank-secondary, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
128k cap (the tightest fit in the endpoint's history — 489 MiB free),
@@ -51,7 +51,7 @@ frank-large, port 7770]
F[flash-server v2 abliterated Swift1.5 .hgn + ngram table + vision tower
2 x 200192 ctx, MTP always-on, image input]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
frank-secondary, port 7771]
H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
2 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
E2 --> E --> F
@@ -103,7 +103,7 @@ frank-small, port 7771]
(`slow`/`advisor` → frank-primary) carried forward. Superseded by the
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
engine moved with the model
- [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
(CUDA sm_86) serving ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-
@@ -120,7 +120,7 @@ frank-small, port 7771]
98/100). Replaces BOTH same-day payloads: the live MiMo Q6_K shape
(below) and the Ornith-1.5-9B Q8_0 swap that was committed
(77decb5..380aafa) but never deployed
- [[frank-small-llama-mimo-q6k-2x200704]] — **RETIRED 2026-10-05 (evening;
- [[frank-secondary-llama-mimo-q6k-2x200704]] — **RETIRED 2026-10-05 (evening;
rollback target; payload on disk)**: llama.cpp master (CUDA sm_86)
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, TWO slots of
200,704 with FULLY UNQUANTIZED KV (f16 K + f16 V) and the mmproj for
@@ -129,7 +129,7 @@ frank-small, port 7771]
(+4.6 % prefill, measured), 2,279 MiB of the card left free. This was
the LIVE payload when Swift cut over. The ubatch A/B table and the
probe-size trap (>=256 px images or the model says "video") live inside
- [[frank-small-llama-mimo-q5km]] — **RETIRED 2026-10-02 → 10-03 (rollback
- [[frank-secondary-llama-mimo-q5km]] — **RETIRED 2026-10-02 → 10-03 (rollback
target; payload on disk; owns the measured boot-pair table)**: the same
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
104.8 tok/s, 3,571 MiB true free
@@ -265,10 +265,10 @@ frank-small, port 7771]
kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the
runtime it replaced
- [[freetoken-frank-small]] — RETIRED 2026-09-15, when the RTX 3090 moved to
frank-large: the FreeToken v0.1.2 frank-small runbook (batsclamp
frank-large: the FreeToken v0.1.2 frank-secondary runbook (batsclamp
Huihui-Qwen3.6-35B-A3B-abliterated-FP8), kept as the retirement record. The
NAME was reused on 2026-09-16 by a different service on Dankclaw — see
[[frank-small-qwen38-gsq-rco]]
[[frank-secondary-qwen38-gsq-rco]]
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
@@ -337,35 +337,35 @@ frank-small, port 7771]
.hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
400,384-position pool, deployed flags, memory model, client contract,
troubleshooting (2026-09-16 → 09-19)
- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02** (the same
- [[frank-secondary-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02** (the same
rung returned ACTIVE 2026-10-03 at the 2 x 200,704 f16-KV shape —
[[frank-small-llama-mimo-q6k-2x200704]]): llama.cpp (master, CUDA sm_86
[[frank-secondary-llama-mimo-q6k-2x200704]]): llama.cpp (master, CUDA sm_86
build) serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO
native 262,144-token sessions with ASYMMETRIC KV (K f16 unquantized + V
q8_0, which forces flash-attn), ~20.9/24.6 GiB measured, text-only (the
GGUF has no vision), reasoning_content, no spec decode; payload still at
`/home/dank/models/MiMo-Q6K/`
- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (hours; replaced
by [[frank-small-llama-mimo-q6k]] for the asymmetric KV SGLang cannot
- [[frank-secondary-sglang-mimo-v26]] — RETIRED 2026-10-01 (hours; replaced
by [[frank-secondary-llama-mimo-q6k]] for the asymmetric KV SGLang cannot
express; venv still on disk, INT8 payload staged to
`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`): SGLang in a
self-contained python3.12 venv serving leteschlead MiMo-V2.6 INT8
W8A16 AutoRound, 2 x 262144 ctx, fp8 KV, in-checkpoint vision,
83.9 tok/s decode
- [[frank-small-gemma4-12b]] — RETIRED 2026-09-17 → 2026-10-01 (replaced by
[[frank-small-sglang-mimo-v26]] → the llama.cpp MiMo shape; payload staged
- [[frank-secondary-gemma4-12b]] — RETIRED 2026-09-17 → 2026-10-01 (replaced by
[[frank-secondary-sglang-mimo-v26]] → the llama.cpp MiMo shape; payload staged
to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/`):
mainline llama.cpp serving
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
3090, with the separate MTP assistant sidecar chained behind ngram-mod,
q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB.
- [[frank-small-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
- [[frank-secondary-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
rollback target): vLLM serving google/gemma-4-12B-it-qat-w4a16-ct (w4a16
Marlin) with the bf16 MTP assistant on the RTX 3090 — bf16 KV, 289,647-
token pool, 2 concurrent full-context sessions, native text+image+audio.
- [[frank-small-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target): the
prior frank-small runbook (2026-09-16 → 09-17), ISTA-DASLab
- [[frank-secondary-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target): the
prior frank-secondary runbook (2026-09-16 → 09-17), ISTA-DASLab
Qwen3.8-27B-GSQ-RCO IQ3_S with the model's own in-file MTP head and
f16 KV at 131k
- [[x4-pair-experiment]] — MEASURED 2026-09-16: what mainline llama.cpp can and
@@ -396,9 +396,9 @@ frank-small, port 7771]
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
on Dankclaw (retired)
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-small, retired 2026-09-02)
Dankserver (frank-secondary, retired 2026-09-02)
- [[ikllama-ornith-dankserver]] — the retired ik_llama.cpp runtime for
frank-small (2026-08-26): q8_0 + Hadamard KV cache,
frank-secondary (2026-08-26): q8_0 + Hadamard KV cache,
split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags
- [[kingjones-strix-mtp]] — the former Q6 AGENT runbook (restored 2026-09-01
after Q8_0 failure, then superseded by the STRIX deployment on 2026-08-27)
@@ -408,7 +408,7 @@ frank-small, port 7771]
(deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the 2x R9700 box (frank-primary)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
the flash-server v2 runtime since 2026-10-06, frank-small on the RTX 3090)
the flash-server v2 runtime since 2026-10-06, frank-secondary on the RTX 3090)
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
the retirement record — the IOMMU/SVA err=-19 root cause and where
@@ -432,7 +432,7 @@ frank-small, port 7771]
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware
notes and sizing (this architecture family is what the retired frank-small
notes and sizing (this architecture family is what the retired frank-secondary
served in the Huihui abliterated FP8 variant — see
[[freetoken-frank-small]]; the Ornith-1.5-35B-A3B CRACK checkpoint
that preceded it is archived at [[dealign-ornith-crack]])