rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
This commit is contained in:
1 parent
c71a3066c0
commit
87721ab0fe
87 files changed
+460
-460
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: ik_llama.cpp on Dankserver (Ornith Q8_0)
|
||||
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-small on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
|
||||
description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-secondary on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
|
||||
tags:
|
||||
- ikllama
|
||||
- llamacpp
|
||||
@@ -16,7 +16,7 @@ timestamp: 2026-08-26
|
||||
|
||||
# ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
|
||||
|
||||
> **Active 2026-08-26.** `frank-small` switched from upstream
|
||||
> **Active 2026-08-26.** `frank-secondary` switched from upstream
|
||||
> llama.cpp (b10121) to `ikawrakow/ik_llama.cpp` (commit `b166e269`) on the
|
||||
> RTX 3090. This page documents the build, the mandatory FA build flags, and
|
||||
> the MoE / CPU-offload runtime arguments.
|
||||
@@ -101,8 +101,8 @@ a hard compile error.
|
||||
|
||||
## Runtime configuration
|
||||
|
||||
Files: `scripts/serve-frank-small.sh`, `scripts/frank-small.env`,
|
||||
`scripts/frank-small.service` in the repo. The service just calls
|
||||
Files: `scripts/serve-frank-secondary.sh`, `scripts/frank-secondary.env`,
|
||||
`scripts/frank-secondary.service` in the repo. The service just calls
|
||||
the serve script; the env file holds all tunables.
|
||||
|
||||
### KV cache: K default f16 / V q6_0 + Hadamard (V only)
|
||||
@@ -241,7 +241,7 @@ apply.
|
||||
The previous runtime `upstream llama.cpp b10121` and its binary
|
||||
(`/home/dank/llama.cpp/build/bin/llama-server`) are **left untouched**. To
|
||||
rollback to the old stack, change `LLAMA_SERVER=` in
|
||||
`scripts/frank-small.env` back to that path and
|
||||
`scripts/frank-secondary.env` back to that path and
|
||||
`CACHE_TYPE_K/V` back to `f16` with Hadamard off, and re-enable the ngram
|
||||
spec / `--spec-type` flags. The model file is not changed either way.
|
||||
|
||||
|
||||
Reference in new issue
Block a user