rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)

Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
This commit is contained in:
Frank committed 2026-10-06 15:37:13 -07:00
1 parent c71a3066c0
commit 87721ab0fe
87 files changed
+460 -460

No files matched your search

@@ -1,7 +1,7 @@
{
"uid": "llm-fleet",
"title": "LLM fleet \u2014 capacity and reuse",
"description": "The signals that explain the LLM endpoints under load. One metric namespace: llamacpp:* across all three endpoints — frank-small (Dankclaw, RTX 3090), frank-large (Dankclaw, gfx1151 iGPU) and frank-primary (Dankserver, 2x R9700, llama.cpp again since the 2026-09-26 swap back from Lucebox DS4.1). The endpoint picker selects between them. See serverAppConfigs-docs/Infrastructure/Observability.md for the namespace split and the addressability rules.",
"description": "The signals that explain the LLM endpoints under load. One metric namespace: llamacpp:* across all three endpoints — frank-secondary (Dankclaw, RTX 3090), frank-large (Dankclaw, gfx1151 iGPU) and frank-primary (Dankserver, 2x R9700, llama.cpp again since the 2026-09-26 swap back from Lucebox DS4.1). The endpoint picker selects between them. See serverAppConfigs-docs/Infrastructure/Observability.md for the namespace split and the addressability rules.",
"tags": [
"llm",
"capacity"
@@ -166,7 +166,7 @@
"id": 5,
"type": "timeseries",
"title": "Decode steps and batch size",
"description": "Per-step counters: steps per second and the busy-slot count per step (the effective batch). With frank-primary's --parallel 1 the slot count stays at 0 or 1 and says nothing about the card; with frank-small's 4 slots it tracks the agent-fleet burst shape.",
"description": "Per-step counters: steps per second and the busy-slot count per step (the effective batch). With frank-primary's --parallel 1 the slot count stays at 0 or 1 and says nothing about the card; with frank-secondary's 4 slots it tracks the agent-fleet burst shape.",
"gridPos": {
"h": 8,
"w": 12,
+4 -4
View File
@@ -36,8 +36,8 @@
Pick a model in the chat's model selector — the three models the fleet runs are
described <a href="#models">below</a>. Open WebUI keeps your chat history; the raw
model endpoints do not.</p>
<p>Images go to <code>frank-primary</code> or <code>frank-small</code>
(both see them). Audio (WAV) goes to <code>frank-small</code> only.</p>
<p>Images go to <code>frank-primary</code> or <code>frank-secondary</code>
(both see them). Audio (WAV) goes to <code>frank-secondary</code> only.</p>
</section>
<section id="models">
@@ -62,7 +62,7 @@
<li><strong>Best for:</strong> software engineering, system administration, and instruction tasks.</li>
</ul>
<h2>frank-small — the fast utility model</h2>
<h2>frank-secondary — the fast utility model</h2>
<ul>
<li><strong>Model:</strong> Gemma4-12B QAT (Balanced, Q4_K_M, 7.4 GiB).</li>
<li><strong>Capacity:</strong> one million tokens of pooled context across four simultaneous conversations (up to 262k each); fastest at images.</li>
@@ -71,7 +71,7 @@
</ul>
<p class="foot-note">Which one to pick? Small utility work, RAG, search, and
screenshots: <code>frank-small</code>. Software engineering, sysadmin, and
screenshots: <code>frank-secondary</code>. Software engineering, sysadmin, and
instruction tasks: <code>frank-primary</code>. Cybersecurity and debugging:
<code>frank-large</code> (with Frank's permission).</p>
</section>
+1 -1
View File
@@ -298,7 +298,7 @@
"showLink": false
},
{
"name": "frank-small",
"name": "frank-secondary",
"category": "user",
"section": "AI",
"details": {
+6 -6
View File
@@ -32,7 +32,7 @@ scrape_configs:
# bridge container, and on this netavark host a container can only reach the
# host's LAN IP at a PUBLISHED port — node-exporter's net=host bind on
# 192.168.0.143:9100 is refused by construction (the same rule that sends
# frank-large and frank-small to the gateway below). node-dankserver is a
# frank-large and frank-secondary to the gateway below). node-dankserver is a
# different machine, so its LAN address works.
- job_name: "node-dankclaw"
static_configs:
@@ -79,20 +79,20 @@ scrape_configs:
# this container, not the guarded vhosts: these are machine clients and
# Prometheus cannot solve an Anubis challenge.
#
# One metric namespace: llamacpp:* — frank-large, frank-small and
# One metric namespace: llamacpp:* — frank-large, frank-secondary and
# frank-primary, all llama.cpp with /metrics (the family verified from
# /metrics on 2026-09-16 and 2026-09-20) and all exporting
# requests_deferred (the queue depth, the only form llama.cpp shows the
# capacity wall in), spec_decode_num_drafts_total, tokens_predicted_total,
# and the prompt_tokens_cached_total / prompt_tokens_total reuse pair.
#
# frank-large and frank-small are scraped over host.containers.internal,
# frank-large and frank-secondary are scraped over host.containers.internal,
# not their own LAN addresses: this Prometheus is a bridge-networked
# container (Network=dankclaw_default), and on this netavark host a
# container can only reach the host's LAN IP at a PUBLISHED port — a
# host-process bind like llama-server's 0.0.0.0:7770 is refused at
# 192.168.0.143:7770 while it answers at 169.254.1.2:7770. That is
# why frank-small was red here for days (up{job="frank-small"} == 0,
# why frank-secondary was red here for days (up{job="frank-secondary"} == 0,
# "dial tcp 192.168.0.143:7771") even though the endpoint was healthy and
# exporting: the target address was unreachable by construction, not the
# service down. Same host, same rule, so both local endpoints use the
@@ -105,12 +105,12 @@ scrape_configs:
# plus radiance:* families, a different namespace — hence the separate
# job below, not a re-add to the llamacpp pair. The netavark quirk above
# is same-host only; plain LAN routing REACHES 192.168.0.235:7770.
- job_name: "frank-small"
- job_name: "frank-secondary"
metrics_path: "/metrics"
static_configs:
- targets: ["host.containers.internal:7771"]
labels:
service: frank-small
service: frank-secondary
- job_name: "frank-large"
metrics_path: "/metrics"
+3 -3
View File
@@ -1,6 +1,6 @@
# Alerting and recording rules for the LLM fleet.
#
# One metric namespace: llamacpp:* — frank-small and frank-large, both
# One metric namespace: llamacpp:* — frank-secondary and frank-large, both
# llama.cpp with /metrics (the family verified from /metrics on 2026-09-16
# and 2026-09-20). frank-primary LEFT the family 2026-10-05: FreeToken (the
# PR587 ROCm engine) has no /metrics endpoint, so its scrape job was
@@ -110,8 +110,8 @@ groups:
annotations:
summary: "{{ $labels.job }}: scrape target down for 2 minutes"
description: >-
Prometheus cannot reach the LLM endpoint. frank-small and
Prometheus cannot reach the LLM endpoint. frank-secondary and
frank-large are root units on Dankclaw: `systemctl is-active
frank-small` / `frank-large`. Both scraped endpoints export
frank-secondary` / `frank-large`. Both scraped endpoints export
the llamacpp:* family. (frank-primary left the scrape set
2026-10-05 — FreeToken exports no /metrics.)