rename frank-small → frank-secondary (Dankclaw llama.cpp :7771)

Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:

- Service trio: scripts/{frank-small.env,frank-small.service,
  serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
  frank-secondary-*.md (9 files); the FreeToken runbook stays
  freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
  LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
  3090, and the Dankserver service that became frank-medium) are
  preserved verbatim — they describe what the service was called
  at the time.
This commit is contained in:
Frank committed 2026-10-06 15:37:13 -07:00
1 parent c71a3066c0
commit 87721ab0fe
87 files changed
+460 -460

No files matched your search

@@ -18,7 +18,7 @@ The old scripts moved to `old/scripts/` in this repository.
| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-large | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
| frank-small | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
| frank-secondary | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
The RTX 3090 runs the whole 27B model standalone. No RPC worker, no
tensor split, no NPU.
@@ -40,7 +40,7 @@ tensor split, no NPU.
`--spec-draft-ngl all`, `--spec-draft-p-min 0`) then ngram-mod fallback
(48/64/24).
### frank-small — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
### frank-secondary — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
- Base: Qwen3.6-35B-A3B (hybrid MoE, 256 experts top-8, ~3B active, 40
layers: 10 blocks of 3 DeltaNet + 1 Attention, native 262144 ctx).
@@ -103,10 +103,10 @@ Production choice: **131072 (128K)** — the maximum that keeps about
5. Verify: `curl -sf http://127.0.0.1:7770/health` and a chat completion.
6. Confirm VRAM headroom with `nvidia-smi`.
### Dankclaw (frank-small)
### Dankclaw (frank-secondary)
1. Stop the old services: `sudo systemctl stop frank-large
frank-small` (retired PP head and NPU endpoint).
frank-secondary` (retired PP head and NPU endpoint).
2. Install uv and create the venv:
```
curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -131,9 +131,9 @@ Production choice: **131072 (128K)** — the maximum that keeps about
vision encoder falls back to torch SDPA and engine init OOMs.
5. Symlink the unit, reload, enable, restart:
```
sudo ln -sf /home/dank/vllmScripts/scripts/frank-small.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-small
sudo systemctl restart frank-small
sudo ln -sf /home/dank/vllmScripts/scripts/frank-secondary.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-secondary
sudo systemctl restart frank-secondary
```
6. Verify: `curl -sf http://127.0.0.1:52625/health` and a chat completion.
@@ -159,7 +159,7 @@ on gfx1151; newer vLLM releases may already include it.
Three changes landed after the initial deployment:
### 1. Tool calling on frank-small (HTTP 400 fix)
### 1. Tool calling on frank-secondary (HTTP 400 fix)
Clients (pi) send `tool_choice: "auto"`. vLLM rejects it with HTTP 400
unless the server enables tool calling. Fixes applied:
@@ -184,7 +184,7 @@ model behavior, which motivated the swap.
### 2. Model swap: Avesed INT8 W8A8 -> Quark W8A8 INT8
`frank-small` now serves
`frank-secondary` now serves
`nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8` (AMD Quark 0.11.1, ~35 GB,
vLLM-native expert layout). The Avesed compressed-tensors quant's
Qwen3-XML tool fidelity was poor. Verified with the Quark model: plain