dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

Browse Source 
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank committed 2026-10-10 17:03:14 -07:00
1 parent 5ced474eb9
commit 55f87fcdce
195 files changed
+2085 -2064
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md
+18 -18
View File
@@ -17,15 +17,15 @@ The old scripts moved to `old/scripts/` in this repository.
| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-large | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
| frank-secondary | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
| frank-dragoon | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
| frank-zealot | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
The RTX 3090 runs the whole 27B model standalone. No RPC worker, no
tensor split, no NPU.
## Models
### frank-large — HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF:IQ4_XS
### frank-dragoon — HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF:IQ4_XS
- Base: Qwen3.8-27B (dense 27B, 64 layers: 48 GatedDeltaNet + 16
gated-attention, hidden 5120, 24 Q heads / 4 KV heads, head dim 256,
@@ -40,7 +40,7 @@ tensor split, no NPU.
`--spec-draft-ngl all`, `--spec-draft-p-min 0`) then ngram-mod fallback
(48/64/24).
### frank-secondary — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
### frank-zealot — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
- Base: Qwen3.6-35B-A3B (hybrid MoE, 256 experts top-8, ~3B active, 40
layers: 10 blocks of 3 DeltaNet + 1 Attention, native 262144 ctx).
@@ -55,7 +55,7 @@ tensor split, no NPU.
instead of the Qwen3 XML structure). The Quark quant follows the format
cleanly; swapped in the evening of 2026-08-17.
## Context Math (frank-large, RTX 3090)
## Context Math (frank-dragoon, RTX 3090)
KV per token: 16 attention layers x 4 KV heads x 256 dim.
K q8_0 = 1088 B/token, V f16 = 2048 B/token, total 50176 B/token.
@@ -72,11 +72,11 @@ Budget (MiB): 24576 total - 14982 weights - 861 draft - 1024 fixed
Production choice: **131072 (128K)** — the maximum that keeps about
1.4 GiB headroom for checkpoints. Lower `CTX_SIZE` in
`scripts/frank-large.env` to 122880 for ~1.8 GiB headroom.
`scripts/frank-dragoon.env` to 122880 for ~1.8 GiB headroom.
## Install Runbook
### Dankserver (frank-large)
### Dankserver (frank-dragoon)
1. Stop the old worker: `sudo systemctl stop frank-pp-worker`.
2. Download the model and draft to `/home/dank/models/hauhaucs-qwen38/`.
@@ -96,17 +96,17 @@ Production choice: **131072 (128K)** — the maximum that keeps about
measured 95 t/s vs 1300+ t/s at 8K on the 3090).
4. Symlink the unit, reload, enable, restart:
```
sudo ln -sf /home/dank/vLLM/scripts/frank-large.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-large
sudo systemctl restart frank-large
sudo ln -sf /home/dank/vLLM/scripts/frank-dragoon.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-dragoon
sudo systemctl restart frank-dragoon
```
5. Verify: `curl -sf http://127.0.0.1:7770/health` and a chat completion.
6. Confirm VRAM headroom with `nvidia-smi`.
### Dankclaw (frank-secondary)
### Dankclaw (frank-zealot)
1. Stop the old services: `sudo systemctl stop frank-large
frank-secondary` (retired PP head and NPU endpoint).
1. Stop the old services: `sudo systemctl stop frank-dragoon
frank-zealot` (retired PP head and NPU endpoint).
2. Install uv and create the venv:
```
curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -131,9 +131,9 @@ Production choice: **131072 (128K)** — the maximum that keeps about
vision encoder falls back to torch SDPA and engine init OOMs.
5. Symlink the unit, reload, enable, restart:
```
sudo ln -sf /home/dank/vllmScripts/scripts/frank-secondary.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-secondary
sudo systemctl restart frank-secondary
sudo ln -sf /home/dank/vllmScripts/scripts/frank-zealot.service /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable frank-zealot
sudo systemctl restart frank-zealot
```
6. Verify: `curl -sf http://127.0.0.1:52625/health` and a chat completion.
@@ -159,7 +159,7 @@ on gfx1151; newer vLLM releases may already include it.
Three changes landed after the initial deployment:
### 1. Tool calling on frank-secondary (HTTP 400 fix)
### 1. Tool calling on frank-zealot (HTTP 400 fix)
Clients (pi) send `tool_choice: "auto"`. vLLM rejects it with HTTP 400
unless the server enables tool calling. Fixes applied:
@@ -184,7 +184,7 @@ model behavior, which motivated the swap.
### 2. Model swap: Avesed INT8 W8A8 -> Quark W8A8 INT8
`frank-secondary` now serves
`frank-zealot` now serves
`nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8` (AMD Quark 0.11.1, ~35 GB,
vLLM-native expert layout). The Avesed compressed-tensors quant's
Qwen3-XML tool fidelity was poor. Verified with the Quark model: plain