llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank
2026-10-10 17:03:14 -07:00
parent 5ced474eb9
commit 55f87fcdce
195 changed files with 2085 additions and 2064 deletions
@@ -1,4 +1,4 @@
# Tunables for frank-large (kingjones777/Qwen3.8-Flash-Next-ROCmFP4- # Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw, # STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
# Strix Halo gfx1151, unified memory). # Strix Halo gfx1151, unified memory).
# #
@@ -56,7 +56,7 @@ CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
# Alias + network. # Alias + network.
ALIAS=frank-large ALIAS=frank-dragoon
BIND_ADDR=0.0.0.0 BIND_ADDR=0.0.0.0
LISTEN_PORT=7770 LISTEN_PORT=7770
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-large) Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -7,8 +7,8 @@ Wants=network-online.target
Type=simple Type=simple
User=dank User=dank
Group=dank Group=dank
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env" Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
Restart=on-failure Restart=on-failure
RestartSec=10 RestartSec=10
+1 -1
View File
@@ -6,7 +6,7 @@
# is reachable on this port regardless of the boot tag. The NPU is a # is reachable on this port regardless of the boot tag. The NPU is a
# single-lane device: generations serialize FIFO, and the first request after a # single-lane device: generations serialize FIFO, and the first request after a
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM # model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
# pool shared with frank-large (iGPU). # pool shared with frank-dragoon (iGPU).
FLM_MODEL=nuextract3:4b FLM_MODEL=nuextract3:4b
FLM_HOST=0.0.0.0 FLM_HOST=0.0.0.0
FLM_PORT=7771 FLM_PORT=7771
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as # Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
# frank-large via the kingjones30/ROCmFPX llama.cpp runtime on # frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
# Dankclaw (Strix Halo gfx1151, unified memory). # Dankclaw (Strix Halo gfx1151, unified memory).
# #
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the # Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
@@ -29,7 +29,7 @@
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}" ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then if [[ -f "$ENV_FILE" ]]; then
set -a set -a
# shellcheck disable=SC1090 # shellcheck disable=SC1090
@@ -44,7 +44,7 @@ fi
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}" : "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}" : "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
: "${CHAT_TEMPLATE_KWARGS:={}}" : "${CHAT_TEMPLATE_KWARGS:={}}"
: "${ALIAS:=frank-large}" : "${ALIAS:=frank-dragoon}"
: "${BIND_ADDR:=0.0.0.0}" : "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7770}" : "${LISTEN_PORT:=7770}"
: "${CTX_SIZE:=401408}" : "${CTX_SIZE:=401408}"
+1 -1
View File
@@ -4,7 +4,7 @@
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a # Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
# single-lane device: generations serialize in FIFO order, so this is a # single-lane device: generations serialize in FIFO order, so this is a
# background/triage endpoint, not an interactive multi-user chat. The NPU # background/triage endpoint, not an interactive multi-user chat. The NPU
# draws from the unified RAM pool shared with frank-large (the iGPU # draws from the unified RAM pool shared with frank-dragoon (the iGPU
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md # llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
# for the memory budget and the model-selection tradeoff. # for the memory budget and the model-selection tradeoff.
# #
+97 -96
View File
@@ -9,11 +9,11 @@ services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
merge absorbed the former `vllmScripts` repo, history preserved). merge absorbed the former `vllmScripts` repo, history preserved).
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty, - **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service. Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki, via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (flash-server Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
container on the gfx1151 iGPU) and `frank-secondary` (llama.cpp CUDA on the container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
RTX 3090). Caddy on RTX 3090). Caddy on
Dankclaw reverse-proxies both hosts. Dankclaw reverse-proxies both hosts.
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | | Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
Retired endpoints (`frank-npu`, and every prior runtime) are in Retired endpoints (`frank-npu`, and every prior runtime) are in
the [runbook index](#runbook-index). The full swap history lives in the [runbook index](#runbook-index). The full swap history lives in
@@ -66,14 +66,14 @@ All edits flow through git. Never edit files on a remote machine directly.
./scripts/deploy.sh dankclaw ./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankserver ./scripts/deploy.sh dankserver
./scripts/deploy.sh dankclaw --service searxng ./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-primary ./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --dry-run ./scripts/deploy.sh dankclaw --dry-run
``` ```
The planner maps changed paths to services; the remote runner restarts only The planner maps changed paths to services; the remote runner restarts only
the selected service and records its revision outside the checkout. the selected service and records its revision outside the checkout.
Documentation-only changes are a no-op. LLM services are selected only by Documentation-only changes are a no-op. LLM services are selected only by
their own files — every endpoint (`frank-large`/`frank-secondary`/`frank-primary`) their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
by its unit/env/serve trio in `scripts/` — so by its unit/env/serve trio in `scripts/` — so
unrelated unrelated
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
@@ -85,7 +85,7 @@ restarts each service whose marker is behind HEAD *and* whose paths changed
since that marker — so a backlog accumulated by service-scoped deploys fires since that marker — so a backlog accumulated by service-scoped deploys fires
all at once, and one failing health gate aborts the run with every later all at once, and one failing health gate aborts the run with every later
service left pending. Name the service unless a drain is what you want service left pending. Name the service unless a drain is what you want
(`./scripts/deploy.sh dankserver --service frank-primary`). (`./scripts/deploy.sh dankserver --service frank-archon`).
### Deployment completeness (MANDATORY) ### Deployment completeness (MANDATORY)
@@ -229,17 +229,17 @@ archive.
**STOP the production service BEFORE benchmarking** — these servers serve **STOP the production service BEFORE benchmarking** — these servers serve
real users, and a bench beside a live service starves the GPU and corrupts real users, and a bench beside a live service starves the GPU and corrupts
results. Use the runtime's own harness: `llama-bench` for the llama.cpp results. Use the runtime's own harness: `llama-bench` for the llama.cpp
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-secondary were measured. Rolling your own bench endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-dragoon and frank-zealot were measured. Rolling your own bench
requires asking first. requires asking first.
Scratch ports are not used on LLM hosts — a Scratch ports are not used on LLM hosts — a
test process cannot share the single GPU anyway; validate against the service test process cannot share the single GPU anyway; validate against the service
itself. itself.
(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the (A gufo-era law; frank-dragoon ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
settings, and will invert the ranking.** settings, and will invert the ranking.**
Measured 2026-09-28 on frank-large at depth 32768: its corpus Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
draft widths, i.e. the same highly-predictable text every run, and 93 % of draft widths, i.e. the same highly-predictable text every run, and 93 % of
every proposal accepted. It reported 29–37 t/s decode where real traffic every proposal accepted. It reported 29–37 t/s decode where real traffic
@@ -264,12 +264,12 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
merely wedged kernel usually still answers ARP. merely wedged kernel usually still answers ARP.
1. Stop and mask (self-healing would restart it): 1. Stop and mask (self-healing would restart it):
- Dankclaw `frank-large` (root unit, halogen podman on the iGPU): - Dankclaw `frank-dragoon` (root unit, halogen podman on the iGPU):
`sudo systemctl stop frank-large && sudo systemctl mask frank-large` `sudo systemctl stop frank-dragoon && sudo systemctl mask frank-dragoon`
- Dankclaw `frank-secondary` (root unit, llama.cpp on the 3090): - Dankclaw `frank-zealot` (root unit, llama.cpp on the 3090):
`sudo systemctl stop frank-secondary && sudo systemctl mask frank-secondary` `sudo systemctl stop frank-zealot && sudo systemctl mask frank-zealot`
- Dankserver `frank-primary` (root unit, radiance on both R9700s, TP=2): - Dankserver `frank-archon` (root unit, radiance on both R9700s, TP=2):
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary` `sudo systemctl stop frank-archon && sudo systemctl mask frank-archon`
2. Bench. 2. Bench.
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
`/etc/systemd/system/` is itself a symlink into the checkout — mask `/etc/systemd/system/` is itself a symlink into the checkout — mask
@@ -287,14 +287,15 @@ The harness reaches the fleet through `.omp/agent/models.yml` in the
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
where the harness runs; there is no copy on the servers. Each provider is one where the harness runs; there is no copy on the servers. Each provider is one
HOST:PORT; each model `id` is the served alias. **Official model ids are HOST:PORT; each model `id` is the served alias. **Official model ids are
the StarCraft unit names (2026-10-08) and since 2026-10-09 they are also the the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
ids on the wire:** `frank-archon` → `192.168.0.235:7770` since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
(`FRANK_PRIMARY_SERVED_NAME`), `frank-dragoon` → `192.168.0.143:7770` `192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
(`HALOGEN_MODEL_ID`), `frank-zealot` → `192.168.0.143:7771` (`ALIAS`). The `192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
SERVICES, units, scripts, env files and runbooks keep the old names `192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
(`frank-primary`, `frank-large`, `frank-secondary`); only the served alias, scripts, env files, containers, deploy services and runbooks to match —
the `models.yml`/`config.yml` ids, the homepage cards, the OpenWebUI grants the three former service names are fully retired and exist nowhere in
and the Hermes routing were renamed. Verify a rename on `/v1/models`, never the repo; historical text in runbooks uses the current names. Verify a
rename on `/v1/models`, never
on a chat response: radiance echoes the **requested** model name, so a 200 on a chat response: radiance echoes the **requested** model name, so a 200
proves nothing about the served id (llama.cpp and halogen ignore it). proves nothing about the served id (llama.cpp and halogen ignore it).
@@ -309,16 +310,16 @@ provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
`models.yml`, then fix every `modelRoles` entry that named the old endpoint — `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
a role pointing at a provider the file does not define fails at call time. As a role pointing at a provider the file does not define fails at call time. As
of 2026-10-06 **all three endpoints accept images** (`input: [text, of 2026-10-06 **all three endpoints accept images** (`input: [text,
image]`) — `frank-large` returned to vision with the flash-server v2 swap image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
(the v2 vision tower loads beside the checkpoint; the text-only halogen (the v2 vision tower loads beside the checkpoint; the text-only halogen
27B window closed 2026-10-06) — while 27B window closed 2026-10-06) — while
`frank-primary` takes them natively (radiance runs the checkpoint's `frank-archon` takes them natively (radiance runs the checkpoint's
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary 27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
are the engine core; `--mm-max-patches` available), are the engine core; `--mm-max-patches` available),
and `frank-secondary` through and `frank-zealot` through
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
Gemma4 era used). `frank-secondary`'s `contextWindow` is Gemma4 era used). `frank-zealot`'s `contextWindow` is
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the **200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144: 2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
the harness must compact before llama.cpp answers a longer prompt with a 400. the harness must compact before llama.cpp answers a longer prompt with a 400.
@@ -333,20 +334,20 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
audio encoder, and left again with the Gemma4 retirement on 2026-10-01 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
(neither the MiMo nor the Qwen3.5 tower carries audio). (neither the MiMo nor the Qwen3.5 tower carries audio).
### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE) ### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
driven by `scripts/serve-frank-large.sh` under a **root systemd unit** driven by `scripts/serve-frank-dragoon.sh` under a **root systemd unit**
(`scripts/frank-large.service`, `User=dank`, `Restart=always`, (`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
`ExecStop=podman stop -t 60 frank-large-halogen`, the script runs `podman `ExecStop=podman stop -t 60 frank-dragoon-halogen`, the script runs `podman
run` FOREGROUND — the container's exit code is the service's exit so run` FOREGROUND — the container's exit code is the service's exit so
`Restart=always` stays real, and the engine ledger lands on disk) on port `Restart=always` stays real, and the engine ledger lands on disk) on port
7770 from the Strix Halo iGPU. Engine stdout goes to 7770 from the Strix Halo iGPU. Engine stdout goes to
`/home/dank/logs/frank_large.log` — **the boot ledger (weights resident, KV `/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**. pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
Cutover 2026-10-06 from the halogen 27B-dense era Cutover 2026-10-06 from the halogen 27B-dense era
([[frank-large-halogen27b-dflash2]], retired the same window). ([[frank-dragoon-halogen27b-dflash2]], retired the same window).
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`: Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1 `qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
@@ -368,10 +369,10 @@ sidecar is deliberately not fetched).
**Product facts vs the 27B era, both regressions FIXED:** the base is **Product facts vs the 27B era, both regressions FIXED:** the base is
**abliterated** (the 27B was a censored-base finetune; refusals were an **abliterated** (the 27B was a censored-base finetune; refusals were an
accepted trade there) and **vision is ON** (the 27B engine silently accepted trade there) and **vision is ON** (the 27B engine silently
ignored its encoder — frank-secondary owned images for exactly one day at this ignored its encoder — frank-zealot owned images for exactly one day at this
endpoint). endpoint).
**Engine laws that shape the config** (all in `scripts/frank-large.env` **Engine laws that shape the config** (all in `scripts/frank-dragoon.env`
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
policy disables the MTP head for whole requests after low acceptance and policy disables the MTP head for whole requests after low acceptance and
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
@@ -413,7 +414,7 @@ page cache beside the checkpoint — the card's quickstart arithmetic does
not bill it as resident, but READ the boot ledger and re-size if the not bill it as resident, but READ the boot ledger and re-size if the
engine prints a clamp. engine prints a clamp.
Fetch: `scripts/download-frank-large-flashnext.sh` into Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS `/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
sha256 + `--verify-only`, no token on the command line; resume goes sha256 + `--verify-only`, no token on the command line; resume goes
against the RESOLVED CDN URL because curl drops `Range` across HF's 302). against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
@@ -422,18 +423,18 @@ n-gram + vision tower, flat tokenizer with all six files as regular
files) — the 2026-10-01 crash-loop lesson. files) — the 2026-10-01 crash-loop lesson.
Runbook: Runbook:
`serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md`. `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
Rollback: revert the swap commit and redeploy — the 27B payload Rollback: revert the swap commit and redeploy — the 27B payload
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the (`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched. that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
### frank-secondary — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE) ### frank-zealot — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build **llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`, `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit** `GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
(`scripts/frank-secondary.service`, `relink_unit`ed into (`scripts/frank-zealot.service`, `relink_unit`ed into
`/etc/systemd/system/`), serving `/etc/systemd/system/`), serving
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K** `HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no (7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
@@ -489,7 +490,7 @@ tokens). With `--parallel 2` a compaction request no longer serializes behind
a live generation (the Gemma4 era's `--parallel 1` law). a live generation (the Gemma4 era's `--parallel 1` law).
Runbook: Runbook:
`serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md`. `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md`.
Rollback: revert the swap commit — the Gemma4 payload Rollback: revert the swap commit — the Gemma4 payload
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload (`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
(`/home/dank/models/MiMo-Q6K/`), the Swift payload (`/home/dank/models/MiMo-Q6K/`), the Swift payload
@@ -497,7 +498,7 @@ Rollback: revert the swap commit — the Gemma4 payload
(`/home/dank/models/Ornith-Q8/`) all remain on disk. (`/home/dank/models/Ornith-Q8/`) all remain on disk.
### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE) ### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
The **radiance engine** (StillDeadcode, Apache-2.0; image The **radiance engine** (StillDeadcode, Apache-2.0; image
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13** `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
@@ -556,7 +557,7 @@ pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against
`RLIMIT_MEMLOCK` — the TTM floor is set just above it. `RLIMIT_MEMLOCK` — the TTM floor is set just above it.
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=140000`) refuses to
start until both knobs are raised in the stop window (sysfs write first, start until both knobs are raised in the stop window (sysfs write first,
kernel cmdline + reboot only if read-only). kernel cmdline + reboot only if read-only).
@@ -570,11 +571,11 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
`--mm-max-patches`); `/metrics` carries vLLM-compatible names + `--mm-max-patches`); `/metrics` carries vLLM-compatible names +
`radiance:*` families — the Dankclaw Prometheus scrape is re-added. `radiance:*` families — the Dankclaw Prometheus scrape is re-added.
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`. Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`. Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`. Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`. Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
## Key Lessons Learned ## Key Lessons Learned
Durable operational lessons from the fleet's history. Full detail in the Durable operational lessons from the fleet's history. Full detail in the
@@ -736,11 +737,11 @@ linked runbooks.
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff` - Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
of the old one, or live pages must land in RAM that `MemAvailable` of the old one, or live pages must land in RAM that `MemAvailable`
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
the co-tenants, not frank-large — pinned weights don't swap, and paging the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
the KV pool is worse than the OOM it prevents. the KV pool is worse than the OOM it prevents.
- Host swap state is invisible to `deploy.sh` (no repo file) — check - Host swap state is invisible to `deploy.sh` (no repo file) — check
`swapon --show` before reasoning about headroom. `swapon --show` before reasoning about headroom.
- **Silent RAM corruption outranks every software theory.** frank-primary's - **Silent RAM corruption outranks every software theory.** frank-archon's
~40k-token output corruption (2026-10-04/05, repetition/garbage in long ~40k-token output corruption (2026-10-04/05, repetition/garbage in long
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
@@ -763,7 +764,7 @@ linked runbooks.
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through `LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
podman to runc without any raise). Upstream halogen's own run lines pass podman to runc without any raise). Upstream halogen's own run lines pass
NO `--ulimit` at all — copy vendor run lines before adding flags NO `--ulimit` at all — copy vendor run lines before adding flags
(measured 2026-10-05 on the frank-large cutover). (measured 2026-10-05 on the frank-dragoon cutover).
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs - **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
`third_party/composable_kernel/include/ck/config.h` generated manually `third_party/composable_kernel/include/ck/config.h` generated manually
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects (untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
@@ -789,7 +790,7 @@ linked runbooks.
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect. 121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
Record the child pid and kill it in a `finally`, then prove `free` came back Record the child pid and kill it in a `finally`, then prove `free` came back
before believing a number. Kill by pid — never `pkill -f llama-server`, before believing a number. Kill by pid — never `pkill -f llama-server`,
which also hits frank-secondary. which also hits frank-zealot.
**llama.cpp serving** **llama.cpp serving**
@@ -813,7 +814,7 @@ linked runbooks.
builds its context against whatever model the draft context holds. Two builds its context against whatever model the draft context holds. Two
context-backed types cannot share one process, and MTP's head only loads context-backed types cannot share one process, and MTP's head only loads
when MTP **is** the draft model. **A LEGAL chain can still lose on a when MTP **is** the draft model. **A LEGAL chain can still lose on a
recurrent model:** `draft-mtp,ngram-mod` on frank-large's qwen35 hybrid recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
logs `recurrent state rollback is not compatible with 'ngram-mod' - logs `recurrent state rollback is not compatible with 'ngram-mod' -
disabling rollback support`, and MTP verification depends on that rollback, disabling rollback support`, and MTP verification depends on that rollback,
@@ -855,7 +856,7 @@ linked runbooks.
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
model — the sweep's or the seam's — can see it.) model — the sweep's or the seam's — can see it.)
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so - **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
frank-secondary ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the frank-zealot ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 → ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a 2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
@@ -864,9 +865,9 @@ linked runbooks.
than the ubatch cannot see it. The logical `--batch-size` was again free. And than the ubatch cannot see it. The logical `--batch-size` was again free. And
the predecessor's "ubatch 2048 failed the scratch boot" was a shape the predecessor's "ubatch 2048 failed the scratch boot" was a shape
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
it by restarting the SERVICE through a transient `FRANK_SECONDARY_ENV_FILE` it by restarting the SERVICE through a transient `FRANK_ZEALOT_ENV_FILE`
drop-in, never a scratch port — one card holds one server. See drop-in, never a scratch port — one card holds one server. See
[[frank-secondary-llama-mimo-q6k-2x200704]]. [[frank-zealot-llama-mimo-q6k-2x200704]].
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
@@ -888,7 +889,7 @@ linked runbooks.
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
(`400 exceed_context_size_error: request (184001 tokens) exceeds the (`400 exceed_context_size_error: request (184001 tokens) exceeds the
available context size (180224 tokens)`) even when the pool has room. So available context size (180224 tokens)`) even when the pool has room. So
frank-secondary's 2 → 3 reslice moved each conversation from the model's native frank-zealot's 2 → 3 reslice moved each conversation from the model's native
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or 262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
the harness would discover the ceiling as a 400 on a long conversation. Side the harness would discover the ceiling as a 400 on a long conversation. Side
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
@@ -899,9 +900,9 @@ linked runbooks.
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
itself is a further measured −10 %, weight-bandwidth, not slot geometry). itself is a further measured −10 %, weight-bandwidth, not slot geometry).
All moves measured; both slot directions cost or buy window, never pool. All moves measured; both slot directions cost or buy window, never pool.
See [[frank-secondary-llama-mimo-q6k-2x200704]]. See [[frank-zealot-llama-mimo-q6k-2x200704]].
- **Price a VRAM budget from two boots, never from a formula — and an mmproj - **Price a VRAM budget from two boots, never from a formula — and an mmproj
costs ~1.5x its file size.** The frank-secondary 3090 fit was solved from two costs ~1.5x its file size.** The frank-zealot 3090 fit was solved from two
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒ measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV **non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file 14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
@@ -909,24 +910,24 @@ linked runbooks.
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+ ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then "a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
solve `KV ≤ card − non-KV − headroom` and round to a multiple of solve `KV ≤ card − non-KV − headroom` and round to a multiple of
`PARALLEL × 4096`. Recorded in `scripts/frank-secondary.env` so the next resize `PARALLEL × 4096`. Recorded in `scripts/frank-zealot.env` so the next resize
starts from measurement. starts from measurement.
- **`images` and `image_url` are NOT interchangeable: only the content part - **`images` and `image_url` are NOT interchangeable: only the content part
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
model still invents an answer if the client used the legacy top-level model still invents an answer if the client used the legacy top-level
`images` field on `/v1/chat/completions` — frank-secondary answered "Brown." for `images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
failure is silent because a hallucinated color is a plausible answer. Probe failure is silent because a hallucinated color is a plausible answer. Probe
vision with a solid color and expect an exact word. vision with a solid color and expect an exact word.
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with - **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that `500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
then served frank-large: every tool-using agent on that endpoint was broken then served frank-dragoon: every tool-using agent on that endpoint was broken
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
`--jinja` on and the serve script pins it; halogen also accepts `tools` and `--jinja` on and the serve script pins it; halogen also accepts `tools` and
returns `tool_calls`. Check tool support before routing a role at an endpoint. returns `tool_calls`. Check tool support before routing a role at an endpoint.
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same - **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
option and the same ~12x prefill collapse described in the ROCm note above. option and the same ~12x prefill collapse described in the ROCm note above.
f16 KV needs nothing; frank-secondary ran f16 for that reason until f16 KV needs nothing; frank-zealot ran f16 for that reason until
2026-09-17, when `build-cuda` was rebuilt with the option ON so the 2026-09-17, when `build-cuda` was rebuilt with the option ON so the
endpoint could take q8_0 KV. endpoint could take q8_0 KV.
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both - **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
@@ -944,7 +945,7 @@ linked runbooks.
raises on anything else — HTTP 500, not a clamp — so the harness's levels raises on anything else — HTTP 500, not a clamp — so the harness's levels
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`). must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
A role whose level is unmapped fails on its FIRST request, and the failure A role whose level is unmapped fails on its FIRST request, and the failure
names Jinja, not the model: every subagent on `frank-secondary` died with names Jinja, not the model: every subagent on `frank-zealot` died with
`Unexpected reasoning effort high` until `high: xhigh` was added on `Unexpected reasoning effort high` until `high: xhigh` was added on
2026-09-16. Halogen accepts the wider set, so the two entries legitimately 2026-09-16. Halogen accepts the wider set, so the two entries legitimately
differ. differ.
@@ -964,7 +965,7 @@ linked runbooks.
times (`RuntimeError: No HIP GPUs are available`). times (`RuntimeError: No HIP GPUs are available`).
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its - SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
ports off the gluetun service. ports off the gluetun service.
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-large, - `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind 2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`. in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
@@ -1018,9 +1019,9 @@ Active:
| Service | Runbook | | Service | Runbook |
|---------|---------| |---------|---------|
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` | | frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
| frank-secondary (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md` | | frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` | | frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` | | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` | | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1028,41 +1029,41 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook | | Runtime | Host | Window | Runbook |
|---------|------|--------|---------| |---------|------|--------|---------|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-secondary`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-secondary-llama-gemma4-q4km-200704.md` | | llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` | | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` | | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` | | llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-zealot-llama-mimo-q6k-2x200704.md` |
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` | | llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-secondary`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` | | llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-zealot`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-dragoon-rocmfpx-q8agent-mtp.md` |
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` | | gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-dragoon-gufo-qwen38-27b.md` |
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` | | vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-dragoon`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-dragoon-vllm-mimo-v26-bf16.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-secondary`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-secondary-llama-mimo-q6k-2x200704.md`) | `frank-secondary-llama-mimo-q6k.md` | | llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-zealot`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-zealot-llama-mimo-q6k-2x200704.md`) | `frank-zealot-llama-mimo-q6k.md` |
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-secondary-llama-mimo-q5km.md` | | llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-zealot-llama-mimo-q5km.md` |
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-secondary`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-secondary-sglang-mimo-v26.md` | | SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-zealot`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-zealot-sglang-mimo-v26.md` |
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`| |tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-archon`)|Dankserver|2026-09-30 → 10-01|`frank-archon-glm53-flash-w4a16.md`|
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` | | llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-21 → 09-28 | `frank-dragoon-unsloth-qwen38-27b-q8xl.md` |
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` | | R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-archon`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-archon-r9v-radiance-qwen38-uncensored.md` |
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` | | llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-archon`) | Dankserver | 2026-09-26 → 09-27 | `frank-archon-llamacpp-qwen38-flash-next-q8.md` |
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` | | ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-dragoon`) | Dankclaw | 2026-09-15 → 09-16 | `frank-dragoon-ik-uncensored.md` |
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` | | vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-primary`) | Dankserver | 2026-09-26 (same-day swap) | `frank-primary-lucebox-ds4.md` | | Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-archon`) | Dankserver | 2026-09-26 (same-day swap) | `frank-archon-lucebox-ds4.md` |
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` | | kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` | | halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-dragoon`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-large`) | Dankclaw | 2026-09-19 → 09-21 | `frank-large-ciru-orca.md` | | CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-dragoon`) | Dankclaw | 2026-09-19 → 09-21 | `frank-dragoon-ciru-orca.md` |
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-unsloth-qwen38-flash-next.md` | | unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-unsloth-qwen38-flash-next.md` |
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-kairic-edge-qwen38-27b.md` | | Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-kairic-edge-qwen38-27b.md` |
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-secondary`) | Dankclaw | 2026-09-16 → 09-17 | `frank-secondary-qwen38-gsq-rco.md` | | ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-zealot`) | Dankclaw | 2026-09-16 → 09-17 | `frank-zealot-qwen38-gsq-rco.md` |
| vLLM Gemma4-12B w4a16 (`frank-secondary`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-secondary-vllm-gemma4.md` | | vLLM Gemma4-12B w4a16 (`frank-zealot`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-zealot-vllm-gemma4.md` |
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-secondary`) | Dankclaw | 2026-09-17 → 10-01 | `frank-secondary-gemma4-12b.md` | | llama.cpp Gemma4-12B QAT Q4_K_M (`frank-zealot`) | Dankclaw | 2026-09-17 → 10-01 | `frank-zealot-gemma4-12b.md` |
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` | | agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` | | eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` | | Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` | | CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` | | DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` | | FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
| FreeToken (official + Huihui FP8, `frank-secondary`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` | | FreeToken (official + Huihui FP8, `frank-zealot`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` | | FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` | | ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` | | llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
+2 -2
View File
@@ -13,7 +13,7 @@ one deploy script.
| Host | User | IP | Podman stack | LLM service | LLM port | | Host | User | IP | Podman stack | LLM service | LLM port |
|------|------|----|--------------|-------------|----------| |------|------|----|--------------|-------------|----------|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 | | dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 | | dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
## Active Services ## Active Services
@@ -30,7 +30,7 @@ one deploy script.
| Gitea | dankclaw | gitea.dankclaw.local | 3001 | | Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 | | Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 | | Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 | | frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
## Structure ## Structure
+9 -9
View File
@@ -1,26 +1,26 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# autoresearch.sh — canonical frank-primary benchmark entry point (dev box). # autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
# #
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live # The service runs on dankserver (192.168.0.235:7770), the two R9700s live
# there, and the benchmark must own those cards, so the whole stop -> boot # there, and the benchmark must own those cards, so the whole stop -> boot
# scratch -> bench -> restore sequence runs ON dankserver (see # scratch -> bench -> restore sequence runs ON dankserver (see
# scripts/autoresearch-frank-primary.sh). This wrapper: # scripts/autoresearch-frank-archon.sh). This wrapper:
# 1. copies the candidate config (the repo's scripts/frank-primary.env, the # 1. copies the candidate config (the repo's scripts/frank-archon.env, the
# file the autoresearch loop edits) and the bench script to dankserver, # file the autoresearch loop edits) and the bench script to dankserver,
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and # 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
# exit code back here verbatim. # exit code back here verbatim.
# It deliberately does NOT push to origin: the scp dirties dankserver's # It deliberately does NOT push to origin: the scp dirties dankserver's
# working tree, and the loop's uncommitted candidate config must not be # working tree, and the loop's uncommitted candidate config must not be
# pushed. Only this file and scripts/autoresearch-frank-primary.sh are # pushed. Only this file and scripts/autoresearch-frank-archon.sh are
# committed; the candidate env rides over ssh each run. # committed; the candidate env rides over ssh each run.
set -uo pipefail set -uo pipefail
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REMOTE=dankserver REMOTE=dankserver
REMOTE_REPO=/home/dank/serverAppConfigs REMOTE_REPO=/home/dank/serverAppConfigs
scp -q "$DIR/scripts/frank-primary.env" \ scp -q "$DIR/scripts/frank-archon.env" \
"$REMOTE:$REMOTE_REPO/scripts/frank-primary.env" || { echo "scp env failed" >&2; exit 1; } "$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
scp -q "$DIR/scripts/autoresearch-frank-primary.sh" \ scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-primary.sh" || { echo "scp script failed" >&2; exit 1; } "$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-primary.sh $REMOTE_REPO/scripts/frank-primary.env" ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
+1 -1
View File
File diff suppressed because one or more lines are too long
@@ -1,9 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-primary-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8 # download-frank-archon-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the # payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
# vLLM frank-primary deployment on dankserver. # vLLM frank-archon deployment on dankserver.
# #
# bash scripts/download-frank-primary-fp8.sh [dest_dir] # bash scripts/download-frank-archon-fp8.sh [dest_dir]
# #
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries # The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an # per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash # Download the z-lab Qwen3.5-9B-DFlash drafter for frank-dragoon's DFlash
# speculative-decode arm (vLLM --speculative-config method=dflash). Public # speculative-decode arm (vLLM --speculative-config method=dflash). Public
# repo — no auth header on the command line. # repo — no auth header on the command line.
# #
@@ -10,7 +10,7 @@
# the draft for that target. # the draft for that target.
# #
# Usage: # Usage:
# scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only] # scripts/download-frank-dragoon-dflash.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
REPO="z-lab/Qwen3.5-9B-DFlash" REPO="z-lab/Qwen3.5-9B-DFlash"
@@ -1,14 +1,14 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large # Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-dragoon
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the # (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
# command line (a Bearer token in argv is world-readable via /proc). # command line (a Bearer token in argv is world-readable via /proc).
# #
# Per-file FRESH downloads, no resume: the house rule (see # Per-file FRESH downloads, no resume: the house rule (see
# download-frank-secondary.sh); -f/--fail-with-body plus size+sha256 against the # download-frank-zealot.sh); -f/--fail-with-body plus size+sha256 against the
# Hub LFS oid is what makes an access error fail instead of "succeeding". # Hub LFS oid is what makes an access error fail instead of "succeeding".
# #
# Usage: # Usage:
# scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only] # scripts/download-frank-dragoon-mimo.sh [DEST_DIR] [--verify-only]
# #
# The verified set below is exactly what the 2026-10-01 bench window served: # The verified set below is exactly what the 2026-10-01 bench window served:
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest). # 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
@@ -1,14 +1,14 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model # Download the source Qwen3.5-9B checkpoint — the ONLY place this model
# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the # family's MTP head (15 mtp.* tensors) exists. frank-dragoon runs it as the
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero # vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
# mtp tensors; that is why SGLang's NEXTN on frank-secondary draws 0.00 # mtp tensors; that is why SGLang's NEXTN on frank-zealot draws 0.00
# acceptance). Public repo — no auth header on the command line. # acceptance). Public repo — no auth header on the command line.
# #
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...). # NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
# #
# Usage: # Usage:
# scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only] # scripts/download-frank-dragoon-qwen35.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
REPO="Qwen/Qwen3.5-9B" REPO="Qwen/Qwen3.5-9B"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-secondary # Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-zealot
# (SGLang, RTX 3090). Public repo — no auth header on the command line. # (SGLang, RTX 3090). Public repo — no auth header on the command line.
# #
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB # Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
@@ -7,7 +7,7 @@
# the house rule. # the house rule.
# #
# Usage: # Usage:
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only] # scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
# #
# LFS shards are verified against the Hub LFS sha256; non-LFS text files # LFS shards are verified against the Hub LFS sha256; non-LFS text files
# against their declared byte size. Idempotent: complete files are skipped. # against their declared byte size. Idempotent: complete files are skipped.
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# bench-frank-large.sh — 64k-context benchmark for the frank-default-large stack. # bench-frank-dragoon.sh — 64k-context benchmark for the frank-default-large stack.
# #
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on # Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
# a single backend at 64k context. It starts llama-server on a scratch port, # a single backend at 64k context. It starts llama-server on a scratch port,
@@ -7,15 +7,15 @@
# prompt, and prints tokens/sec. # prompt, and prints tokens/sec.
# #
# Usage: # Usage:
# ./scripts/bench-frank-large.sh <ROCm0|Vulkan0> [--generations N] # ./scripts/bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]
# #
# Example: # Example:
# ./scripts/bench-frank-large.sh ROCm0 # ./scripts/bench-frank-dragoon.sh ROCm0
# ./scripts/bench-frank-large.sh Vulkan0 --generations 3 # ./scripts/bench-frank-dragoon.sh Vulkan0 --generations 3
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEVICE="${1:?usage: bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]}" DEVICE="${1:?usage: bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]}"
GENERATIONS=1 GENERATIONS=1
if [ "${2:-}" = "--generations" ]; then if [ "${2:-}" = "--generations" ]; then
GENERATIONS="${3:?--generations needs a count}" GENERATIONS="${3:?--generations needs a count}"
+1 -1
View File
@@ -41,7 +41,7 @@ SERVICE_HEALTH[dankclaw]="http://127.0.0.1:7770/health"
# Drop page caches before restart: with the model's pages fully cached, the # Drop page caches before restart: with the model's pages fully cached, the
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md. # KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
DROP_CACHES[dankclaw]=1 DROP_CACHES[dankclaw]=1
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-large). # Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-dragoon).
# When present, deploy.sh uses it instead of inline sudo commands. # When present, deploy.sh uses it instead of inline sudo commands.
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh" HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
# Health wait in iterations of 2s. Cold loads after the cache drop measured # Health wait in iterations of 2s. Cold loads after the cache drop measured
+2 -2
View File
@@ -1,6 +1,6 @@
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-large-ling). # Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-dragoon-ling).
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch # This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
# back: systemctl stop frank-large-ling && systemctl start frank-default-large. # back: systemctl stop frank-dragoon-ling && systemctl start frank-default-large.
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT. # Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
# #
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical): # Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-large-ling TRIAL) Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-dragoon-ling TRIAL)
After=network.target After=network.target
[Service] [Service]
+1 -1
View File
@@ -8,7 +8,7 @@
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same # Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
# GGUF (block 42) - no separate --spec-draft-model needed. # GGUF (block 42) - no separate --spec-draft-model needed.
# #
# Tie-back: to restore the Qwen3.6-27B service, stop frank-large-ling and # Tie-back: to restore the Qwen3.6-27B service, stop frank-dragoon-ling and
# start frank-default-large (its env/unit are committed and unchanged). # start frank-default-large (its env/unit are committed and unchanged).
# #
# Tunables come from frank-default-large.ling.env in this directory unless # Tunables come from frank-default-large.ling.env in this directory unless
@@ -4,11 +4,11 @@
# Runtime: llama.cpp llama-server (>= b10411). # Runtime: llama.cpp llama-server (>= b10411).
# #
# Tunables come from frank-default-large.env in this directory unless the # Tunables come from frank-default-large.env in this directory unless the
# FRANK_LARGE_ENV_FILE environment variable overrides that path. # FRANK_DRAGOON_ENV_FILE environment variable overrides that path.
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}" ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
if [[ -f "$ENV_FILE" ]]; then if [[ -f "$ENV_FILE" ]]; then
set -a set -a
# shellcheck disable=SC1090 # shellcheck disable=SC1090
+1 -1
View File
File diff suppressed because one or more lines are too long
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Decode throughput benchmark for frank-large. Decode throughput benchmark for frank-dragoon.
Starts the server with a given --threads-batch value, sends an 8K+ token prompt, Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
generates 256 tokens, measures generation tok/s. generates 256 tokens, measures generation tok/s.
@@ -13,12 +13,12 @@ SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
PORT = 7770 PORT = 7770
N_GEN = 256 N_GEN = 256
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt" PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-large.env") ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
SERVER_DIR = os.path.dirname(SERVE) SERVER_DIR = os.path.dirname(SERVE)
ROCM_PATH = "/opt/rocm" ROCM_PATH = "/opt/rocm"
def make_env(): def make_env():
"""Build the same env as serve-frank-large.sh.""" """Build the same env as serve-frank-dragoon.sh."""
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"} env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
# Source env file for tunables # Source env file for tunables
@@ -59,7 +59,7 @@ def wait_ready():
def send_request(prompt, max_tokens=N_GEN): def send_request(prompt, max_tokens=N_GEN):
"""Send chat completion, measure generation throughput.""" """Send chat completion, measure generation throughput."""
body = json.dumps({ body = json.dumps({
"model": "frank-large", "model": "frank-dragoon",
"messages": [{"role": "user", "content": prompt}], "messages": [{"role": "user", "content": prompt}],
"max_tokens": max_tokens, "max_tokens": max_tokens,
"stream": False, "stream": False,
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Benchmark n-gram speculative decoding vs no-spec for frank-large. Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
Tests prompt-processing throughput with 16K token prompts (manageable request size). Tests prompt-processing throughput with 16K token prompts (manageable request size).
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy). Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
@@ -103,7 +103,7 @@ def start_server(port, spec_type):
def run_test(prompt_text, spec_type, port): def run_test(prompt_text, spec_type, port):
"""Send a request and extract prompt processing metrics.""" """Send a request and extract prompt processing metrics."""
data = json.dumps({ data = json.dumps({
"model": "frank-large", "model": "frank-dragoon",
"messages": [{"role": "user", "content": prompt_text}], "messages": [{"role": "user", "content": prompt_text}],
"max_tokens": TEST_TOKENS, "max_tokens": TEST_TOKENS,
"stream": False, "stream": False,
@@ -189,7 +189,7 @@ def main():
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes") print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
print("=" * 60) print("=" * 60)
print(" N-gram Mod Benchmark — frank-large") print(" N-gram Mod Benchmark — frank-dragoon")
print(f" STRIX ROCmFP4 · 64K context · 16K prompts") print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
print("=" * 60) print("=" * 60)
@@ -1,19 +1,19 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Benchmark n-gram speculative decoding vs no-spec for frank-large. # Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
# #
# This tests prompt-processing throughput at 64k context with two distinct # This tests prompt-processing throughput at 64k context with two distinct
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch # prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms. # ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
# #
# Run on Dankclaw after stopping the production service: # Run on Dankclaw after stopping the production service:
# sudo systemctl stop frank-large # sudo systemctl stop frank-dragoon
# ./scripts/bench-ngram-mod.sh # ./scripts/bench-ngram-mod.sh
# sudo systemctl start frank-large # sudo systemctl start frank-dragoon
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}" ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a set -a; source "$ENV_FILE"; set +a
fi fi
@@ -184,7 +184,7 @@ run_single_test() {
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \ response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d "{ -d "{
\"model\": \"frank-large\", \"model\": \"frank-dragoon\",
\"prompt\": \"$prompt_text\", \"prompt\": \"$prompt_text\",
\"max_tokens\": 128, \"max_tokens\": 128,
\"temperature\": 0, \"temperature\": 0,
@@ -220,7 +220,7 @@ run_single_test() {
} }
echo "╔══════════════════════════════════════════════════════════╗" echo "╔══════════════════════════════════════════════════════════╗"
echo "║ N-gram Mod Benchmark — frank-large ║" echo "║ N-gram Mod Benchmark — frank-dragoon ║"
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║" echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
echo "╚══════════════════════════════════════════════════════════╝" echo "╚══════════════════════════════════════════════════════════╝"
echo "" echo ""
@@ -15,9 +15,9 @@ import time
import urllib.request import urllib.request
import urllib.error import urllib.error
SERVICE = "frank-large" SERVICE = "frank-dragoon"
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-large.env" ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ALIAS = "frank-large" ALIAS = "frank-dragoon"
DEFAULT_HOST = "localhost" DEFAULT_HOST = "localhost"
DEFAULT_PORT = 7770 DEFAULT_PORT = 7770
DEFAULT_MAX_TOKENS = 16384 DEFAULT_MAX_TOKENS = 16384
@@ -1,4 +1,4 @@
# Tunables for frank-large (mradermacher i1-Q4_K_M of # Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw, # orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15). # RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
# #
@@ -65,7 +65,7 @@ MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
# Alias + network. # Alias + network.
ALIAS=frank-large ALIAS=frank-dragoon
BIND_ADDR=0.0.0.0 BIND_ADDR=0.0.0.0
LISTEN_PORT=7770 LISTEN_PORT=7770
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-large) Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -12,8 +12,8 @@ Group=dank
# budget still needs room. Parity with the halogen quadlet's # budget still needs room. Parity with the halogen quadlet's
# Ulimit=memlock=-1:-1. # Ulimit=memlock=-1:-1.
LimitMEMLOCK=infinity LimitMEMLOCK=infinity
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env" Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
Restart=on-failure Restart=on-failure
RestartSec=10 RestartSec=10
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-large # Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-dragoon
# via ik_llama.cpp on Dankclaw. # via ik_llama.cpp on Dankclaw.
# #
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE): # Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
@@ -26,7 +26,7 @@
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}" ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [[ -f "$ENV_FILE" ]]; then if [[ -f "$ENV_FILE" ]]; then
set -a set -a
# shellcheck disable=SC1090 # shellcheck disable=SC1090
@@ -66,7 +66,7 @@ fi
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file # CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
# sets it. Refuse to run without it. # sets it. Refuse to run without it.
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
echo "frank-large: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2 echo "frank-dragoon: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
exit 1 exit 1
fi fi
@@ -112,7 +112,7 @@ fi
: "${SPEC_DRAFT_POLL_BATCH:=1}" : "${SPEC_DRAFT_POLL_BATCH:=1}"
# Text only: the server must not try to autoload the vision tower. Vision # Text only: the server must not try to autoload the vision tower. Vision
# stays on frank-large (Dankclaw), which is where the fleet routes images. # stays on frank-dragoon (Dankclaw), which is where the fleet routes images.
NO_MMPROJ="${NO_MMPROJ:-1}" NO_MMPROJ="${NO_MMPROJ:-1}"
export HIP_VISIBLE_DEVICES export HIP_VISIBLE_DEVICES
+1 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""autoresearch-bench.py — deterministic greedy battery for frank-large. """autoresearch-bench.py — deterministic greedy battery for frank-dragoon.
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770. Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
Fixed embedded corpus, temperature 0, streaming so decode speed is measured Fixed embedded corpus, temperature 0, streaming so decode speed is measured
@@ -1,12 +1,12 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# autoresearch-frank-primary.sh — RUNS ON DANKSERVER, driven over ssh by the # autoresearch-frank-archon.sh — RUNS ON DANKSERVER, driven over ssh by the
# dev-box autoresearch.sh. It is the canonical frank-primary benchmark entry # dev-box autoresearch.sh. It is the canonical frank-archon benchmark entry
# point for the autoresearch loop. # point for the autoresearch loop.
# #
# Method (mandated by the Benchmarking Policy): the production service serves # Method (mandated by the Benchmarking Policy): the production service serves
# real users and shares the two R9700s, so a bench beside it starves the GPUs # real users and shares the two R9700s, so a bench beside it starves the GPUs
# and corrupts results. This script therefore: # and corrupts results. This script therefore:
# 1. stops the production frank-primary unit (root, relink symlink: stop, # 1. stops the production frank-archon unit (root, relink symlink: stop,
# NOT mask — mask refuses to replace a symlinked unit file), # NOT mask — mask refuses to replace a symlinked unit file),
# 2. boots a scratch radiance container named `arbench` on the SAME two # 2. boots a scratch radiance container named `arbench` on the SAME two
# cards with the candidate engine argv (the env file the loop edits), # cards with the candidate engine argv (the env file the loop edits),
@@ -18,7 +18,7 @@
# prefill and decode rates for that single request. # prefill and decode rates for that single request.
# 5. prints METRIC lines and tears the scratch down, # 5. prints METRIC lines and tears the scratch down,
# 6. restores production (systemctl start), via a trap so a failed bench # 6. restores production (systemctl start), via a trap so a failed bench
# never leaves frank-primary down (the stale-probe-tail lesson). # never leaves frank-archon down (the stale-probe-tail lesson).
# #
# Determinism: the prompt is built from a fixed, sorted corpus of repo files # Determinism: the prompt is built from a fixed, sorted corpus of repo files
# with a fixed random seed, so prompt_n is stable across runs. Sampling is # with a fixed random seed, so prompt_n is stable across runs. Sampling is
@@ -32,10 +32,10 @@
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY # /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
# differences: the container name (arbench, so prod's identity is untouched) # differences: the container name (arbench, so prod's identity is untouched)
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold # and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
# prefill is cold — the prod tier at kvcache/frank-primary is left intact. # prefill is cold — the prod tier at kvcache/frank-archon is left intact.
set -uo pipefail set -uo pipefail
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-primary.env}" ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
SCRATCH=arbench SCRATCH=arbench
PORT=7770 PORT=7770
HEALTH="http://127.0.0.1:$PORT/health" HEALTH="http://127.0.0.1:$PORT/health"
@@ -45,7 +45,7 @@ LOG=/home/dank/logs/arbench.log
restore() { restore() {
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
sudo systemctl start frank-primary >/dev/null 2>&1 || true sudo systemctl start frank-archon >/dev/null 2>&1 || true
} }
trap restore EXIT trap restore EXIT
@@ -57,13 +57,13 @@ fi
set -a; . "$ENV_FILE"; set +a set -a; . "$ENV_FILE"; set +a
# --- stop production; the scratch needs both cards to itself ---------------- # --- stop production; the scratch needs both cards to itself ----------------
sudo systemctl stop frank-primary >/dev/null 2>&1 || true sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------ # --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
FRESH_KV="$FRANK_PRIMARY_STATE_DIR/kvcache/$SCRATCH" FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV" rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune" mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# --- boot the scratch: the production run line, scratch name + fresh KV ------ # --- boot the scratch: the production run line, scratch name + fresh KV ------
podman run -d --name "$SCRATCH" \ podman run -d --name "$SCRATCH" \
@@ -72,31 +72,31 @@ podman run -d --name "$SCRATCH" \
--security-opt seccomp=unconfined \ --security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \ --ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \ --device /dev/kfd --device /dev/dri \
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \ -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_PRIMARY_STATE_DIR":/data \ -v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \ -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \ -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_PRIMARY_IMAGE" \ "$FRANK_ARCHON_IMAGE" \
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \ --model "/models/$FRANK_ARCHON_MODEL_FILE" \
--host :: --port "$PORT" \ --host :: --port "$PORT" \
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \ --served-model-name "$FRANK_ARCHON_SERVED_NAME" \
--override-chat-template /templates/template.jinja \ --override-chat-template /templates/template.jinja \
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \ --tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \ --max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \ --max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
--placement "$FRANK_PRIMARY_PLACEMENT" \ --placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \ --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \ --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \ --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \ --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \ --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \ --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir "/data/kvcache/$SCRATCH" \ --prefix-cache-dir "/data/kvcache/$SCRATCH" \
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \ --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \ --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \ --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \ --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \ ${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
> "$LOG" 2>&1 > "$LOG" 2>&1
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
@@ -1,19 +1,19 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# frank-primary-arms.sh — run a SEQUENCE of frank-primary bench arms and keep # frank-archon-arms.sh — run a SEQUENCE of frank-archon bench arms and keep
# every arm's evidence. Runs on the operator box; each arm is one full window # every arm's evidence. Runs on the operator box; each arm is one full window
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure -> # of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
# restore production and verify), so the fleet is only ever down for one arm at # restore production and verify), so the fleet is only ever down for one arm at
# a time and the production unit is back up between arms. # a time and the production unit is back up between arms.
# #
# An arm is the repo's own frank-primary.env plus an OVERLAY of engine knobs # An arm is the repo's own frank-archon.env plus an OVERLAY of engine knobs
# appended after it (later assignment wins when the serve script sources it), # appended after it (later assignment wins when the serve script sources it),
# staged through FRANK_PRIMARY_AR_ENV_FILE so the checkout is never mutated and # staged through FRANK_ARCHON_AR_ENV_FILE so the checkout is never mutated and
# each arm gets its own fingerprint (the memo can never answer one arm with # each arm gets its own fingerprint (the memo can never answer one arm with
# another arm's numbers). # another arm's numbers).
# #
# Usage: scripts/autoresearch/frank-primary-arms.sh [arm ...] # Usage: scripts/autoresearch/frank-archon-arms.sh [arm ...]
# scripts/autoresearch/frank-primary-arms.sh --list # scripts/autoresearch/frank-archon-arms.sh --list
# Output, under /tmp/frank-primary-arms-<UTC stamp>/ (override with ARMS_DIR): # Output, under /tmp/frank-archon-arms-<UTC stamp>/ (override with ARMS_DIR):
# #
# <arm>.out full autoresearch output (metrics + engine evidence) # <arm>.out full autoresearch output (metrics + engine evidence)
# <arm>.env the exact staged env # <arm>.env the exact staged env
@@ -26,8 +26,8 @@ set -uo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)" REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
BASE_ENV="$REPO/scripts/frank-primary.env" BASE_ENV="$REPO/scripts/frank-archon.env"
ARMS_DIR="${ARMS_DIR:-/tmp/frank-primary-arms-$(date -u +%Y%m%dT%H%M%SZ)}" ARMS_DIR="${ARMS_DIR:-/tmp/frank-archon-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
# --- arms ------------------------------------------------------------------ # --- arms ------------------------------------------------------------------
# baseline: the shipped shape E, measured with THIS harness so the comparison is # baseline: the shipped shape E, measured with THIS harness so the comparison is
@@ -45,18 +45,18 @@ LADDER3='{"cudagraph_capture_sizes":[4,8,12],"max_cudagraph_capture_size":12,"in
# the engine as {temperature:0.7}. # the engine as {temperature:0.7}.
GEN07='{"temperature":0.7}' GEN07='{"temperature":0.7}'
OVER_B=( OVER_B=(
"FRANK_PRIMARY_SPEC='$SPEC3'" "FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'" "FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'" 'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
) )
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory # B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
# OFF. It is the fallback if B dies at graph capture, and it prices the arena # OFF. It is the fallback if B dies at graph capture, and it prices the arena
# that precise memory buys (7,522 slots -> ~6,496). # that precise memory buys (7,522 slots -> ~6,496).
OVER_B2=( OVER_B2=(
"FRANK_PRIMARY_SPEC='$SPEC3'" "FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'" "FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'" 'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
'FRANK_PRIMARY_PRECISE_MEMORY=' 'FRANK_ARCHON_PRECISE_MEMORY='
) )
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util # C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
# 0.95. This is also the direct test of the theory that pinning the capture # 0.95. This is also the direct test of the theory that pinning the capture
@@ -64,12 +64,12 @@ OVER_B2=(
# the constant 3, which is NOT in the default ladder; 4 is). # the constant 3, which is NOT in the default ladder; 4 is).
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}' LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
OVER_C=( OVER_C=(
"FRANK_PRIMARY_SPEC='$SPEC3'" "FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER16'" "FRANK_ARCHON_COMPCONFIG='$LADDER16'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'" 'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
'FRANK_PRIMARY_SEQS=16' 'FRANK_ARCHON_SEQS=16'
'FRANK_PRIMARY_MNBT=4096' 'FRANK_ARCHON_MNBT=4096'
'FRANK_PRIMARY_UTIL=0.95' 'FRANK_ARCHON_UTIL=0.95'
) )
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the # D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
@@ -83,11 +83,11 @@ OVER_C=(
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result # D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the # says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
# batch shape. # batch shape.
KV401K='FRANK_PRIMARY_KV_BYTES=4050000000' KV401K='FRANK_ARCHON_KV_BYTES=4050000000'
OVER_D=( OVER_D=(
"FRANK_PRIMARY_SPEC='$SPEC3'" "FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'" "FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'" 'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
"$KV401K" "$KV401K"
) )
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT # E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
@@ -97,12 +97,12 @@ OVER_D=(
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC # only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed. # estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
OVER_E=( OVER_E=(
"FRANK_PRIMARY_SPEC='$SPEC3'" "FRANK_ARCHON_SPEC='$SPEC3'"
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'" "FRANK_ARCHON_COMPCONFIG='$LADDER3'"
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'" 'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
"$KV401K" "$KV401K"
'FRANK_PRIMARY_MNBT=4096' 'FRANK_ARCHON_MNBT=4096'
'FRANK_PRIMARY_UTIL=0.99' 'FRANK_ARCHON_UTIL=0.99'
) )
arm_overlay() { arm_overlay() {
@@ -142,7 +142,7 @@ for arm in "${ARMS[@]}"; do
# holds JSON must still be one valid JSON object after the file is SOURCED # holds JSON must still be one valid JSON object after the file is SOURCED
# (that is the step that silently eats unquoted braces). # (that is the step that silently eats unquoted braces).
if ! bash -c 'set -a; . "$1" >/dev/null; set +a if ! bash -c 'set -a; . "$1" >/dev/null; set +a
for k in FRANK_PRIMARY_SPEC FRANK_PRIMARY_COMPCONFIG FRANK_PRIMARY_GEN_CONFIG; do for k in FRANK_ARCHON_SPEC FRANK_ARCHON_COMPCONFIG FRANK_ARCHON_GEN_CONFIG; do
v="${!k:-}" v="${!k:-}"
[ -n "$v" ] || continue [ -n "$v" ] || continue
case "$v" in case "$v" in
@@ -158,7 +158,7 @@ for arm in "${ARMS[@]}"; do
fi fi
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2 echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
FRANK_PRIMARY_AR_ENV_FILE="$ARMS_DIR/$arm.env" \ FRANK_ARCHON_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1 "$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
rc=$? rc=$?
echo "$rc" >"$ARMS_DIR/$arm.rc" echo "$rc" >"$ARMS_DIR/$arm.rc"
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""frank-primary decode measurement — runs INSIDE the engine container. """frank-archon decode measurement — runs INSIDE the engine container.
The method is the fleet's established one (the 2026-09-29 / 2026-10-01 The method is the fleet's established one (the 2026-09-29 / 2026-10-01
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
@@ -28,7 +28,7 @@ Workload (fixed, deterministic, offline):
Env: Env:
AR_BASE endpoint (default http://127.0.0.1:7770) AR_BASE endpoint (default http://127.0.0.1:7770)
AR_MODEL served name (default frank-primary) AR_MODEL served name (default frank-archon)
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000) AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
AR_TEMPS comma-separated sampling temperatures (default "0,0.7") AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# frank-primary-window.sh — one full bench window for frank-primary, run ON # frank-archon-window.sh — one full bench window for frank-archon, run ON
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh). # DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
# #
# Discipline (AGENTS.md Benchmarking Policy): the production unit is # Discipline (AGENTS.md Benchmarking Policy): the production unit is
@@ -12,9 +12,9 @@
# #
# Staging layout (created by the orchestrator, ephemeral — never a config # Staging layout (created by the orchestrator, ephemeral — never a config
# source; the repo checkout is): # source; the repo checkout is):
# /home/dank/.autoresearch-frank-primary/ # /home/dank/.autoresearch-frank-archon/
# serve.sh staged candidate serve script # serve.sh staged candidate serve script
# frank-primary.env staged candidate env (the candidate knobs) # frank-archon.env staged candidate env (the candidate knobs)
# measure.py the in-container measure client # measure.py the in-container measure client
# hash candidate fingerprint # hash candidate fingerprint
# run.json result (metrics + boot + kv + fingerprint) # run.json result (metrics + boot + kv + fingerprint)
@@ -27,16 +27,16 @@
# outcome; a failed restore forces exit 1). # outcome; a failed restore forces exit 1).
set -uo pipefail set -uo pipefail
STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}" STAGE="${FRANK_ARCHON_AR_STAGE:-/home/dank/.autoresearch-frank-archon}"
UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}" UNIT="${FRANK_ARCHON_AR_UNIT:-frank-archon}"
CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}" CTR="${FRANK_ARCHON_AR_CTR:-frank-archon-qwen}"
HEALTH_URL="http://127.0.0.1:7770/health" HEALTH_URL="http://127.0.0.1:7770/health"
MODELS_URL="http://127.0.0.1:7770/v1/models" MODELS_URL="http://127.0.0.1:7770/v1/models"
BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}" BOOT_TIMEOUT="${FRANK_ARCHON_AR_BOOT_TIMEOUT:-1200}"
RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}" RESTORE_TIMEOUT="${FRANK_ARCHON_AR_RESTORE_TIMEOUT:-1200}"
MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}" MEASURE_TIMEOUT="${FRANK_ARCHON_AR_MEASURE_TIMEOUT:-1800}"
SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)" SERVED="$(sed -n 's/^FRANK_ARCHON_SERVED_NAME=//p' "$STAGE/frank-archon.env" 2>/dev/null | head -1)"
SERVED="${SERVED:-frank-primary}" SERVED="${SERVED:-frank-archon}"
RESTORED=0 RESTORED=0
@@ -72,7 +72,7 @@ capture_bench_failure() {
echo "=== staged serve.out (tail 60) ===" echo "=== staged serve.out (tail 60) ==="
tail -60 "$STAGE/serve.out" 2>/dev/null || true tail -60 "$STAGE/serve.out" 2>/dev/null || true
echo "=== engine log (tail 60) ===" echo "=== engine log (tail 60) ==="
tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true tail -60 /home/dank/logs/frank_archon.log 2>/dev/null || true
} >"$STAGE/fail.log" 2>&1 || true } >"$STAGE/fail.log" 2>&1 || true
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \ grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true "$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
@@ -114,7 +114,7 @@ trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
main() { main() {
local f local f
for f in serve.sh frank-primary.env measure.py hash; do for f in serve.sh frank-archon.env measure.py hash; do
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; } [ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
done done
mkdir -p "$STAGE/last" mkdir -p "$STAGE/last"
@@ -133,7 +133,7 @@ main() {
t0="$(date +%s)" t0="$(date +%s)"
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))" log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \ FRANK_ARCHON_ENV_FILE="$STAGE/frank-archon.env" \
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null & setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
echo "$!" >"$STAGE/serve.pid" echo "$!" >"$STAGE/serve.pid"
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Decode throughput benchmark for frank-large. # Decode throughput benchmark for frank-dragoon.
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens. # Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
set -euo pipefail set -euo pipefail
@@ -75,7 +75,7 @@ print(json.dumps(' '.join(words)))
t0=$(date +%s%N) t0=$(date +%s%N)
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \ resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d "{\"model\":\"frank-large\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1) -d "{\"model\":\"frank-dragoon\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
t1=$(date +%s%N) t1=$(date +%s%N)
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
HTTP-based prefill-throughput sweep for frank-large. HTTP-based prefill-throughput sweep for frank-dragoon.
Starts the server with a given --threads-batch value, sends a 64K-prompt via Starts the server with a given --threads-batch value, sends a 64K-prompt via
the chat API (max_tokens=0), measures prompt_tokens/wall_time. the chat API (max_tokens=0), measures prompt_tokens/wall_time.
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Threads-batch sweep for frank-large on dankclaw # Threads-batch sweep for frank-dragoon on dankclaw
# Benches prefill throughput at 65536 prompt tokens across threads-batch values. # Benches prefill throughput at 65536 prompt tokens across threads-batch values.
set -uo pipefail set -uo pipefail
@@ -10,7 +10,7 @@ TB_LIST=(16 32 48 64 80 96 128)
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin" ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
echo "========================================" echo "========================================"
echo "THREADS-BATCH SWEEP — frank-large" echo "THREADS-BATCH SWEEP — frank-dragoon"
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))" echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
echo "Threads-batch values: ${TB_LIST[*]}" echo "Threads-batch values: ${TB_LIST[*]}"
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)" echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
+4 -4
View File
@@ -1,14 +1,14 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Benchmark ubatch with llama-bench for frank-large. # Benchmark ubatch with llama-bench for frank-dragoon.
# #
# This is intentionally a ubatch-only benchmark. It uses one llama-bench # This is intentionally a ubatch-only benchmark. It uses one llama-bench
# process, an 8K prompt workload, and equal batch/ubatch values. No server # process, an 8K prompt workload, and equal batch/ubatch values. No server
# process, HTTP requests, or log scraping are used. # process, HTTP requests, or log scraping are used.
# #
# Run on Dankclaw after stopping frank-large: # Run on Dankclaw after stopping frank-dragoon:
# sudo systemctl stop frank-large # sudo systemctl stop frank-dragoon
# ./scripts/bench-ubatch.sh # ./scripts/bench-ubatch.sh
# sudo systemctl start frank-large # sudo systemctl start frank-dragoon
set -euo pipefail set -euo pipefail
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}" : "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
@@ -1,11 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# build-frank-primary-rad.sh — build frank-primary's .rad container from the # build-frank-archon-rad.sh — build frank-archon's .rad container from the
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision. # OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
# #
# bash scripts/build-frank-primary-rad.sh # convert # bash scripts/build-frank-archon-rad.sh # convert
# bash scripts/build-frank-primary-rad.sh --plan-only # declare, no I/O # bash scripts/build-frank-archon-rad.sh --plan-only # declare, no I/O
# bash scripts/build-frank-primary-rad.sh --restage # redo staging # bash scripts/build-frank-archon-rad.sh --restage # redo staging
# bash scripts/build-frank-primary-rad.sh --force # rebuild existing # bash scripts/build-frank-archon-rad.sh --force # rebuild existing
# #
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its # WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
# architecture plugin by the checkpoint's quantisation DESCRIPTOR — # architecture plugin by the checkpoint's quantisation DESCRIPTOR —
@@ -55,8 +55,8 @@ OUT="$OUT_DIR/qwen4exp-fp8-native.rad"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091 # shellcheck disable=SC1091
source "$SCRIPT_DIR/frank-primary.env" source "$SCRIPT_DIR/frank-archon.env"
IMAGE="$FRANK_PRIMARY_IMAGE" IMAGE="$FRANK_ARCHON_IMAGE"
plan_only=0 restage=0 force=0 plan_only=0 restage=0 force=0
for a in "$@"; do for a in "$@"; do
@@ -69,11 +69,11 @@ for a in "$@"; do
done done
if [ ! -f "$SRC/config.json" ]; then if [ ! -f "$SRC/config.json" ]; then
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-primary.sh --fetch)" >&2 echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-archon.sh --fetch)" >&2
exit 1 exit 1
fi fi
echo "verifying raw checkpoint first..." echo "verifying raw checkpoint first..."
bash "$SCRIPT_DIR/download-frank-primary.sh" bash "$SCRIPT_DIR/download-frank-archon.sh"
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
echo "container already exists: $OUT" echo "container already exists: $OUT"
@@ -83,7 +83,7 @@ fi
# ---- staging ---------------------------------------------------------------- # ---- staging ----------------------------------------------------------------
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-stage" TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-stage"
mkdir -p "$TRASH" mkdir -p "$TRASH"
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)" mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
echo "old staging dir moved to $TRASH" echo "old staging dir moved to $TRASH"
@@ -138,7 +138,7 @@ CONVERT_ARGS=(/models/stage-qwen4exp-fp8 -o /models/rad/qwen4exp-fp8-native.rad
# exactly and the plugin's one-element scale plane is produced. # exactly and the plugin's one-element scale plane is produced.
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE") CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
[ "$force" -eq 1 ] && [ -f "$OUT" ] && { [ "$force" -eq 1 ] && [ -f "$OUT" ] && {
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-rad" TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-rad"
mkdir -p "$TRASH" mkdir -p "$TRASH"
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)" mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
echo "previous container moved to $TRASH" echo "previous container moved to $TRASH"
@@ -158,8 +158,8 @@ echo "rad-info summary:"
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \ podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20 "$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
MIB=$(( $(stat -c %s "$OUT") / 1048576 )) MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
echo "container: $OUT (${MIB} MiB; env floor $FRANK_PRIMARY_RAD_MIN_MIB MiB)" echo "container: $OUT (${MIB} MiB; env floor $FRANK_ARCHON_RAD_MIN_MIB MiB)"
if [ "$MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then if [ "$MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
echo "ERROR: container below the env floor — the convert was truncated" >&2 echo "ERROR: container below the env floor — the convert was truncated" >&2
exit 1 exit 1
fi fi
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build the gufo runtime that serves frank-large on Dankclaw. # Build the gufo runtime that serves frank-dragoon on Dankclaw.
# #
# bash scripts/build-gufo.sh # build + install the pinned tag # bash scripts/build-gufo.sh # build + install the pinned tag
# bash scripts/build-gufo.sh --verify # only check the installed tree # bash scripts/build-gufo.sh --verify # only check the installed tree
@@ -93,4 +93,4 @@ cmake --install build/release
echo "== verify ==" echo "== verify =="
"$PREFIX/bin/gufo" diagnose "$PREFIX/bin/gufo" diagnose
echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)" echo "frank-dragoon runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
+4 -4
View File
@@ -1,11 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build the ROCmFPX runtime that serves frank-large on Dankclaw. # Build the ROCmFPX runtime that serves frank-dragoon on Dankclaw.
# #
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin # bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree # bash scripts/build-rocmfpx.sh --verify # only check the installed tree
# #
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card # Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's # patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-dragoon's
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of # payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so # GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
# stock llama.cpp, gufo and every other loader on the host reject the file. # stock llama.cpp, gufo and every other loader on the host reject the file.
@@ -30,7 +30,7 @@
set -euo pipefail set -euo pipefail
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}" ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
# The commit frank-large was qualified with (measured 2026-10-02). # The commit frank-dragoon was qualified with (measured 2026-10-02).
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}" ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}" SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}" BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
@@ -86,4 +86,4 @@ cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
echo "== verify ==" echo "== verify =="
"$BUILD_DIR/bin/llama-server" --version | head -1 "$BUILD_DIR/bin/llama-server" --version | head -1
echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)" echo "frank-dragoon runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
+3 -3
View File
@@ -57,9 +57,9 @@ else
echo " none" echo " none"
fi fi
section "frank-primary unit" section "frank-archon unit"
printf ' active: %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)" printf ' active: %s\n' "$(systemctl is-active frank-archon 2>/dev/null | head -1)"
printf ' enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)" printf ' enabled: %s\n' "$(systemctl is-enabled frank-archon 2>/dev/null | head -1)"
section "glm serve tail" section "glm serve tail"
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
+23 -23
View File
@@ -61,9 +61,9 @@ SERVICE_ORDER = (
"sonarr", "sonarr",
"lidarr", "lidarr",
"readarr", "readarr",
"frank-large", "frank-dragoon",
"frank-secondary", "frank-zealot",
"frank-primary", "frank-archon",
) )
DANKCLAW_SERVICES = { DANKCLAW_SERVICES = {
@@ -93,8 +93,8 @@ DANKCLAW_SERVICES = {
"authentik-outpost-ldap", "authentik-outpost-ldap",
"caddy", "caddy",
"homepage", "homepage",
"frank-large", "frank-dragoon",
"frank-secondary", "frank-zealot",
} }
DANKSERVER_SERVICES = { DANKSERVER_SERVICES = {
@@ -113,7 +113,7 @@ DANKSERVER_SERVICES = {
"sonarr", "sonarr",
"lidarr", "lidarr",
"readarr", "readarr",
"frank-primary", "frank-archon",
"node-exporter", "node-exporter",
} }
@@ -316,7 +316,7 @@ def services_for_path(path: str, target: str) -> set[str]:
if len(parts) >= 2 and parts[1] == "open-webui": if len(parts) >= 2 and parts[1] == "open-webui":
_add(services, "open-webui") _add(services, "open-webui")
return services return services
# frank-large left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host # frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
# build, root unit): its files live in scripts/ and map there. # build, root unit): its files live in scripts/ and map there.
if len(parts) >= 2 and parts[1] == "observability": if len(parts) >= 2 and parts[1] == "observability":
# The five observability containers became Quadlet units on # The five observability containers became Quadlet units on
@@ -371,10 +371,10 @@ def services_for_path(path: str, target: str) -> set[str]:
if name == "node-exporter": if name == "node-exporter":
_add(services, "node-exporter") _add(services, "node-exporter")
return services return services
# frank-primary's quadlet-era directory left the tree on 2026-09-26 # frank-archon's quadlet-era directory left the tree on 2026-09-26
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit — # (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
# the trio in scripts/ now maps it). Nothing lives under # the trio in scripts/ now maps it). Nothing lives under
# dankserver/frank-primary/ except dated trash, and trash plans # dankserver/frank-archon/ except dated trash, and trash plans
# nothing (the retire rule above the service blocks). # nothing (the retire rule above the service blocks).
return services return services
@@ -400,32 +400,32 @@ def services_for_path(path: str, target: str) -> set[str]:
# change to it redeploying alertmanager is the only path that # change to it redeploying alertmanager is the only path that
# reaches it on the host. # reaches it on the host.
_add(services, "alertmanager") _add(services, "alertmanager")
if name in {"frank-secondary.env", "frank-secondary.service", "serve-frank-secondary.sh"}: if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
# llama.cpp frank-secondary (2026-09-16) keeps its unit, env and serve # llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-secondary only. The ik_llama.cpp # files here; each one selects frank-zealot only. The ik_llama.cpp
# trio that lived here for two days is in scripts/.AGENTTRASH/, # trio that lived here for two days is in scripts/.AGENTTRASH/,
# which is archive material and plans nothing. # which is archive material and plans nothing.
_add(services, "frank-secondary") _add(services, "frank-zealot")
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}: if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
# CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and # CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
# serve files here the same way: a root unit on a host-built # serve files here the same way: a root unit on a host-built
# runtime. The download script fetches the payload, which is # runtime. The download script fetches the payload, which is
# host-side state, so it selects nothing. # host-side state, so it selects nothing.
_add(services, "frank-large") _add(services, "frank-dragoon")
if name == "qwen-sharp-v22.4.0.jinja": if name == "qwen-sharp-v22.4.0.jinja":
# The Qwen-Sharp chat template frank-large loads via # The Qwen-Sharp chat template frank-dragoon loads via
# --chat-template-file. It lived under dankserver/frank-medium/ until # --chat-template-file. It lived under dankserver/frank-medium/ until
# the 2026-09-26 cutover retired that directory; editing it changes # the 2026-09-26 cutover retired that directory; editing it changes
# what the live frank-large endpoint renders, so it plans # what the live frank-dragoon endpoint renders, so it plans
# frank-large. # frank-dragoon.
_add(services, "frank-large") _add(services, "frank-dragoon")
if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}: if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
# llama.cpp frank-primary (2026-09-26, replacing the Lucebox # llama.cpp frank-archon (2026-09-26, replacing the Lucebox
# DS4.1 quadlet) keeps its unit, env and serve files here the # DS4.1 quadlet) keeps its unit, env and serve files here the
# same way: a root unit on the unsloth b11030-mix prebuilt # same way: a root unit on the unsloth b11030-mix prebuilt
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches # serving Qwen3.8-Flash-Next Q8_0. The download script fetches
# the payload, which is host-side state, so it selects nothing. # the payload, which is host-side state, so it selects nothing.
_add(services, "frank-primary") _add(services, "frank-archon")
# Deployment/test/benchmark scripts and other repo tooling do not # Deployment/test/benchmark scripts and other repo tooling do not
# restart a runtime service. # restart a runtime service.
return services return services
+113 -113
View File
@@ -90,12 +90,12 @@ service_is_allowed() {
dankclaw:authentik-postgres|dankclaw:authentik-server|\ dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\ dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\ dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
dankclaw:frank-large|dankclaw:frank-secondary|\ dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\ dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\ dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\ dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\ dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
dankserver:frank-primary|dankserver:node-exporter|\ dankserver:frank-archon|dankserver:node-exporter|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\ frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\ frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr) frankbook-pro:readarr)
@@ -192,7 +192,7 @@ add_pending_services() {
continue continue
fi fi
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue [[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
if [[ "$TARGET" == dankserver && "$service" == frank-large ]]; then continue; fi if [[ "$TARGET" == dankserver && "$service" == frank-dragoon ]]; then continue; fi
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \ if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then --repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
echo "WARN: unable to compare state for $service; retaining it as pending" >&2 echo "WARN: unable to compare state for $service; retaining it as pending" >&2
@@ -1092,21 +1092,21 @@ deploy_anubis() {
wait_container_healthy anubis 60 || return 1 wait_container_healthy anubis 60 || return 1
} }
# frank-large is the halogen-flash-server CONTAINER (0.16.2, digest-pinned # frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
# through scripts/frank-large.env) serving the Quat3rnion ABLITERATED # through scripts/frank-dragoon.env) serving the Quat3rnion ABLITERATED
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint — # Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB # qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
# resident) + the REQUIRED n-gram lookup table + the vision tower — # resident) + the REQUIRED n-gram lookup table + the vision tower —
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input — # MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script # on the Strix Halo iGPU. It stays a ROOT unit because the serve script
# drives `podman run` directly (the frank-primary pattern): relink_unit # drives `podman run` directly (the frank-archon pattern): relink_unit
# points /etc/systemd/system/frank-large.service at the repo file, boot # points /etc/systemd/system/frank-dragoon.service at the repo file, boot
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is # persistence is that unit's [Install], and LimitMEMLOCK=infinity is
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape — # fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
# rollback = revert the swap commit's trio; the 27B payload stays on # rollback = revert the swap commit's trio; the 27B payload stays on
# disk (~/models/halogen-qwen3.8-27b). # disk (~/models/halogen-qwen3.8-27b).
# #
# The gates mirror serve-frank-large.sh's predicates EXACTLY (the # The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage, # 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact # each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
# byte count, the tokenizer directory holding every named file as a # byte count, the tokenizer directory holding every named file as a
@@ -1114,23 +1114,23 @@ deploy_anubis() {
# artifact would leave a restart-looping unit and a dark endpoint. The # artifact would leave a restart-looping unit and a dark endpoint. The
# port gate is the same as every cutover since 2026-09-16: nothing # port gate is the same as every cutover since 2026-09-16: nothing
# starts until 7770 is released. # starts until 7770 is released.
LLM_LARGE_IMAGE="${LLM_LARGE_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}" LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
LLM_LARGE_CKPT="${LLM_LARGE_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}" LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
LLM_LARGE_CKPT_BYTES="${LLM_LARGE_CKPT_BYTES:-66687678464}" LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678464}"
LLM_LARGE_NGRAM="${LLM_LARGE_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}" LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
LLM_LARGE_NGRAM_BYTES="${LLM_LARGE_NGRAM_BYTES:-51200246144}" LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}"
LLM_LARGE_VISION="${LLM_LARGE_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}" LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
LLM_LARGE_VISION_BYTES="${LLM_LARGE_VISION_BYTES:-897916416}" LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
LLM_LARGE_TOK_DIR="${LLM_LARGE_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}" LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
LLM_LARGE_TOK_FILES="${LLM_LARGE_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}" LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
retire_halogen_large_quadlet() { retire_halogen_dragoon_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet" local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
systemctl --user stop frank-large.service 2>/dev/null || true systemctl --user stop frank-dragoon.service 2>/dev/null || true
if [[ -e "$quadlet_unit_dir/frank-large.container" ]]; then if [[ -e "$quadlet_unit_dir/frank-dragoon.container" ]]; then
mkdir -p "$trash" mkdir -p "$trash"
mv -- "$quadlet_unit_dir/frank-large.container" "$trash/frank-large.container" mv -- "$quadlet_unit_dir/frank-dragoon.container" "$trash/frank-dragoon.container"
echo "retired quadlet frank-large.container -> $trash" echo "retired quadlet frank-dragoon.container -> $trash"
fi fi
systemctl --user daemon-reload systemctl --user daemon-reload
} }
@@ -1154,165 +1154,165 @@ wait_port_free() {
return 1 return 1
} }
deploy_llm_large() { deploy_llm_dragoon() {
# Refuse rather than restart-loop; every gate below has a twin in # Refuse rather than restart-loop; every gate below has a twin in
# serve-frank-large.sh and they must agree (2026-10-01 lesson). # serve-frank-dragoon.sh and they must agree (2026-10-01 lesson).
# Capture podman output before grepping: `podman images | grep -q` # Capture podman output before grepping: `podman images | grep -q`
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30). # races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
if ! grep -qF "${LLM_LARGE_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then if ! grep -qF "${LLM_DRAGOON_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
echo "ERROR: halogen image absent: $LLM_LARGE_IMAGE" >&2 echo "ERROR: halogen image absent: $LLM_DRAGOON_IMAGE" >&2
echo "frank-large: pull it: podman pull ${LLM_LARGE_IMAGE%@*}" >&2 echo "frank-dragoon: pull it: podman pull ${LLM_DRAGOON_IMAGE%@*}" >&2
return 1 return 1
fi fi
if [[ ! -f "$LLM_LARGE_CKPT" ]]; then if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
echo "ERROR: halogen checkpoint absent at $LLM_LARGE_CKPT" >&2 echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1 return 1
fi fi
if [[ "$(stat -c %s "$LLM_LARGE_CKPT")" != "$LLM_LARGE_CKPT_BYTES" ]]; then if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
echo "ERROR: halogen checkpoint is not $LLM_LARGE_CKPT_BYTES bytes (partial download?)" >&2 echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2 echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1 return 1
fi fi
for gate in "$LLM_LARGE_NGRAM:$LLM_LARGE_NGRAM_BYTES:n-gram table" \ for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
"$LLM_LARGE_VISION:$LLM_LARGE_VISION_BYTES:vision tower"; do "$LLM_DRAGOON_VISION:$LLM_DRAGOON_VISION_BYTES:vision tower"; do
IFS=: read -r gpath gbytes gname <<<"$gate" IFS=: read -r gpath gbytes gname <<<"$gate"
if [[ ! -f "$gpath" ]]; then if [[ ! -f "$gpath" ]]; then
echo "ERROR: halogen $gname absent at $gpath" >&2 echo "ERROR: halogen $gname absent at $gpath" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1 return 1
fi fi
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2 echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2 echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
return 1 return 1
fi fi
done done
if [[ -L "$LLM_LARGE_TOK_DIR" || ! -d "$LLM_LARGE_TOK_DIR" ]]; then if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
echo "ERROR: tokenizer dir missing or a symlink at $LLM_LARGE_TOK_DIR" >&2 echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1 return 1
fi fi
for tf in $LLM_LARGE_TOK_FILES; do for tf in $LLM_DRAGOON_TOK_FILES; do
if [[ ! -f "$LLM_LARGE_TOK_DIR/$tf" || -L "$LLM_LARGE_TOK_DIR/$tf" ]]; then if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
echo "ERROR: tokenizer file missing or a symlink: $LLM_LARGE_TOK_DIR/$tf" >&2 echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
return 1 return 1
fi fi
done done
retire_halogen_large_quadlet retire_halogen_dragoon_quadlet
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops # The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
# the container) before waiting on the port. The stray sweep is keyed # the container) before waiting on the port. The stray sweep is keyed
# on the FULL container name, which appears in the argv of a manual # on the FULL container name, which appears in the argv of a manual
# `podman run --name frank-large-halogen` / `podman wait` but NOT in # `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
# this deployer's own command line (`bash deploy-remote.sh # this deployer's own command line (`bash deploy-remote.sh
# frank-large`) — so it cannot shoot itself in the foot (the fleet # frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
# pkill law), and it can never match frank-secondary's llama-server. # pkill law), and it can never match frank-zealot's llama-server.
run_privileged systemctl stop frank-large.service 2>/dev/null || true run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
pkill -f 'frank-large-halogen' 2>/dev/null || true pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
podman rm -f frank-large frank-large-halogen >/dev/null 2>&1 || true podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
# Nothing may start until the predecessor has released the port # Nothing may start until the predecessor has released the port
# (2026-09-16: "Deactivated successfully" was not "port free"). # (2026-09-16: "Deactivated successfully" was not "port free").
wait_port_free 7770 || return 1 wait_port_free 7770 || return 1
relink_unit frank-large relink_unit frank-dragoon
run_privileged systemctl daemon-reload run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-large.service run_privileged systemctl enable frank-dragoon.service
run_privileged systemctl restart frank-large.service run_privileged systemctl restart frank-dragoon.service
wait_for_health frank-large http://127.0.0.1:7770/health 1200 wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
} }
# deploy_llm_secondary — frank-secondary is llama.cpp (master build, CUDA sm_86) on # deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under # the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
# ~/GitPackages/llama-master/build-cuda, so there is no container image and # ~/GitPackages/llama-master/build-cuda, so there is no container image and
# no generated unit to avoid. relink_unit points # no generated unit to avoid. relink_unit points
# /etc/systemd/system/frank-secondary.service at the repo file and boot # /etc/systemd/system/frank-zealot.service at the repo file and boot
# persistence is that unit's [Install] section. # persistence is that unit's [Install] section.
# #
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is # It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
# not shared: frank-large's engine runs on the gfx1151 iGPU. # not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}" LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
LLM_SECONDARY_RUNTIME="${LLM_SECONDARY_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}" LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
# The projector IS the vision capability here: the model is natively # The projector IS the vision capability here: the model is natively
# multimodal and the Q6_K file carries no vision tensors — the publisher ships # multimodal and the Q6_K file carries no vision tensors — the publisher ships
# its own BF16 projector in the same repo, so a deploy that starts without # its own BF16 projector in the same repo, so a deploy that starts without
# this file silently ships a text-only endpoint. Checked unconditionally, like # this file silently ships a text-only endpoint. Checked unconditionally, like
# frank-large's. # frank-dragoon's.
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}" LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
deploy_llm_secondary() { deploy_llm_zealot() {
# Refuse rather than restart-loop: a unit whose runtime or payload is # Refuse rather than restart-loop: a unit whose runtime or payload is
# missing never becomes healthy, and a green deploy would leave the # missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why. # endpoint dark with nothing in the log to say why.
if [[ ! -x "$LLM_SECONDARY_RUNTIME" ]]; then if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
echo "ERROR: llama.cpp runtime absent at $LLM_SECONDARY_RUNTIME" >&2 echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
echo "frank-secondary: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2 echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
echo "frank-secondary: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2 echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
echo "frank-secondary: && cmake --build build-cuda --target llama-server) first." >&2 echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
return 1 return 1
fi fi
for f in "$LLM_SECONDARY_MODEL_FILE" "$LLM_SECONDARY_MMPROJ"; do for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
if [[ ! -f "$f" ]]; then if [[ ! -f "$f" ]]; then
echo "ERROR: frank-secondary payload file absent at $f" >&2 echo "ERROR: frank-zealot payload file absent at $f" >&2
echo "frank-secondary: run scripts/download-frank-secondary.sh first;" >&2 echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
echo "frank-secondary: refusing to start a unit that cannot load." >&2 echo "frank-zealot: refusing to start a unit that cannot load." >&2
return 1 return 1
fi fi
done done
relink_unit frank-secondary relink_unit frank-zealot
run_privileged systemctl daemon-reload run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-secondary.service run_privileged systemctl enable frank-zealot.service
run_privileged systemctl restart frank-secondary.service run_privileged systemctl restart frank-zealot.service
wait_for_health frank-secondary http://127.0.0.1:7771/health 900 || return 1 wait_for_health frank-zealot http://127.0.0.1:7771/health 900 || return 1
# False-green hardening (frank-large's lesson): the port answering # False-green hardening (frank-dragoon's lesson): the port answering
# proves nothing about WHO answers — the unit must be active with a # proves nothing about WHO answers — the unit must be active with a
# live MainPID, or a stray SGLang holdover is masquerading as deployed. # live MainPID, or a stray SGLang holdover is masquerading as deployed.
local main_pid local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-secondary.service)" main_pid="$(run_privileged systemctl show -p MainPID --value frank-zealot.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7771 answered but frank-secondary.service has no MainPID — a stray process holds the port" >&2 echo "FAIL: :7771 answered but frank-zealot.service has no MainPID — a stray process holds the port" >&2
return 1 return 1
fi fi
echo "frank-secondary: health answered by unit MainPID $main_pid" echo "frank-zealot: health answered by unit MainPID $main_pid"
} }
# deploy_llm_primary — frank-primary is the radiance engine (StillDeadcode, # deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL # digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the # Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a # rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
# ROOT unit whose serve script drives `podman run` directly (the engine is # ROOT unit whose serve script drives `podman run` directly (the engine is
# the image entrypoint; the image lives in dank's rootless podman storage). # the image entrypoint; the image lives in dank's rootless podman storage).
# 2026-10-05 cutover from FreeToken (pre-cutover tag # 2026-10-05 cutover from FreeToken (pre-cutover tag
# frank-primary-freetoken-pr587-20261005); before that, the tcclaviger vLLM # frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
# fork (tag frank-primary-vllm-shapeG-20261003). Same port and alias; the # fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
# harness models.yml id is unchanged. # harness models.yml id is unchanged.
# #
# Gates: the image must exist under its digest ref in dank's rootless # Gates: the image must exist under its digest ref in dank's rootless
# storage (FRANK_PRIMARY_IMAGE), the .rad container file must exist at the # storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
# env's size floor (FRANK_PRIMARY_MODEL_FILE >= FRANK_PRIMARY_RAD_MIN_MIB — # env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
# a truncated convert is a guaranteed long-timeout boot), and the unit env # a truncated convert is a guaranteed long-timeout boot), and the unit env
# must be complete. Per-file sha256 verification is the fetch script's job # must be complete. Per-file sha256 verification is the fetch script's job
# at download time and rad-convert/rad-info's job at build time, never # at download time and rad-convert/rad-info's job at build time, never
# re-hashed here. # re-hashed here.
LLM_PRIMARY_CONTAINER=frank-primary LLM_ARCHON_CONTAINER=frank-archon
LLM_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe" LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
deploy_llm_primary() { deploy_llm_archon() {
# Refuse rather than restart-loop: a unit whose image or payload is # Refuse rather than restart-loop: a unit whose image or payload is
# missing never becomes healthy, and a green deploy would leave the # missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the log to say why. # endpoint dark with nothing in the log to say why.
local env_file="/home/dank/serverAppConfigs/scripts/frank-primary.env" local env_file="/home/dank/serverAppConfigs/scripts/frank-archon.env"
local image model_dir model_file rad_min mib local image model_dir model_file rad_min mib
if [[ ! -f "$env_file" ]]; then if [[ ! -f "$env_file" ]]; then
echo "ERROR: frank-primary env file absent: $env_file" >&2 echo "ERROR: frank-archon env file absent: $env_file" >&2
return 1 return 1
fi fi
image="$(sed -n 's/^FRANK_PRIMARY_IMAGE=//p' "$env_file" | tail -1)" image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
model_dir="$(sed -n 's/^FRANK_PRIMARY_MODEL_DIR=//p' "$env_file" | tail -1)" model_dir="$(sed -n 's/^FRANK_ARCHON_MODEL_DIR=//p' "$env_file" | tail -1)"
model_file="$(sed -n 's/^FRANK_PRIMARY_MODEL_FILE=//p' "$env_file" | tail -1)" model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
rad_min="$(sed -n 's/^FRANK_PRIMARY_RAD_MIN_MIB=//p' "$env_file" | tail -1)" rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2 echo "ERROR: frank-archon.env missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
return 1 return 1
fi fi
# The image gate is the digest ref itself: a name@sha256 ref exists in # The image gate is the digest ref itself: a name@sha256 ref exists in
@@ -1320,18 +1320,18 @@ deploy_llm_primary() {
# pipefail/grep race (the 2026-09-30 lesson). # pipefail/grep race (the 2026-09-30 lesson).
if ! podman image exists "$image"; then if ! podman image exists "$image"; then
echo "ERROR: runtime image absent from storage: $image" >&2 echo "ERROR: runtime image absent from storage: $image" >&2
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest" >&2 echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest" >&2
return 1 return 1
fi fi
if [[ ! -f "$model_dir/$model_file" ]]; then if [[ ! -f "$model_dir/$model_file" ]]; then
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2 echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2 echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
return 1 return 1
fi fi
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 )) mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
if [[ "$mib" -lt "$rad_min" ]]; then if [[ "$mib" -lt "$rad_min" ]]; then
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2 echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
echo "frank-primary: rebuild it (scripts/build-frank-primary-rad.sh --force)" >&2 echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
return 1 return 1
fi fi
@@ -1343,19 +1343,19 @@ deploy_llm_primary() {
# The legacy identities (the retired R9V container, the ad hoc # The legacy identities (the retired R9V container, the ad hoc
# experiment container qwen-fp8, the retired glm-probe) hold the same # experiment container qwen-fp8, the retired glm-probe) hold the same
# GPUs and die here too. # GPUs and die here too.
run_privileged systemctl stop frank-primary.service 2>/dev/null || true run_privileged systemctl stop frank-archon.service 2>/dev/null || true
local ctr local ctr
for ctr in $LLM_PRIMARY_LEGACY_CONTAINERS "$LLM_PRIMARY_CONTAINER"; do for ctr in $LLM_ARCHON_LEGACY_CONTAINERS "$LLM_ARCHON_CONTAINER"; do
podman rm -f "$ctr" >/dev/null 2>&1 || true podman rm -f "$ctr" >/dev/null 2>&1 || true
done done
# Nothing starts until the port is released: the lucebox cutover # Nothing starts until the port is released: the lucebox cutover
# (2026-09-26) and the frank-large cutover (2026-09-16) both failed # (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
# exactly here when the next engine bound 7770 a second too early. # exactly here when the next engine bound 7770 a second too early.
wait_port_free 7770 || return 1 wait_port_free 7770 || return 1
relink_unit frank-primary relink_unit frank-archon
run_privileged systemctl daemon-reload run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-primary.service run_privileged systemctl enable frank-archon.service
run_privileged systemctl restart frank-primary.service run_privileged systemctl restart frank-archon.service
# Cold start: .rad load + PLE pin + expert host-pool upload (the # Cold start: .rad load + PLE pin + expert host-pool upload (the
# publisher's compose healthcheck allows a 15-minute start_period). # publisher's compose healthcheck allows a 15-minute start_period).
# Parity with every predecessor on this port. The HTTP listener binds # Parity with every predecessor on this port. The HTTP listener binds
@@ -1366,14 +1366,14 @@ deploy_llm_primary() {
local i ok=0 local i ok=0
for ((i=1; i<=900; i++)); do for ((i=1; i<=900; i++)); do
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
echo "frank-primary engine healthy after approximately $((i * 2))s" echo "frank-archon engine healthy after approximately $((i * 2))s"
ok=1 ok=1
break break
fi fi
sleep 2 sleep 2
done done
if [[ "$ok" != 1 ]]; then if [[ "$ok" != 1 ]]; then
echo "FAIL: frank-primary /health never answered 200 in 1800s" >&2 echo "FAIL: frank-archon /health never answered 200 in 1800s" >&2
return 1 return 1
fi fi
# False-green hardening (the 2026-09-26 lesson): a bare curl against the # False-green hardening (the 2026-09-26 lesson): a bare curl against the
@@ -1381,12 +1381,12 @@ deploy_llm_primary() {
# it while the unit is stopped. The deploy is only green if the unit # it while the unit is stopped. The deploy is only green if the unit
# itself is active with a live MainPID behind the health answer. # itself is active with a live MainPID behind the health answer.
local main_pid local main_pid
main_pid="$(run_privileged systemctl show -p MainPID --value frank-primary.service)" main_pid="$(run_privileged systemctl show -p MainPID --value frank-archon.service)"
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
echo "FAIL: :7770 answered but frank-primary.service has no MainPID — a stray process holds the port" >&2 echo "FAIL: :7770 answered but frank-archon.service has no MainPID — a stray process holds the port" >&2
return 1 return 1
fi fi
echo "frank-primary: health answered by unit MainPID $main_pid" echo "frank-archon: health answered by unit MainPID $main_pid"
} }
wait_for_health() { wait_for_health() {
@@ -1573,9 +1573,9 @@ deploy_service() {
anubis) deploy_anubis ;; anubis) deploy_anubis ;;
seerr) deploy_seerr ;; seerr) deploy_seerr ;;
marreta) deploy_marreta ;; marreta) deploy_marreta ;;
frank-large) deploy_llm_large ;; frank-dragoon) deploy_llm_dragoon ;;
frank-secondary) deploy_llm_secondary ;; frank-zealot) deploy_llm_zealot ;;
frank-primary) deploy_llm_primary ;; frank-archon) deploy_llm_archon ;;
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;; hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;; romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
jellyfin) deploy_jellyfin ;; jellyfin) deploy_jellyfin ;;
+3 -3
View File
@@ -35,7 +35,7 @@ Options:
Examples: Examples:
./scripts/deploy.sh dankclaw ./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankclaw --service searxng ./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-primary ./scripts/deploy.sh dankserver --service frank-archon
./scripts/deploy.sh dankclaw --dry-run ./scripts/deploy.sh dankclaw --dry-run
EOF EOF
} }
@@ -141,12 +141,12 @@ valid_service() {
dankclaw:alertmanager|\ dankclaw:alertmanager|\
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\ dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\ dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
dankclaw:frank-large|dankclaw:frank-secondary|\ dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\ dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\ dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\ dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\ dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
dankserver:frank-primary|\ dankserver:frank-archon|\
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\ frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\ frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
frankbook-pro:readarr) frankbook-pro:readarr)
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-primary-swift15.sh — payload fetch/verify for the planned # download-frank-archon-swift15.sh — payload fetch/verify for the planned
# frank-primary swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next # frank-archon swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors # (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
# shards + weight-map index + tokenizer + vision preprocessing configs + chat # shards + weight-map index + tokenizer + vision preprocessing configs + chat
# template + licenses). Key set verified identical to the official # template + licenses). Key set verified identical to the official
@@ -8,8 +8,8 @@
# apply verbatim; the checkpoint ships BF16 with NO quantization_config. # apply verbatim; the checkpoint ships BF16 with NO quantization_config.
# Repo is UNGATED — no token needed (never put one on the command line). # Repo is UNGATED — no token needed (never put one on the command line).
# #
# bash scripts/download-frank-primary-swift15.sh --fetch resume/complete # bash scripts/download-frank-archon-swift15.sh --fetch resume/complete
# bash scripts/download-frank-primary-swift15.sh verify only # bash scripts/download-frank-archon-swift15.sh verify only
# #
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0 # Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against # listing/resume downloader, curl+jq only): per-file size checks against
@@ -1,13 +1,13 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-primary.sh — payload fetch/verify for frank-primary: # download-frank-archon.sh — payload fetch/verify for frank-archon:
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors # the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
# shards + weight-map index + tokenizer + vision preprocessing configs, # shards + weight-map index + tokenizer + vision preprocessing configs,
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the # ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
# FreeToken era (that script lived here; its repo was gated and needed # FreeToken era (that script lived here; its repo was gated and needed
# HF_TOKEN — this one is UNGATED and needs no token at all). # HF_TOKEN — this one is UNGATED and needs no token at all).
# #
# bash scripts/download-frank-primary.sh --fetch resume/complete # bash scripts/download-frank-archon.sh --fetch resume/complete
# bash scripts/download-frank-primary.sh verify only # bash scripts/download-frank-archon.sh verify only
# #
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0 # Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against # listing/resume downloader, curl+jq only): per-file size checks against
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-large-flashnext.sh — fetch frank-large's flash-server v2 # download-frank-dragoon-flashnext.sh — fetch frank-dragoon's flash-server v2
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2 # payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
# .hgn, the REQUIRED lookup table and the optional vision tower from # .hgn, the REQUIRED lookup table and the optional vision tower from
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model # peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
@@ -29,7 +29,7 @@
# fetched fresh rather than symlinked, per the no-symlink law. # fetched fresh rather than symlinked, per the no-symlink law.
set -euo pipefail set -euo pipefail
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}" DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main" ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve" PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26" NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-large-halogen.sh — fetch frank-large's halogen payload: # download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the # the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet # PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
# secret law, none would ever touch argv). # secret law, none would ever touch argv).
@@ -10,7 +10,7 @@
# existing payload without downloading. # existing payload without downloading.
set -euo pipefail set -euo pipefail
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}" DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main" BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
FILES=(qwen3.8-27b-p1w4d-d2.hgn FILES=(qwen3.8-27b-p1w4d-d2.hgn
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for # Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token # frank-dragoon (Dankclaw iGPU). Public repo — no auth header, and never a token
# on the command line (it is world-readable in /proc/<pid>/cmdline). # on the command line (it is world-readable in /proc/<pid>/cmdline).
# #
# The payload is the repo's Q6 AGENT rung: # The payload is the repo's Q6 AGENT rung:
@@ -10,7 +10,7 @@
# names for MTP serving ("Take Q6 AGENT (114) if you want protected # names for MTP serving ("Take Q6 AGENT (114) if you want protected
# heads"). Like every ROCmFPX file it loads on nothing but the fork. # heads"). Like every ROCmFPX file it loads on nothing but the fork.
# #
# The other two files frank-large needs are NOT re-downloaded: # The other two files frank-dragoon needs are NOT re-downloaded:
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the # * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid # ALIGNED repo's draft head byte for byte (identical Hub LFS oid
# 051a1764…), so it is hardlinked out of the aligned payload dir when that # 051a1764…), so it is hardlinked out of the aligned payload dir when that
@@ -25,7 +25,7 @@
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped. # byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
# #
# Usage: # Usage:
# scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only] # scripts/download-frank-dragoon-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF" REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large # Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-dragoon
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the # (Dankclaw iGPU). Public repo — no auth header, and never a token on the
# command line (it is world-readable in /proc/<pid>/cmdline). # command line (it is world-readable in /proc/<pid>/cmdline).
# #
@@ -24,7 +24,7 @@
# come from the Hub LFS oids (api/models/<repo>/tree/main). # come from the Hub LFS oids (api/models/<repo>/tree/main).
# #
# Usage: # Usage:
# scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only] # scripts/download-frank-dragoon-kingjones.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF" REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large # Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-dragoon
# model) on Dankclaw. # model) on Dankclaw.
# #
# bash scripts/download-frank-large-orca.sh # bash scripts/download-frank-dragoon-orca.sh
# bash scripts/download-frank-large-orca.sh --verify-only # bash scripts/download-frank-dragoon-orca.sh --verify-only
# #
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at # The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head # 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
@@ -105,4 +105,4 @@ fi
echo "verifying assembled files against SHA256SUMS" echo "verifying assembled files against SHA256SUMS"
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS) (cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS) (cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
echo "frank-large payload complete: $MODEL_DIR" echo "frank-dragoon payload complete: $MODEL_DIR"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the Q4_K_XL frank-large payload on Dankclaw: # Download the Q4_K_XL frank-dragoon payload on Dankclaw:
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf # unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
# #
# bash scripts/download-frank-large-q4.sh # bash scripts/download-frank-dragoon-q4.sh
# bash scripts/download-frank-large-q4.sh --verify-only # bash scripts/download-frank-dragoon-q4.sh --verify-only
# #
# Integrity: the file is checked against its declared byte count and sha256 # Integrity: the file is checked against its declared byte count and sha256
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED: # (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
@@ -21,12 +21,12 @@
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB. # x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
# #
# The speculative drafter is UNCHANGED and still fetched by its own script # The speculative drafter is UNCHANGED and still fetched by its own script
# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not # (scripts/download-frank-dragoon.sh, z-lab's Hub repo). That name must not
# appear here at all — the frank-large seam greps THIS file for it to prove # appear here at all — the frank-dragoon seam greps THIS file for it to prove
# the drafter was not folded into the payload fetch. # the drafter was not folded into the payload fetch.
# #
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one # NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it — # separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-dragoon does not need it —
# spec decode here is the block-diffusion drafter, not MTP. # spec decode here is the block-diffusion drafter, not MTP.
set -euo pipefail set -euo pipefail
@@ -1,10 +1,10 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the Swift-1.5 frank-large candidate on Dankclaw: # Download the Swift-1.5 frank-dragoon candidate on Dankclaw:
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF # ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf # Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
# #
# bash scripts/download-frank-large-swift.sh # bash scripts/download-frank-dragoon-swift.sh
# bash scripts/download-frank-large-swift.sh --verify-only # bash scripts/download-frank-dragoon-swift.sh --verify-only
# #
# Integrity: the file is checked against its declared byte count and sha256 # Integrity: the file is checked against its declared byte count and sha256
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is # (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
@@ -15,9 +15,9 @@
# Layout after a successful run (MODEL_DIR): # Layout after a successful run (MODEL_DIR):
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272 # Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
# #
# This is the CANDIDATE payload, not the deployed one. frank-large still # This is the CANDIDATE payload, not the deployed one. frank-dragoon still
# serves scripts/download-frank-large.sh's output; do not repoint # serves scripts/download-frank-dragoon.sh's output; do not repoint
# frank-large.env at MODEL_DIR until the gufo load test passes. # frank-dragoon.env at MODEL_DIR until the gufo load test passes.
set -euo pipefail set -euo pipefail
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL" MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
@@ -1,9 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the # Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
# frank-large model) on Dankclaw. # frank-dragoon model) on Dankclaw.
# #
# bash scripts/download-frank-large-unsloth.sh # bash scripts/download-frank-dragoon-unsloth.sh
# bash scripts/download-frank-large-unsloth.sh --verify-only # bash scripts/download-frank-dragoon-unsloth.sh --verify-only
# #
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at # The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
# 50 GB). The parts ARE the canonical payload: the serve script points -m at # 50 GB). The parts ARE the canonical payload: the serve script points -m at
@@ -112,4 +112,4 @@ if [[ "$mode" == "download" ]]; then
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/" [[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
fi fi
echo "frank-large payload complete: $MODEL_DIR" echo "frank-dragoon payload complete: $MODEL_DIR"
@@ -1,10 +1,10 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the frank-large payload on Dankclaw: # Download the frank-dragoon payload on Dankclaw:
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF) # - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter) # - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
# #
# bash scripts/download-frank-large.sh # bash scripts/download-frank-dragoon.sh
# bash scripts/download-frank-large.sh --verify-only # bash scripts/download-frank-dragoon.sh --verify-only
# #
# Integrity: every file is checked against its declared byte count and sha256 # Integrity: every file is checked against its declared byte count and sha256
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token # (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
@@ -86,4 +86,4 @@ if [[ "$mode" == "download" ]]; then
done done
fi fi
echo "frank-large payload complete: $MODEL_DIR" echo "frank-dragoon payload complete: $MODEL_DIR"
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Download the frank-secondary payload for llama.cpp on the RTX 3090: # Download the frank-zealot payload for llama.cpp on the RTX 3090:
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the # HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively # PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
# multimodal, so this file is the endpoint's eyes). Public repo — no auth # multimodal, so this file is the endpoint's eyes). Public repo — no auth
@@ -21,7 +21,7 @@
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/. # replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
# #
# Usage: # Usage:
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only] # scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
set -euo pipefail set -euo pipefail
DEST="" DEST=""
@@ -1,5 +1,5 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Measure per-block / per-expert expert-routing activity on live frank-primary. """Measure per-block / per-expert expert-routing activity on live frank-archon.
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`) Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
on the llama-server process while concurrent generations of mixed content run. on the llama-server process while concurrent generations of mixed content run.
@@ -1,13 +1,13 @@
# frank-primary.env — radiance (StillDeadcode, Apache-2.0) serving the # frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision # OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2), # (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
# dankserver :7770. # dankserver :7770.
# #
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in # 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
# pinned host RAM; git tag frank-primary-freetoken-pr587-20261005 is the # pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the # pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag # tag and rebuildable). Before that: tcclaviger vLLM fork (tag
# frank-primary-vllm-shapeG-20261003). # frank-archon-vllm-shapeG-20261003).
# #
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for # WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
# gfx1201, drives `podman run` directly from the digest-pinned # gfx1201, drives `podman run` directly from the digest-pinned
@@ -29,7 +29,7 @@
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built # qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
# containers carry. Per-weight encodings are data, not the descriptor — # containers carry. Per-weight encodings are data, not the descriptor —
# removing the key changes nothing about what the weights hold. Built by # removing the key changes nothing about what the weights hold. Built by
# scripts/build-frank-primary-rad.sh (staging + convert + rad-info). # scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
# #
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml): # SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start() # - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
@@ -67,14 +67,14 @@
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of # is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this # the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
# file already uses the pair, which BOTH pins accept (rollback symmetry). # file already uses the pair, which BOTH pins accept (rollback symmetry).
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
# The .rad container FILE (mounted under /models) and the state dir for the # The .rad container FILE (mounted under /models) and the state dir for the
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY # prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
# .rad files; the raw HF download and its staging dir live beside it and # .rad files; the raw HF download and its staging dir live beside it and
# are never mounted. # are never mounted.
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection # The state dir's tune/ subdirectory holds rad-tune's kernel-selection
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune # cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the # at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
@@ -84,48 +84,48 @@ FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
# name across systemctl and podman ps. The runtime changes; the name does # name across systemctl and podman ps. The runtime changes; the name does
# not. The legacy list retires every predecessor identity that could hold # not. The legacy list retires every predecessor identity that could hold
# the GPUs or the port. # the GPUs or the port.
FRANK_PRIMARY_CONTAINER=frank-primary FRANK_ARCHON_CONTAINER=frank-archon
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe" FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
# Served model id — Protoss name (wire rename 2026-10-09). The container, # Served model id — Protoss name (wire rename 2026-10-09). The container,
# unit, scripts and this file keep the frank-primary name above; only the # unit, scripts and this file keep the frank-archon name above; only the
# alias on the wire moved. Radiance validates the requested model name, so # alias on the wire moved. Radiance validates the requested model name, so
# every client (harness, bench scripts, probes) MUST send this value. # every client (harness, bench scripts, probes) MUST send this value.
FRANK_PRIMARY_SERVED_NAME=frank-archon FRANK_ARCHON_SERVED_NAME=frank-archon
FRANK_PRIMARY_PORT=7770 FRANK_ARCHON_PORT=7770
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the # Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
# repo's root chat_template.jinja, vendored here; the checkpoint's own # repo's root chat_template.jinja, vendored here; the checkpoint's own
# template is baked into the .rad by rad-convert and this overrides it). # template is baked into the .rad by rad-convert and this overrides it).
# Radiance derives the reasoning/tool-call reply format from it. # Radiance derives the reasoning/tool-call reply format from it.
FRANK_PRIMARY_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
# --- engine shape (first-boot conservative; the boot ledger arbitrates) -- # --- engine shape (first-boot conservative; the boot ledger arbitrates) --
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850 # Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
# x8/x8 on root ports) made collectives legal on this host; radiance's TP # x8/x8 on root ports) made collectives legal on this host; radiance's TP
# wire is libr4d's own, not RCCL. # wire is libr4d's own, not RCCL.
FRANK_PRIMARY_TP=2 FRANK_ARCHON_TP=2
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice: # Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the # +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
# engine default. # engine default.
FRANK_PRIMARY_TP_WIRE=wht6 FRANK_ARCHON_TP_WIRE=wht6
# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to # Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
# start"); raise only after a real-traffic look at slot occupancy and the # start"); raise only after a real-traffic look at slot occupancy and the
# KV pool's headroom line. The recurrent-state cost scales with it (54 # KV pool's headroom line. The recurrent-state cost scales with it (54
# MiB/copy/card, 2 copies with speculation). # MiB/copy/card, 2 copies with speculation).
FRANK_PRIMARY_MAX_SEQS=3 FRANK_ARCHON_MAX_SEQS=3
# Served window. Native is 262,144; 200,000 is the publisher's production # Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer # value (compose) and what models.yml's contextWindow must quote — a longer
# prompt is refused by the engine, never truncated. # prompt is refused by the engine, never truncated.
FRANK_PRIMARY_MAX_MODEL_LEN=200000 FRANK_ARCHON_MAX_MODEL_LEN=200000
FRANK_PRIMARY_PLACEMENT=expert_tiered FRANK_ARCHON_PLACEMENT=expert_tiered
# Pinned host pool for the expert rows that do not fit VRAM. Native FP8 # Pinned host pool for the expert rows that do not fit VRAM. Native FP8
# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit # experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
# publisher shape's ~29 GiB — this pool is the cost of native precision. # publisher shape's ~29 GiB — this pool is the cost of native precision.
# The boot ledger prints the pool's coverage ("the pool backs N% of the # The boot ledger prints the pool's coverage ("the pool backs N% of the
# worst case ... M full-length sessions"); resize from that line, not from # worst case ... M full-length sessions"); resize from that line, not from
# arithmetic. # arithmetic.
FRANK_PRIMARY_HOST_POOL_MIB=40960 FRANK_ARCHON_HOST_POOL_MIB=40960
# VRAM left unclaimed per card (publisher default 96 MiB). # VRAM left unclaimed per card (publisher default 96 MiB).
FRANK_PRIMARY_GPU_HEADROOM_MIB=96 FRANK_ARCHON_GPU_HEADROOM_MIB=96
# VRAM split of claimed memory per card, MiB (0 = derive from the card). # VRAM split of claimed memory per card, MiB (0 = derive from the card).
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace # Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82: # it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
@@ -133,36 +133,36 @@ FRANK_PRIMARY_GPU_HEADROOM_MIB=96
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor # Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The # with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
# boot ledger arbitrates; re-derive after any payload change. # boot ledger arbitrates; re-derive after any payload change.
FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071 FRANK_ARCHON_VRAM_WEIGHTS_MIB=26071
FRANK_PRIMARY_VRAM_KV_MIB=3543 FRANK_ARCHON_VRAM_KV_MIB=3543
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers, # KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
# 2 KV heads, 256 head_dim, K+V). # 2 KV heads, 256 head_dim, K+V).
FRANK_PRIMARY_KV_DTYPE=fp8 FRANK_ARCHON_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3 # Spec decode: the checkpoint's own MTP head at the publisher's depth-3
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the # operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
# flag states it so the ledger records the intent). # flag states it so the ledger records the intent).
FRANK_PRIMARY_SPEC_TOKENS=2 FRANK_ARCHON_SPEC_TOKENS=2
FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots). # Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
FRANK_PRIMARY_PREFIX_HOST_MIB=4096 FRANK_ARCHON_PREFIX_HOST_MIB=4096
FRANK_PRIMARY_PREFIX_DISK_MIB=65536 FRANK_ARCHON_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host # Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too # RAM the cards read. `ram` REFUSES the start if the memlock limit is too
# small — the unit's LimitMEMLOCK=infinity is the other half of this law. # small — the unit's LimitMEMLOCK=infinity is the other half of this law.
FRANK_PRIMARY_NGRAM_PLACEMENT=ram FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB # TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable # prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the # pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
# plan, so the serve script refuses to start until both are raised. This is # plan, so the serve script refuses to start until both are raised. This is
# the floor it enforces (in MiB), with ~8 GiB of margin. # the floor it enforces (in MiB), with ~8 GiB of margin.
FRANK_PRIMARY_TTM_MIN_MIB=140000 FRANK_ARCHON_TTM_MIN_MIB=140000
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep # Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
# multi-flag values as one string; the serve script expands it unquoted on # multi-flag values as one string; the serve script expands it unquoted on
# purpose. # purpose.
FRANK_PRIMARY_EXTRA_ARGS= FRANK_ARCHON_EXTRA_ARGS=
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a # The .rad's minimum size in MiB (deployer gate; a truncated convert is a
# guaranteed long-timeout boot instead of a fast failure). The converted # guaranteed long-timeout boot instead of a fast failure). The converted
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the # container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
# plan-only estimate 172.76 GiB covers the data planes but not the metadata # plan-only estimate 172.76 GiB covers the data planes but not the metadata
# tail) - floor sits just under the payload. # tail) - floor sits just under the payload.
FRANK_PRIMARY_RAD_MIN_MIB=110000 FRANK_ARCHON_RAD_MIN_MIB=110000
@@ -1,8 +1,8 @@
# frank-primary.service — radiance (StillDeadcode, Apache-2.0) endpoint # frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the # (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2, # rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag # wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
# frank-primary-freetoken-pr587-20261005 is the pre-cutover state; the # frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
# retired unit/env/serve trio and Dockerfile are in that tag). # retired unit/env/serve trio and Dockerfile are in that tag).
# #
# Same port, same served alias, same container identity as the unit name; # Same port, same served alias, same container identity as the unit name;
@@ -19,7 +19,7 @@
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the # unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
# engine still runs out of dank's rootless podman storage. # engine still runs out of dank's rootless podman storage.
[Unit] [Unit]
Description=frank-primary — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2 Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
# Never let the start-rate limiter give up after a crash loop (a failed LLM # Never let the start-rate limiter give up after a crash loop (a failed LLM
@@ -33,14 +33,14 @@ Group=dank
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start. # RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
# PID1 grants infinity here — parity with the fleet's other LLM units. # PID1 grants infinity here — parity with the fleet's other LLM units.
LimitMEMLOCK=infinity LimitMEMLOCK=infinity
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env" Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
# systemd stop: wait alone cannot stop the container child; stop it # systemd stop: wait alone cannot stop the container child; stop it
# explicitly. No graceful drain contract is claimed (the OpenAI front end # explicitly. No graceful drain contract is claimed (the OpenAI front end
# holds no queue to drain — in-flight decodes are lost, which a stop # holds no queue to drain — in-flight decodes are lost, which a stop
# during a bench window accepts; radiance takes SIGTERM on the engine # during a bench window accepts; radiance takes SIGTERM on the engine
# process and podman reaps the container). # process and podman reaps the container).
ExecStop=/usr/bin/podman stop -t 120 frank-primary ExecStop=/usr/bin/podman stop -t 120 frank-archon
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The # Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
# publisher's compose healthcheck allows a 15-minute start_period; the # publisher's compose healthcheck allows a 15-minute start_period; the
# vLLM-era budget (1800 s) carried forward. # vLLM-era budget (1800 s) carried forward.
@@ -1,11 +1,11 @@
# frank-large.env — tunables for the frank-large endpoint on Dankclaw's # frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
# Strix Halo iGPU (gfx1151), host port 7770. # Strix Halo iGPU (gfx1151), host port 7770.
# #
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the # RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server, # closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune # digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under # of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
# a root unit, driven by scripts/serve-frank-large.sh. This CUTS OVER the # a root unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s): # 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
# rollback = revert the swap commit's trio; the 27B payload stays on disk at # rollback = revert the swap commit's trio; the 27B payload stays on disk at
# /home/dank/models/halogen-qwen3.8-27b/. # /home/dank/models/halogen-qwen3.8-27b/.
@@ -36,7 +36,7 @@
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 % # target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and # est-error ≈ 219k <= 262144 — the 45k gap between harness window and
# engine ceiling absorbs the estimate error with ~7x margin (robust to a # engine ceiling absorbs the estimate error with ~7x margin (robust to a
# 10 % under-count). Compaction rides on frank-secondary: input ≈ # 10 % under-count). Compaction rides on frank-zealot: input ≈
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot. # threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit # HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
# form of the engine's default, so the seam test can pin the product. # form of the engine's default, so the seam test can pin the product.
@@ -52,7 +52,7 @@
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 => # weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2 # ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host # = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ) # honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it # ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
# after deploy and re-size if the engine prints a clamp. # after deploy and re-size if the engine prints a clamp.
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy # HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
@@ -91,47 +91,47 @@
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest # Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
# is what was qualified). 0.16.2 = the version the model card names as its # is what was qualified). 0.16.2 = the version the model card names as its
# quickstart pin; RepoDigest captured at pull on 2026-10-06. # quickstart pin; RepoDigest captured at pull on 2026-10-06.
FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# Container identity: kept from the 27B era (unit ExecStop names it). # Container identity: kept from the 27B era (unit ExecStop names it).
FRANK_LARGE_CONTAINER=frank-large-halogen FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
# Old quadlet-era container identity; the serve script clears stale names. # Old quadlet-era container identity; the serve script clears stale names.
FRANK_LARGE_LEGACY_CONTAINERS=frank-large FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT # host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
# default); the engine port (8730) is never published. # default); the engine port (8730) is never published.
FRANK_LARGE_PUBLISH=7770:8731 FRANK_DRAGOON_PUBLISH=7770:8731
FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-flashnext-v2 FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated # The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's # qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
# append-only original (byte-identical live tensors, padding dropped: # append-only original (byte-identical live tensors, padding dropped:
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch. # 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
FRANK_LARGE_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
FRANK_LARGE_CHECKPOINT_BYTES=66687678464 FRANK_DRAGOON_CHECKPOINT_BYTES=66687678464
FRANK_LARGE_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037 FRANK_DRAGOON_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
# The n-gram lookup table: REQUIRED (the engine reads it beside the # The n-gram lookup table: REQUIRED (the engine reads it beside the
# checkpoint; the card's quickstart names it explicitly). From # checkpoint; the card's quickstart names it explicitly). From
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea. # peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
FRANK_LARGE_NGRAM_TABLE=qwen38-flash-next-ngram.hgn FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
FRANK_LARGE_NGRAM_BYTES=51200246144 FRANK_DRAGOON_NGRAM_BYTES=51200246144
FRANK_LARGE_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
# The vision tower: OPTIONAL on the engine, ENABLED here (image input). # The vision tower: OPTIONAL on the engine, ENABLED here (image input).
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488. # From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
FRANK_LARGE_VISION_TOWER=qwen38-flash-next-vision.hgn FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
FRANK_LARGE_VISION_BYTES=897916416 FRANK_DRAGOON_VISION_BYTES=897916416
FRANK_LARGE_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10 FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer, # The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
# mounted whole at /tokenizer. The two repos ship the same one; fetched # mounted whole at /tokenizer. The two repos ship the same one; fetched
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to # fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
# the 27B-era copy; the rest are plain git blobs — size-checked only). The # the 27B-era copy; the rest are plain git blobs — size-checked only). The
# serve gate and the deployer refuse a missing file or ANY symlink: an HF # serve gate and the deployer refuse a missing file or ANY symlink: an HF
# cache snapshot symlinks each file into blobs/, and those links dangle # cache snapshot symlinks each file into blobs/, and those links dangle
# inside the container mount. # inside the container mount.
FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json" FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset # --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
# stays at the image default, which is what was qualified) ------------------ # stays at the image default, which is what was qualified) ------------------
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit, # Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
# scripts and this file keep the frank-large name. OpenWebUI api_configs # scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
# model_ids for the :7770 backend MUST name this value. # model_ids for the :7770 backend MUST name this value.
HALOGEN_MODEL_ID=frank-dragoon HALOGEN_MODEL_ID=frank-dragoon
HALOGEN_CTX=262144 HALOGEN_CTX=262144
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-large, Strix Halo iGPU) Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -11,11 +11,11 @@ Group=dank
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the # RLIMIT smaller than the pinned set cannot fail the load. Parity with the
# other LLM units on this fleet. # other LLM units on this fleet.
LimitMEMLOCK=infinity LimitMEMLOCK=infinity
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env" Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
# systemd stop alone cannot stop the container child; stop it explicitly # systemd stop alone cannot stop the container child; stop it explicitly
# (frank-primary pattern). The engine drains on SIGTERM. # (frank-archon pattern). The engine drains on SIGTERM.
ExecStop=/usr/bin/podman stop -t 60 frank-large-halogen ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
TimeoutStartSec=1200 TimeoutStartSec=1200
TimeoutStopSec=120 TimeoutStopSec=120
# always, not on-failure: an engine that exits 0 after an internal fault # always, not on-failure: an engine that exits 0 after an internal fault
@@ -23,7 +23,7 @@ TimeoutStopSec=120
Restart=always Restart=always
RestartSec=10 RestartSec=10
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set # The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
# on a box that also hosts frank-secondary, ComfyUI and the Caddy/observability # on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
# stack; make it a late OOM candidate, not the first. # stack; make it a late OOM candidate, not the first.
OOMScoreAdjust=-500 OOMScoreAdjust=-500
@@ -1,4 +1,4 @@
# frank-secondary — llama.cpp (master, CUDA sm_86) serving # frank-zealot — llama.cpp (master, CUDA sm_86) serving
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090, # HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
# :7771. # :7771.
# #
@@ -38,7 +38,7 @@ MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensore
# Served model id — the harness/family name (Protoss rename completed to the # Served model id — the harness/family name (Protoss rename completed to the
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases # wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
# this day). The SERVICE, unit, scripts and this env file keep the # this day). The SERVICE, unit, scripts and this env file keep the
# frank-secondary name; only the alias on the wire changed. Consumers that # frank-zealot name; only the alias on the wire changed. Consumers that
# key on the served id: OpenWebUI (model row, access_grant, api_configs # key on the served id: OpenWebUI (model row, access_grant, api_configs
# model_ids) and the harness models.yml. # model_ids) and the harness models.yml.
ALIAS=frank-zealot ALIAS=frank-zealot
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary) Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -10,8 +10,8 @@ Group=dank
# CUDA pins its staging buffers, so the process needs the memlock budget even # CUDA pins its staging buffers, so the process needs the memlock budget even
# though the weights themselves are copied into VRAM. # though the weights themselves are copied into VRAM.
LimitMEMLOCK=infinity LimitMEMLOCK=infinity
Environment="FRANK_SECONDARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-secondary.env" Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-secondary.sh ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
Restart=on-failure Restart=on-failure
RestartSec=10 RestartSec=10
+1 -1
View File
@@ -24,7 +24,7 @@ KLD_OUT=/home/dank/radiance-state/kld-gate.json
: > "$KLD_OUT" : > "$KLD_OUT"
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \ cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
/home/dank/models/tpl.jinja 2>/dev/null || true /home/dank/models/tpl.jinja 2>/dev/null || true
sudo systemctl stop frank-primary >/dev/null 2>&1 || true sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$C" >/dev/null 2>&1 || true podman rm -f "$C" >/dev/null 2>&1 || true
podman run --rm --name "$C" \ podman run --rm --name "$C" \
+5 -5
View File
@@ -44,17 +44,17 @@ JSONL_EXTS = (".jsonl", ".json")
DOC_DIR = "serverAppConfigs-docs" DOC_DIR = "serverAppConfigs-docs"
FLEET_CMDS = [ FLEET_CMDS = [
"systemctl status frank-primary --no-pager", "systemctl status frank-archon --no-pager",
"journalctl -u frank-primary -n 100 --no-pager", "journalctl -u frank-archon -n 100 --no-pager",
"podman ps --format '{{.Names}} {{.Status}}'", "podman ps --format '{{.Names}} {{.Status}}'",
"podman logs --tail 50 frank-primary", "podman logs --tail 50 frank-archon",
"curl -s http://192.168.0.235:7770/health", "curl -s http://192.168.0.235:7770/health",
"curl -s http://192.168.0.235:7770/metrics | grep radiance_", "curl -s http://192.168.0.235:7770/metrics | grep radiance_",
"df -h /home/dank | tail -1", "df -h /home/dank | tail -1",
"free -g", "free -g",
"sudo systemctl restart frank-primary", "sudo systemctl restart frank-archon",
"cd /home/dank/serverAppConfigs && git pull --ff-only", "cd /home/dank/serverAppConfigs && git pull --ff-only",
"./scripts/deploy.sh dankserver --service frank-primary", "./scripts/deploy.sh dankserver --service frank-archon",
"./tests/run.sh 2>&1 | tail -5", "./tests/run.sh 2>&1 | tail -5",
"jq . /home/dank/models/rad/index.json | head -40", "jq . /home/dank/models/rad/index.json | head -40",
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"", "python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
@@ -2,7 +2,7 @@
# #
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The # STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of # set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
# the deployed frank-primary shape. Nothing in scripts/ mounts these files. # the deployed frank-archon shape. Nothing in scripts/ mounts these files.
# #
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred): # WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
# #
+1 -1
View File
@@ -1,6 +1,6 @@
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap. # gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
# #
# STATUS: NOT part of the deployed frank-primary shape. It exists because the # STATUS: NOT part of the deployed frank-archon shape. It exists because the
# fork's plan under-models this model's KV, and it is the enabler for a future # fork's plan under-models this model's KV, and it is the enabler for a future
# MTP attempt (the production GLM shape runs without MTP and boots fine on the # MTP attempt (the production GLM shape runs without MTP and boots fine on the
# plan's own reserve). # plan's own reserve).
@@ -1,6 +1,6 @@
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next # qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official # (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
# base) as a radiance container for frank-primary: EVERY routed expert -- gate_up AND down -- # base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law # through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served # ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 + # beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
+2 -2
View File
@@ -30,9 +30,9 @@ KLD_REF_DIR=/home/dank/corpora/kld-ref
CALIB_DIR=/home/dank/corpora/calib CALIB_DIR=/home/dank/corpora/calib
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
restore() { sudo systemctl start frank-primary >/dev/null 2>&1 || true; } restore() { sudo systemctl start frank-archon >/dev/null 2>&1 || true; }
trap restore EXIT trap restore EXIT
sudo systemctl stop frank-primary >/dev/null 2>&1 || true sudo systemctl stop frank-archon >/dev/null 2>&1 || true
podman rm -f "$C" >/dev/null 2>&1 || true podman rm -f "$C" >/dev/null 2>&1 || true
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
@@ -1,10 +1,10 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-primary.sh — frank-primary endpoint: radiance (StillDeadcode, # serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at # Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
# NATIVE block-FP8 precision from a rad-convert container # NATIVE block-FP8 precision from a rad-convert container
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2, # (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587, # wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
# one card; git tag frank-primary-freetoken-pr587-20261005 is the # one card; git tag frank-archon-freetoken-pr587-20261005 is the
# pre-cutover state). # pre-cutover state).
# #
# The run line below IS the engine argv: the image's entrypoint is the # The run line below IS the engine argv: the image's entrypoint is the
@@ -16,18 +16,18 @@
# 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV, # 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV,
# MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM. # MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM.
# #
# The root unit (scripts/frank-primary.service, User=dank, # The root unit (scripts/frank-archon.service, User=dank,
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools + # LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools +
# the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned # the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned
# pages, which is ALSO why this script refuses to start when ttm's # pages, which is ALSO why this script refuses to start when ttm's
# pages_limit/pages_max cap is under FRANK_PRIMARY_TTM_MIN_MIB (see below). # pages_limit/pages_max cap is under FRANK_ARCHON_TTM_MIN_MIB (see below).
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}" ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
if [ ! -f "$ENV_FILE" ]; then if [ ! -f "$ENV_FILE" ]; then
echo "ERROR: frank-primary env file absent: $ENV_FILE" >&2 echo "ERROR: frank-archon env file absent: $ENV_FILE" >&2
exit 1 exit 1
fi fi
set -a set -a
@@ -35,74 +35,74 @@ set -a
source "$ENV_FILE" source "$ENV_FILE"
set +a set +a
: "${FRANK_PRIMARY_IMAGE:?FRANK_PRIMARY_IMAGE must name the radiance image}" : "${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
: "${FRANK_PRIMARY_MODEL_DIR:?FRANK_PRIMARY_MODEL_DIR must name the .rad dir}" : "${FRANK_ARCHON_MODEL_DIR:?FRANK_ARCHON_MODEL_DIR must name the .rad dir}"
: "${FRANK_PRIMARY_MODEL_FILE:?FRANK_PRIMARY_MODEL_FILE must name the .rad file}" : "${FRANK_ARCHON_MODEL_FILE:?FRANK_ARCHON_MODEL_FILE must name the .rad file}"
: "${FRANK_PRIMARY_STATE_DIR:?FRANK_PRIMARY_STATE_DIR must name the /data state dir}" : "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the /data state dir}"
: "${FRANK_PRIMARY_CONTAINER:?FRANK_PRIMARY_CONTAINER must name the container}" : "${FRANK_ARCHON_CONTAINER:?FRANK_ARCHON_CONTAINER must name the container}"
: "${FRANK_PRIMARY_SERVED_NAME:?FRANK_PRIMARY_SERVED_NAME must name the alias}" : "${FRANK_ARCHON_SERVED_NAME:?FRANK_ARCHON_SERVED_NAME must name the alias}"
: "${FRANK_PRIMARY_PORT:?FRANK_PRIMARY_PORT must be the port}" : "${FRANK_ARCHON_PORT:?FRANK_ARCHON_PORT must be the port}"
: "${FRANK_PRIMARY_TEMPLATE:?FRANK_PRIMARY_TEMPLATE must name the chat template}" : "${FRANK_ARCHON_TEMPLATE:?FRANK_ARCHON_TEMPLATE must name the chat template}"
: "${FRANK_PRIMARY_TP:?FRANK_PRIMARY_TP must be set}" : "${FRANK_ARCHON_TP:?FRANK_ARCHON_TP must be set}"
: "${FRANK_PRIMARY_TP_WIRE:?FRANK_PRIMARY_TP_WIRE must be set}" : "${FRANK_ARCHON_TP_WIRE:?FRANK_ARCHON_TP_WIRE must be set}"
: "${FRANK_PRIMARY_MAX_SEQS:?FRANK_PRIMARY_MAX_SEQS must be set}" : "${FRANK_ARCHON_MAX_SEQS:?FRANK_ARCHON_MAX_SEQS must be set}"
: "${FRANK_PRIMARY_MAX_MODEL_LEN:?FRANK_PRIMARY_MAX_MODEL_LEN must be set}" : "${FRANK_ARCHON_MAX_MODEL_LEN:?FRANK_ARCHON_MAX_MODEL_LEN must be set}"
: "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}" : "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
: "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}" : "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
: "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}" : "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
: "${FRANK_PRIMARY_VRAM_WEIGHTS_MIB:?FRANK_PRIMARY_VRAM_WEIGHTS_MIB must be set}" : "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
: "${FRANK_PRIMARY_VRAM_KV_MIB:?FRANK_PRIMARY_VRAM_KV_MIB must be set}" : "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}" : "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
: "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}" : "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
: "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}" : "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
: "${FRANK_PRIMARY_PREFIX_HOST_MIB:?FRANK_PRIMARY_PREFIX_HOST_MIB must be set}" : "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
: "${FRANK_PRIMARY_PREFIX_DISK_MIB:?FRANK_PRIMARY_PREFIX_DISK_MIB must be set}" : "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
: "${FRANK_PRIMARY_NGRAM_PLACEMENT:?FRANK_PRIMARY_NGRAM_PLACEMENT must be set}" : "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
: "${FRANK_PRIMARY_TTM_MIN_MIB:?FRANK_PRIMARY_TTM_MIN_MIB must be set}" : "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
: "${FRANK_PRIMARY_RAD_MIN_MIB:?FRANK_PRIMARY_RAD_MIN_MIB must be set}" : "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists # The image gate is the digest-pinned ref itself: a name@sha256 ref exists
# in storage only if THAT manifest was pulled — a retagged look-alike or a # in storage only if THAT manifest was pulled — a retagged look-alike or a
# newer latest does not satisfy it. `podman image exists` takes a # newer latest does not satisfy it. `podman image exists` takes a
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30 # name@digest ref in one command — no pipefail/grep race (the 2026-09-30
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman # lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
# storage (root unit, User=dank — the fleet's frank-primary shape). # storage (root unit, User=dank — the fleet's frank-archon shape).
if ! podman image exists "$FRANK_PRIMARY_IMAGE"; then if ! podman image exists "$FRANK_ARCHON_IMAGE"; then
echo "ERROR: radiance image absent: $FRANK_PRIMARY_IMAGE" >&2 echo "ERROR: radiance image absent: $FRANK_ARCHON_IMAGE" >&2
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2 echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
exit 1 exit 1
fi fi
# The payload gate: the .rad container file, sized. A truncated convert # The payload gate: the .rad container file, sized. A truncated convert
# would otherwise buy a guaranteed long-timeout boot instead of this fast # would otherwise buy a guaranteed long-timeout boot instead of this fast
# failure. The staging dir / raw checkpoint are NOT this script's concern # failure. The staging dir / raw checkpoint are NOT this script's concern
# (scripts/build-frank-primary-rad.sh built the file; its rad-info output # (scripts/build-frank-archon-rad.sh built the file; its rad-info output
# is the record). # is the record).
RAD="$FRANK_PRIMARY_MODEL_DIR/$FRANK_PRIMARY_MODEL_FILE" RAD="$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE"
if [ ! -f "$RAD" ]; then if [ ! -f "$RAD" ]; then
echo "ERROR: .rad container missing: $RAD" >&2 echo "ERROR: .rad container missing: $RAD" >&2
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2 echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
exit 1 exit 1
fi fi
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 )) RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
if [ "$RAD_MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then if [ "$RAD_MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_PRIMARY_RAD_MIN_MIB" >&2 echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_ARCHON_RAD_MIN_MIB" >&2
echo "frank-primary: the convert was truncated — rebuild it" >&2 echo "frank-archon: the convert was truncated — rebuild it" >&2
exit 1 exit 1
fi fi
if [ ! -f "$FRANK_PRIMARY_TEMPLATE" ]; then if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
echo "ERROR: chat template missing: $FRANK_PRIMARY_TEMPLATE" >&2 echo "ERROR: chat template missing: $FRANK_ARCHON_TEMPLATE" >&2
exit 1 exit 1
fi fi
mkdir -p "$FRANK_PRIMARY_STATE_DIR/kvcache/frank-primary" mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
# The engine's create is NOT recursive: with only the state dir mounted, its # The engine's create is NOT recursive: with only the state dir mounted, its
# "cannot create /data/kvcache/frank-primary: No such file or directory" # "cannot create /data/kvcache/frank-archon: No such file or directory"
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create # left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
# the full path on the host so the tier opens on first boot. # the full path on the host so the tier opens on first boot.
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune" mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"): # rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot; # the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so # $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
@@ -118,7 +118,7 @@ fi
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB # TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's # prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
# pages_limit (the current kernel exposes NO pages_max; the cap function # pages_limit (the current kernel exposes NO pages_max; the cap function
# takes the min over the knobs that exist). Below FRANK_PRIMARY_TTM_MIN_MIB # takes the min over the knobs that exist). Below FRANK_ARCHON_TTM_MIN_MIB
# the load dies mid-pin with an allocator error instead of this named # the load dies mid-pin with an allocator error instead of this named
# refusal — raise the knob first (runtime sysfs write; kernel cmdline # refusal — raise the knob first (runtime sysfs write; kernel cmdline
# fallback + one reboot only if the file is read-only), see the runbook. # fallback + one reboot only if the file is read-only), see the runbook.
@@ -138,22 +138,22 @@ ttm_cap_mib() {
echo "${cur:-0}" echo "${cur:-0}"
} }
TTM_CAP="$(ttm_cap_mib)" TTM_CAP="$(ttm_cap_mib)"
if [ "$TTM_CAP" -lt "$FRANK_PRIMARY_TTM_MIN_MIB" ]; then if [ "$TTM_CAP" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_PRIMARY_TTM_MIN_MIB" >&2 echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
echo "frank-primary: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_PRIMARY_TTM_MIN_MIB in 4 KiB pages" >&2 echo "frank-archon: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_ARCHON_TTM_MIN_MIB in 4 KiB pages" >&2
exit 1 exit 1
fi fi
# A crashed/killed unit can leave the container name registered (systemd # A crashed/killed unit can leave the container name registered (systemd
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy # SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
# identities (the FreeToken-era frank-primary-qwen, the retired R9V # identities (the FreeToken-era frank-archon-qwen, the retired R9V
# container and the ad hoc experiment containers all hold this GPU); all # container and the ad hoc experiment containers all hold this GPU); all
# idempotent. # idempotent.
for CTR in $FRANK_PRIMARY_LEGACY_CONTAINERS "$FRANK_PRIMARY_CONTAINER"; do for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true podman rm -f "$CTR" >/dev/null 2>&1 || true
done done
LOG=/home/dank/logs/frank_primary.log LOG=/home/dank/logs/frank_archon.log
mkdir -p "$(dirname "$LOG")" mkdir -p "$(dirname "$LOG")"
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml): # Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
@@ -166,41 +166,41 @@ mkdir -p "$(dirname "$LOG")"
# /dev/kfd + /dev/dri (render/video groups are moot for a root container). # /dev/kfd + /dev/dri (render/video groups are moot for a root container).
# The template rides a single-file mount; the .rad dir is ro; /data holds # The template rides a single-file mount; the .rad dir is ro; /data holds
# the prefix-cache disk tier. # the prefix-cache disk tier.
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
--network host \ --network host \
--init \ --init \
--security-opt seccomp=unconfined \ --security-opt seccomp=unconfined \
--ulimit memlock=-1:-1 \ --ulimit memlock=-1:-1 \
--device /dev/kfd --device /dev/dri \ --device /dev/kfd --device /dev/dri \
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \ -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-v "$FRANK_PRIMARY_STATE_DIR":/data \ -v "$FRANK_ARCHON_STATE_DIR":/data \
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \ -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \ -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
"$FRANK_PRIMARY_IMAGE" \ "$FRANK_ARCHON_IMAGE" \
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \ --model "/models/$FRANK_ARCHON_MODEL_FILE" \
--host :: --port "$FRANK_PRIMARY_PORT" \ --host :: --port "$FRANK_ARCHON_PORT" \
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \ --served-model-name "$FRANK_ARCHON_SERVED_NAME" \
--override-chat-template /templates/template.jinja \ --override-chat-template /templates/template.jinja \
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \ --tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \ --max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \ --max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
--placement "$FRANK_PRIMARY_PLACEMENT" \ --placement "$FRANK_ARCHON_PLACEMENT" \
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \ --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \ --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \ --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \ --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \ --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \ --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
--prefix-cache-dir /data/kvcache/frank-primary \ --prefix-cache-dir /data/kvcache/frank-archon \
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \ --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \ --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \ --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \ --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \ ${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
> "$LOG" 2>&1 > "$LOG" 2>&1
# From here on, this script's lifetime IS the container's: wait returns the # From here on, this script's lifetime IS the container's: wait returns the
# container's exit code, so a crash is a nonzero ExecStart and the unit's # container's exit code, so a crash is a nonzero ExecStart and the unit's
# Restart=always is real. (systemd stop goes through ExecStop in the unit — # Restart=always is real. (systemd stop goes through ExecStop in the unit —
# podman wait alone cannot stop the container child.) # podman wait alone cannot stop the container child.)
exec podman wait "$FRANK_PRIMARY_CONTAINER" exec podman wait "$FRANK_ARCHON_CONTAINER"
@@ -1,13 +1,13 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-large.sh — frank-large endpoint: halogen-flash-server 0.16.2 # serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server, # (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned in frank-large.env) serving the Quat3rnion abliterated # digest-pinned in frank-dragoon.env) serving the Quat3rnion abliterated
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on # Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the # Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
# halogen 27B-dense shape — revert the swap commit's trio to roll back # halogen 27B-dense shape — revert the swap commit's trio to roll back
# (the 27B payload stays on disk). # (the 27B payload stays on disk).
# #
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-primary # The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon
# pattern): the image starts engine + OpenAI frontend in one container. # pattern): the image starts engine + OpenAI frontend in one container.
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is # The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
# published as host 7770; the engine's own token port (8730, no auth) # published as host 7770; the engine's own token port (8730, no auth)
@@ -26,13 +26,13 @@
# Why containers instead of a bare binary: the payload is .hgn and only # Why containers instead of a bare binary: the payload is .hgn and only
# the flash-server line reads it; it ships as a container; the root unit # the flash-server line reads it; it ships as a container; the root unit
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session, # keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
# exactly like frank-primary's. # exactly like frank-archon's.
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}" ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
if [ ! -f "$ENV_FILE" ]; then if [ ! -f "$ENV_FILE" ]; then
echo "frank-large: env file not found: $ENV_FILE" >&2 echo "frank-dragoon: env file not found: $ENV_FILE" >&2
exit 1 exit 1
fi fi
set -a set -a
@@ -40,17 +40,17 @@ set -a
source "$ENV_FILE" source "$ENV_FILE"
set +a set +a
: "${FRANK_LARGE_IMAGE:?FRANK_LARGE_IMAGE must name the digest-pinned image}" : "${FRANK_DRAGOON_IMAGE:?FRANK_DRAGOON_IMAGE must name the digest-pinned image}"
: "${FRANK_LARGE_CONTAINER:?FRANK_LARGE_CONTAINER must name the container}" : "${FRANK_DRAGOON_CONTAINER:?FRANK_DRAGOON_CONTAINER must name the container}"
: "${FRANK_LARGE_PUBLISH:?FRANK_LARGE_PUBLISH must be host:container}" : "${FRANK_DRAGOON_PUBLISH:?FRANK_DRAGOON_PUBLISH must be host:container}"
: "${FRANK_LARGE_MODEL_DIR:?FRANK_LARGE_MODEL_DIR must name the payload dir}" : "${FRANK_DRAGOON_MODEL_DIR:?FRANK_DRAGOON_MODEL_DIR must name the payload dir}"
: "${FRANK_LARGE_CHECKPOINT:?FRANK_LARGE_CHECKPOINT must name the .hgn inside it}" : "${FRANK_DRAGOON_CHECKPOINT:?FRANK_DRAGOON_CHECKPOINT must name the .hgn inside it}"
: "${FRANK_LARGE_CHECKPOINT_BYTES:?FRANK_LARGE_CHECKPOINT_BYTES must pin the size}" : "${FRANK_DRAGOON_CHECKPOINT_BYTES:?FRANK_DRAGOON_CHECKPOINT_BYTES must pin the size}"
: "${FRANK_LARGE_NGRAM_TABLE:?FRANK_LARGE_NGRAM_TABLE must name the lookup table}" : "${FRANK_DRAGOON_NGRAM_TABLE:?FRANK_DRAGOON_NGRAM_TABLE must name the lookup table}"
: "${FRANK_LARGE_NGRAM_BYTES:?FRANK_LARGE_NGRAM_BYTES must pin the table size}" : "${FRANK_DRAGOON_NGRAM_BYTES:?FRANK_DRAGOON_NGRAM_BYTES must pin the table size}"
: "${FRANK_LARGE_VISION_TOWER:?FRANK_LARGE_VISION_TOWER must name the tower}" : "${FRANK_DRAGOON_VISION_TOWER:?FRANK_DRAGOON_VISION_TOWER must name the tower}"
: "${FRANK_LARGE_VISION_BYTES:?FRANK_LARGE_VISION_BYTES must pin the tower size}" : "${FRANK_DRAGOON_VISION_BYTES:?FRANK_DRAGOON_VISION_BYTES must pin the tower size}"
: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}" : "${FRANK_DRAGOON_TOKENIZER_FILES:?FRANK_DRAGOON_TOKENIZER_FILES must list the tokenizer dir contents}"
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}" : "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}" : "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}" : "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
@@ -62,9 +62,9 @@ set +a
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the # podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
# 2026-09-30 deploy refusal). # 2026-09-30 deploy refusal).
DIGESTS="$(podman images --format '{{.RepoDigests}}')" DIGESTS="$(podman images --format '{{.RepoDigests}}')"
if ! grep -qF "${FRANK_LARGE_IMAGE#*@}" <<<"$DIGESTS"; then if ! grep -qF "${FRANK_DRAGOON_IMAGE#*@}" <<<"$DIGESTS"; then
echo "frank-large: image $FRANK_LARGE_IMAGE not in podman storage;" >&2 echo "frank-dragoon: image $FRANK_DRAGOON_IMAGE not in podman storage;" >&2
echo " pull it: podman pull ${FRANK_LARGE_IMAGE%@*}" >&2 echo " pull it: podman pull ${FRANK_DRAGOON_IMAGE%@*}" >&2
exit 1 exit 1
fi fi
@@ -74,45 +74,45 @@ fi
ck() { ck() {
# ck <env-name> <path> <expected-bytes> # ck <env-name> <path> <expected-bytes>
if [ ! -f "$2" ]; then if [ ! -f "$2" ]; then
echo "frank-large: $1 $2 missing;" >&2 echo "frank-dragoon: $1 $2 missing;" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1 exit 1
fi fi
local sz local sz
sz="$(stat -c %s "$2")" sz="$(stat -c %s "$2")"
if [ "$sz" != "$3" ]; then if [ "$sz" != "$3" ]; then
echo "frank-large: $1 is $sz bytes, expected $3 (partial download?)" >&2 echo "frank-dragoon: $1 is $sz bytes, expected $3 (partial download?)" >&2
exit 1 exit 1
fi fi
} }
ck checkpoint "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_CHECKPOINT" "$FRANK_LARGE_CHECKPOINT_BYTES" ck checkpoint "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" "$FRANK_DRAGOON_CHECKPOINT_BYTES"
ck n-gram-table "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_NGRAM_TABLE" "$FRANK_LARGE_NGRAM_BYTES" ck n-gram-table "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" "$FRANK_DRAGOON_NGRAM_BYTES"
ck vision-tower "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_VISION_TOWER" "$FRANK_LARGE_VISION_BYTES" ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_DRAGOON_VISION_BYTES"
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache # The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
# snapshot symlinks each file into a sibling blobs/, and those symlinks # snapshot symlinks each file into a sibling blobs/, and those symlinks
# dangle inside the container mount — so the directory must be real and # dangle inside the container mount — so the directory must be real and
# every named file must be a regular file, not a link. # every named file must be a regular file, not a link.
TOK_DIR="$FRANK_LARGE_MODEL_DIR/tokenizer" TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
echo "frank-large: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2 echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1 exit 1
fi fi
for TF in $FRANK_LARGE_TOKENIZER_FILES; do for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
echo "frank-large: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2 echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2 echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
exit 1 exit 1
fi fi
done done
# GPU access is mandatory but the node paths are variables so the seam can # GPU access is mandatory but the node paths are variables so the seam can
# drive the gate on a box without an AMD GPU. # drive the gate on a box without an AMD GPU.
DEV_KFD="${FRANK_LARGE_DEV_KFD:-/dev/kfd}" DEV_KFD="${FRANK_DRAGOON_DEV_KFD:-/dev/kfd}"
DEV_DRI="${FRANK_LARGE_DEV_DRI:-/dev/dri}" DEV_DRI="${FRANK_DRAGOON_DEV_DRI:-/dev/dri}"
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
echo "frank-large: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2 echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
exit 1 exit 1
fi fi
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run. # Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
@@ -125,15 +125,15 @@ fi
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and # self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
# WAIT for the RAM to clear, so a transient dip boots once instead of # WAIT for the RAM to clear, so a transient dip boots once instead of
# crash-looping; if it never clears, refuse loudly. # crash-looping; if it never clears, refuse loudly.
MEMINFO="${FRANK_LARGE_MEMINFO:-/proc/meminfo}" MEMINFO="${FRANK_DRAGOON_MEMINFO:-/proc/meminfo}"
PIN_KB=$(( FRANK_LARGE_CHECKPOINT_BYTES / 1024 )) PIN_KB=$(( FRANK_DRAGOON_CHECKPOINT_BYTES / 1024 ))
NEED_KB=$(( PIN_KB + 16 * 1048576 )) NEED_KB=$(( PIN_KB + 16 * 1048576 ))
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; } ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB="" MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
WAIT_S="${FRANK_LARGE_RAM_WAIT:-600}" WAIT_S="${FRANK_DRAGOON_RAM_WAIT:-600}"
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; } GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
echo "frank-large: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2 echo "frank-dragoon: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
deadline=$(( $(date +%s) + WAIT_S )) deadline=$(( $(date +%s) + WAIT_S ))
while [ "$(date +%s)" -lt "$deadline" ]; do while [ "$(date +%s)" -lt "$deadline" ]; do
sleep 5 sleep 5
@@ -143,16 +143,16 @@ if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
fi fi
done done
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
echo "frank-large: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2 echo "frank-dragoon: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
exit 1 exit 1
fi fi
echo "frank-large: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2 echo "frank-dragoon: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
fi fi
for CTR in ${FRANK_LARGE_LEGACY_CONTAINERS:-} "$FRANK_LARGE_CONTAINER"; do for CTR in ${FRANK_DRAGOON_LEGACY_CONTAINERS:-} "$FRANK_DRAGOON_CONTAINER"; do
podman rm -f "$CTR" >/dev/null 2>&1 || true podman rm -f "$CTR" >/dev/null 2>&1 || true
done done
LOG="${FRANK_LARGE_LOG:-/home/dank/logs/frank_large.log}" LOG="${FRANK_DRAGOON_LOG:-/home/dank/logs/frank_dragoon.log}"
mkdir -p "$(dirname "$LOG")" mkdir -p "$(dirname "$LOG")"
# --ipc=host for the udev-granted supplementary group, seccomp unconfined # --ipc=host for the udev-granted supplementary group, seccomp unconfined
# for the ROCm ioctl surface, and unbounded memlock (the unit's # for the ROCm ioctl surface, and unbounded memlock (the unit's
@@ -163,17 +163,17 @@ mkdir -p "$(dirname "$LOG")"
# and both sibling serve scripts pass no nofile either. No comment lines # and both sibling serve scripts pass no nofile either. No comment lines
# inside the backslash chain — a mid-chain # terminates the continuation # inside the backslash chain — a mid-chain # terminates the continuation
# (the 2026-10-05 exit-127 crash loop on the sibling serve script). # (the 2026-10-05 exit-127 crash loop on the sibling serve script).
podman run --name "$FRANK_LARGE_CONTAINER" \ podman run --name "$FRANK_DRAGOON_CONTAINER" \
--publish "$FRANK_LARGE_PUBLISH" \ --publish "$FRANK_DRAGOON_PUBLISH" \
--ipc=host \ --ipc=host \
--ulimit memlock=-1:-1 \ --ulimit memlock=-1:-1 \
--device "$DEV_KFD" --device "$DEV_DRI" \ --device "$DEV_KFD" --device "$DEV_DRI" \
--group-add keep-groups \ --group-add keep-groups \
--security-opt seccomp=unconfined \ --security-opt seccomp=unconfined \
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \ -e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
-e HALOGEN_CHECKPOINT="/models/$FRANK_LARGE_CHECKPOINT" \ -e HALOGEN_CHECKPOINT="/models/$FRANK_DRAGOON_CHECKPOINT" \
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_LARGE_NGRAM_TABLE" \ -e HALOGEN_NGRAM_TABLE="/models/$FRANK_DRAGOON_NGRAM_TABLE" \
-e HALOGEN_VISION_TOWER="/models/$FRANK_LARGE_VISION_TOWER" \ -e HALOGEN_VISION_TOWER="/models/$FRANK_DRAGOON_VISION_TOWER" \
-e HALOGEN_TOKENIZER=/tokenizer \ -e HALOGEN_TOKENIZER=/tokenizer \
-e HALOGEN_CTX="$HALOGEN_CTX" \ -e HALOGEN_CTX="$HALOGEN_CTX" \
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \ -e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
@@ -187,9 +187,9 @@ podman run --name "$FRANK_LARGE_CONTAINER" \
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \ -e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \ -e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \ ${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
-v "$FRANK_LARGE_MODEL_DIR:/models:ro" \ -v "$FRANK_DRAGOON_MODEL_DIR:/models:ro" \
-v "$FRANK_LARGE_MODEL_DIR/tokenizer:/tokenizer:ro" \ -v "$FRANK_DRAGOON_MODEL_DIR/tokenizer:/tokenizer:ro" \
"$FRANK_LARGE_IMAGE" \ "$FRANK_DRAGOON_IMAGE" \
> "$LOG" 2>&1 > "$LOG" 2>&1
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's # FOREGROUND `podman run`: the engine's stdout/stderr are this script's
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as # Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
# frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86). # frank-zealot on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
# #
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations # 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the # (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
@@ -24,11 +24,11 @@
# Sampling: llama.cpp takes it per request (the fleet harness sends the # Sampling: llama.cpp takes it per request (the fleet harness sends the
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20). # publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
# #
# Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides. # Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_SECONDARY_ENV_FILE:-$SCRIPT_DIR/frank-secondary.env}" ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
if [[ -f "$ENV_FILE" ]]; then if [[ -f "$ENV_FILE" ]]; then
set -a set -a
# shellcheck disable=SC1090 # shellcheck disable=SC1090
@@ -83,11 +83,11 @@ fi
# has not finished, and Restart=always would turn it into a crash loop that # has not finished, and Restart=always would turn it into a crash loop that
# reads like a runtime fault. # reads like a runtime fault.
for path in "$LLAMA_BIN" "$MODEL_FILE"; do for path in "$LLAMA_BIN" "$MODEL_FILE"; do
[[ -e "$path" ]] || { echo "frank-secondary: missing $path" >&2; exit 1; } [[ -e "$path" ]] || { echo "frank-zealot: missing $path" >&2; exit 1; }
done done
EXTRA_ARGS=() EXTRA_ARGS=()
if [[ -n "$MMPROJ_FILE" ]]; then if [[ -n "$MMPROJ_FILE" ]]; then
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-secondary: missing $MMPROJ_FILE" >&2; exit 1; } [[ -e "$MMPROJ_FILE" ]] || { echo "frank-zealot: missing $MMPROJ_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE") EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
fi fi
@@ -97,7 +97,7 @@ fi
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX") EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
if [[ -n "$SPEC_DRAFT_FILE" ]]; then if [[ -n "$SPEC_DRAFT_FILE" ]]; then
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-secondary: missing $SPEC_DRAFT_FILE" >&2; exit 1; } [[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-zealot: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE") EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
fi fi
fi fi
@@ -108,7 +108,7 @@ nvidia-modprobe -u -c=0 2>/dev/null || true
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0), # Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so # and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
# the q8_0 KV contract in frank-secondary.env is a real path — without this # the q8_0 KV contract in frank-zealot.env is a real path — without this
# flag or that build option the engine silently falls back and the # flag or that build option the engine silently falls back and the
# contract becomes a lie. # contract becomes a lie.
ARGS=( ARGS=(
+11 -11
View File
@@ -145,12 +145,12 @@ for path in "${CHANGED[@]}"; do
add_test tests/homepage_caddy.bats add_test tests/homepage_caddy.bats
add_test tests/deployment_scoping.bats add_test tests/deployment_scoping.bats
;; ;;
# frank-large is a rootless quadlet again since 2026-09-16 (the # frank-dragoon is a rootless quadlet again since 2026-09-16 (the
# halogen-flash-server restore). Its unit carries the fleet's GPU and # halogen-flash-server restore). Its unit carries the fleet's GPU and
# podman posture, so it gets its own seam rather than the generic # podman posture, so it gets its own seam rather than the generic
# fallback below. # fallback below.
dankclaw/frank-large/*) dankclaw/frank-dragoon/*)
add_test tests/frank_large_unit.bats add_test tests/frank_dragoon_unit.bats
add_test tests/deployment_scoping.bats add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats add_test tests/docs_completeness.bats
;; ;;
@@ -187,29 +187,29 @@ for path in "${CHANGED[@]}"; do
# as additions, with "no focused-test mapping". # as additions, with "no focused-test mapping".
.AGENTTRASH/*|*/.AGENTTRASH/*) .AGENTTRASH/*|*/.AGENTTRASH/*)
;; ;;
# frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the # frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the # two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all # Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
# live in scripts/, which is why this case has to precede the generic # live in scripts/, which is why this case has to precede the generic
# scripts/*.service and scripts/* fallbacks below. # scripts/*.service and scripts/* fallbacks below.
scripts/frank-primary.*|scripts/serve-frank-primary.sh) scripts/frank-archon.*|scripts/serve-frank-archon.sh)
add_test tests/frank_primary_unit.bats add_test tests/frank_archon_unit.bats
add_test tests/llm_units.bats add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats add_test tests/scripts_syntax.bats
;; ;;
# frank-large's chat template lives in scripts/ (moved there on # frank-dragoon's chat template lives in scripts/ (moved there on
# 2026-09-26 when its old home was retired): a template edit changes # 2026-09-26 when its old home was retired): a template edit changes
# what the live endpoint renders, so it selects the large unit seam. # what the live endpoint renders, so it selects the large unit seam.
scripts/qwen-sharp-v22.4.0.jinja) scripts/qwen-sharp-v22.4.0.jinja)
add_test tests/frank_large_unit.bats add_test tests/frank_dragoon_unit.bats
;; ;;
# frank-secondary is llama.cpp on the 3090 as a ROOT unit: a native CUDA # frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
# binary, so no generated quadlet is involved. Its env, unit and serve # binary, so no generated quadlet is involved. Its env, unit and serve
# script all live in scripts/, which is why this case has to precede # script all live in scripts/, which is why this case has to precede
# the generic scripts/*.service and scripts/* fallbacks below. # the generic scripts/*.service and scripts/* fallbacks below.
scripts/frank-secondary.*|scripts/serve-frank-secondary.sh) scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
add_test tests/frank_secondary_unit.bats add_test tests/frank_zealot_unit.bats
add_test tests/llm_units.bats add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats add_test tests/deployment_scoping.bats
add_test tests/scripts_syntax.bats add_test tests/scripts_syntax.bats
+10 -10
View File
@@ -16,13 +16,13 @@ this file is repo-managed; edits are harness-side only.
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire **Model ids are the StarCraft unit names (official 2026-10-08, on the wire
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance), since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw `frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
:7771, the 3090 compaction hub) — formerly `frank-primary`, `frank-large`, :7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
`frank-secondary`. The SERVICES keep the old names (units, scripts, env `frank-zealot`. The SERVICES keep the old names (units, scripts, env
files, runbook pages); pages written in the old era name the service, files, runbook pages); pages written in the old era name the service,
the harness names the model. The wire now matches: each endpoint's the harness names the model. The wire now matches: each endpoint's
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI `/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
picker/grants, the homepage docs and the raw API all use one id per model picker/grants, the homepage docs and the raw API all use one id per model
(see [[OpenWebUI]] for the grant re-key and the [[frank-primary-radiance-swift15-w4dn8]] (see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
note on radiance echoing the requested name). note on radiance echoing the requested name).
## Compaction chain arithmetic (verify before ANY window change) ## Compaction chain arithmetic (verify before ANY window change)
@@ -45,14 +45,14 @@ The arithmetic (the coupled contract every endpoint's maxTokens comment
encodes): encodes):
1. **Source trigger** = `thresholdPercent × source contextWindow` 1. **Source trigger** = `thresholdPercent × source contextWindow`
(0.70 × 200,704 = 140,492 for frank-primary on the current shape) (0.70 × 200,704 = 140,492 for frank-archon on the current shape)
2. **Output budget** = capped at the target model's `maxTokens` 2. **Output budget** = capped at the target model's `maxTokens`
(32,768 for frank-secondary; 36,864 frank-primary; 57,344 frank-large — the (32,768 for frank-zealot; 36,864 frank-archon; 57,344 frank-dragoon — the
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it) largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot 3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤ window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
200,704 (frank-primary); 140,490 + 57,344 = 197,834 ≤ 200,700 200,704 (frank-archon); 140,490 + 57,344 = 197,834 ≤ 200,700
(frank-large); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-secondary). (frank-dragoon); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-zealot).
Procedure (recompute from the LIVE files every time — windows rotate with Procedure (recompute from the LIVE files every time — windows rotate with
every swap; the numbers above are the 2026-10-05 state): every swap; the numbers above are the 2026-10-05 state):
@@ -112,19 +112,19 @@ Use **wire names**, not OMP camelCase. Two traps:
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the `qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
*auto-detected* `thinkingFormat` + local baseUrl, **before** *auto-detected* `thinkingFormat` + local baseUrl, **before**
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in `applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
models.yml does **not** enable it. Custom-named models (`frank-large`) models.yml does **not** enable it. Custom-named models (`frank-dragoon`)
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`. match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy 2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
also sets it — `Object.assign` is shallow and replaces the whole object, also sets it — `Object.assign` is shallow and replaces the whole object,
silently dropping the `reasoning_effort` kwarg the compat policy set. silently dropping the `reasoning_effort` kwarg the compat policy set.
Top-level `preserve_thinking` is honoured by halogen. (frank-primary's Top-level `preserve_thinking` is honoured by halogen. (frank-archon's
FreeToken entry currently carries a `chat_template_kwargs` extraBody with FreeToken entry currently carries a `chat_template_kwargs` extraBody with
a comment asking to verify-and-prune — the exact trap to check at that a comment asking to verify-and-prune — the exact trap to check at that
endpoint's next swap.) endpoint's next swap.)
**Gotcha: silent greedy decode.** halogen has **no server-side sampler **Gotcha: silent greedy decode.** halogen has **no server-side sampler
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
endpoint decodes greedy while looking configured. This is why frank-large's endpoint decodes greedy while looking configured. This is why frank-dragoon's
entry carries the recipe on the wire and a comment saying "do not drop it". entry carries the recipe on the wire and a comment saying "do not drop it".
**Reasoning-effort folding.** Qwen3.8's template accepts only **Reasoning-effort folding.** Qwen3.8's template accepts only
@@ -23,7 +23,7 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service **Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
a 100+ GiB payload — and frank-primary (dankserver:7770) is also the a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
harness's own model backend, so a blind deploy is a self-inflicted outage for harness's own model backend, so a blind deploy is a self-inflicted outage for
possibly zero behavioral change. possibly zero behavioral change.
@@ -62,8 +62,8 @@ service on the other host.
|---|---|---| |---|---|---|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. | | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. | | SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
| `frank-large` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. | | `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
| `frank-secondary` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. | | `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. | | `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. | | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
@@ -95,7 +95,7 @@ host-side `workflows/` dir and the repo copy.
## Recommended additions ## Recommended additions
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
frank-secondary's). All repos verified 2026-10-05. None are on disk yet. frank-zealot's). All repos verified 2026-10-05. None are on disk yet.
| # | What | Repo | Grab | Size | Why | | # | What | Repo | Grab | Size | Why |
|---|---|---|---|---|---| |---|---|---|---|---|---|
@@ -13,7 +13,7 @@ timestamp: 2026-08-26
# Dankclaw Stack # Dankclaw Stack
Dankclaw (192.168.0.143) runs the AI, search, development, and observability Dankclaw (192.168.0.143) runs the AI, search, development, and observability
stack. Its LLM inference endpoint `frank-large` (`:7770`) is a ROCmFPX stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`, the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
FLM v1.0.1) serves sight+sound models. The completion endpoint FLM v1.0.1) serves sight+sound models. The completion endpoint
@@ -30,7 +30,7 @@ graph LR
S --> V[Gluetun VPN] S --> V[Gluetun VPN]
O --> L[llama.cpp dankserver :7770] O --> L[llama.cpp dankserver :7770]
O --> H[Hermes] O --> H[Hermes]
O --> E[frank-large :7770] O --> E[frank-dragoon :7770]
``` ```
## Services ## Services
@@ -48,14 +48,14 @@ graph LR
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] | | Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — | | Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] | | Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
| frank-large (host service) | — | `7770` | [[deployment]] | | frank-dragoon (host service) | — | `7770` | [[deployment]] |
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] | | frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] | | frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
The LLM rows are bare-metal systemd services, not containers. This The LLM rows are bare-metal systemd services, not containers. This
repo owns their scripts, units, and knowledge base (see repo owns their scripts, units, and knowledge base (see
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03): [LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
`frank-large` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF `frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build; on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU; `frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving `frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
+3 -3
View File
@@ -47,7 +47,7 @@ is not durable. The tracked, sanitized configuration baseline is
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN. `0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
- The data volume persists skills, memory, and session history. - The data volume persists skills, memory, and session history.
- Hermes has two named local providers: `dankclaw-large` reaches - Hermes has two named local providers: `dankclaw-large` reaches
`http://192.168.0.143:7770/v1` and exposes `frank-large`; `http://192.168.0.143:7770/v1` and exposes `frank-dragoon`;
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes `dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
`frank-medium` (renamed from `frank-small` on 2026-09-15). `frank-medium` (renamed from `frank-small` on 2026-09-15).
- The live `config.yaml` contains secrets and user state and is not committed; - The live `config.yaml` contains secrets and user state and is not committed;
@@ -64,7 +64,7 @@ is not durable. The tracked, sanitized configuration baseline is
## Gotchas ## Gotchas
- Hermes is under active development; pin the image tag for reproducibility. - Hermes is under active development; pin the image tag for reproducibility.
- Model choice drives quality; the local `frank-large` (Dankclaw, 7770) is the intended backend. - Model choice drives quality; the local `frank-dragoon` (Dankclaw, 7770) is the intended backend.
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth. - Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
## Browser automation (Chrome DevTools Protocol daemon) ## Browser automation (Chrome DevTools Protocol daemon)
@@ -189,7 +189,7 @@ What actually breaks, and the fix:
auxiliary: auxiliary:
title_generation: title_generation:
provider: dankserver-small # names track the live config; diff first provider: dankserver-small # names track the live config; diff first
model: frank-secondary model: frank-zealot
api_mode: codex_responses # real per-task key api_mode: codex_responses # real per-task key
extra_body: extra_body:
reasoning: reasoning:
+13 -13
View File
@@ -24,7 +24,7 @@ https://github.com/open-webui/open-webui
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). | | Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. | | Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` | | Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
| Backends | Picker exposes exactly one model — `frank-secondary` (host.containers.internal:7771). The `frank-large` and `frank-primary` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) | | Backends | Picker exposes exactly one model — `frank-zealot` (host.containers.internal:7771). The `frank-dragoon` and `frank-archon` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) | | Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` | | Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` | | Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
@@ -34,15 +34,15 @@ https://github.com/open-webui/open-webui
The picker is built from the `model` table in `webui.db` (one row per model) The picker is built from the `model` table in `webui.db` (one row per model)
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
connection discovery — so the picker shows exactly what is seeded there. connection discovery — so the picker shows exactly what is seeded there.
As of 2026-10-07 the picker exposes **exactly one model: `frank-secondary`**. As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`**.
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host | | Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-large` | halogen flash-server (Strix Halo iGPU) | Dankclaw | | 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-primary` | radiance TP=2 (2× R9700) | Dankserver | | 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-secondary` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw | | 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
The `model` table carries exactly one decoration row, `frank-secondary` The `model` table carries exactly one decoration row, `frank-zealot`
(context length 200,704 = the per-slot window; description: Qwen3.5-9B (context length 200,704 = the per-slot window; description: Qwen3.5-9B
Q6_K + BF16 vision mmproj). Q6_K + BF16 vision mmproj).
@@ -62,7 +62,7 @@ Two gotchas make the wiring non-obvious:
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns - **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the `{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
the llama.cpp endpoints (frank-large, frank-secondary). The fix is to set the llama.cpp endpoints (frank-dragoon, frank-zealot). The fix is to set
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI `model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
builds the model list directly and never fetches `/v1/models`. frank-medium builds the model list directly and never fetches `/v1/models`. frank-medium
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids` (vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
@@ -116,7 +116,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
models. Response key is `items`, not `data`. models. Response key is `items`, not `data`.
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply 2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
with exactly one word: pong") → each returns content `pong`. Reasoning with exactly one word: pong") → each returns content `pong`. Reasoning
models need a generous budget: frank-secondary (Qwen3.5 template) returns models need a generous budget: frank-zealot (Qwen3.5 template) returns
empty content + `finish_reason: "length"` at `max_tokens 200` (the empty content + `finish_reason: "length"` at `max_tokens 200` (the
thinking block eats it) and answers `pong` at 1500. thinking block eats it) and answers `pong` at 1500.
3. A 500 "Server Connection Error" → `podman logs open-webui` for 3. A 500 "Server Connection Error" → `podman logs open-webui` for
@@ -191,8 +191,8 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
run with host `python3` over ssh against run with host `python3` over ssh against
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on `/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
Dankclaw; table is `user`, singular; no `active` column). No per-user model Dankclaw; table is `user`, singular; no `active` column). No per-user model
grant is needed: the picker model `frank-secondary` carries the global grant is needed: the picker model `frank-zealot` carries the global
`access_grant` row `(model, frank-secondary, user, *, read)`, so any `access_grant` row `(model, frank-zealot, user, *, read)`, so any
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does `role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
not exist until the account signs in (2026-10-09: five provisioned accounts not exist until the account signs in (2026-10-09: five provisioned accounts
had no row). had no row).
@@ -271,18 +271,18 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns. - **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-large`/`frank-medium`/`frank-secondary` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`. - **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-dragoon`/`frank-medium`/`frank-zealot` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`. - **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access). - **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
- **2026-10-07 — picker narrowed to `frank-secondary` only.** Per Frank: the only model accessible in OpenWebUI is `frank-secondary`. Rebuilt the `model` table to a single `frank-secondary` row (the old rows `frank-large`/`frank-primary`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-large) and 1 (frank-primary) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-secondary` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-secondary"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)). - **2026-10-07 — picker narrowed to `frank-zealot` only.** Per Frank: the only model accessible in OpenWebUI is `frank-zealot`. Rebuilt the `model` table to a single `frank-zealot` row (the old rows `frank-dragoon`/`frank-archon`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-dragoon) and 1 (frank-archon) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-zealot` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-zealot"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints - **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
id-keyed objects had to move in the same window as the alias flip: id-keyed objects had to move in the same window as the alias flip:
`model.id`+`name` (`frank-secondary` → `frank-zealot`, `params` `model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
`{"context":{"length":200704}}` and the `meta.description` kept), `{"context":{"length":200704}}` and the `meta.description` kept),
`access_grant` (`resource_type='model'`, `resource_id`, principal `access_grant` (`resource_type='model'`, `resource_id`, principal
`user`/`*`, permission `read`), and `openai.api_configs` per-backend `user`/`*`, permission `read`), and `openai.api_configs` per-backend
@@ -33,10 +33,10 @@ it is not invisible.
## Why 64 GiB ## Why 64 GiB
Dankclaw is shared. `frank-large` alone holds **103.5 GiB of the 121.2 GiB** Dankclaw is shared. `frank-dragoon` alone holds **103.5 GiB of the 121.2 GiB**
host (`68.0 weights + 14.4 KV pool + 21.1 working`), leaving roughly host (`68.0 weights + 14.4 KV pool + 21.1 working`), leaving roughly
**7.1 GiB** for 28 co-tenant containers whose working set is ~4.3 GiB. The **7.1 GiB** for 28 co-tenant containers whose working set is ~4.3 GiB. The
engine itself warns that when that runs out, frank-large and its neighbours engine itself warns that when that runs out, frank-dragoon and its neighbours
"can stall for minutes at a time at full CPU with no disk activity and no "can stall for minutes at a time at full CPU with no disk activity and no
output". Swap is the valve that turns an OOM kill into slow paging. output". Swap is the valve that turns an OOM kill into slow paging.
@@ -96,20 +96,20 @@ Then confirm the inference service is unaffected — a completion, not just
```bash ```bash
curl -s http://127.0.0.1:7770/v1/chat/completions \ curl -s http://127.0.0.1:7770/v1/chat/completions \
-H 'Content-Type: application/json' \ -H 'Content-Type: application/json' \
-d '{"model":"frank-large","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}' -d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}'
``` ```
As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up, As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up,
frank-large `health=200` throughout. frank-dragoon `health=200` throughout.
## Caveats ## Caveats
- **Swap cannot help the pinned weights.** frank-large's ~68 GiB of weights are - **Swap cannot help the pinned weights.** frank-dragoon's ~68 GiB of weights are
held deliberately; paging them is a stall, not a relief. More swap does not held deliberately; paging them is a stall, not a relief. More swap does not
make the doubled KV pool affordable — see the reserve/pool ceiling in make the doubled KV pool affordable — see the reserve/pool ceiling in
[[halogen-flash-server]]. [[halogen-flash-server]].
- **`vm.swappiness=60` is untested against this workload.** Under real - **`vm.swappiness=60` is untested against this workload.** Under real
pressure the kernel will happily page anonymous memory, and frank-large's KV pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
pool and working set *are* anonymous. Swapping the KV pool would be far worse pool and working set *are* anonymous. Swapping the KV pool would be far worse
than the OOM kill it prevents. If the box ever starts thrashing, lowering than the OOM kill it prevents. If the box ever starts thrashing, lowering
swappiness (or capping the engine's cgroup) is the knob, not more swap. swappiness (or capping the engine's cgroup) is the knob, not more swap.
@@ -22,8 +22,8 @@ The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the stat
**Docs page shipped (2026-09-17):** `/docs` serves a static user documentation **Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
page (`shared/homepage/docs.html`) — access walkthrough, the running-model page (`shared/homepage/docs.html`) — access walkthrough, the running-model
guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
remain a separate thread. The three LLM cards (`frank-large`, `frank-medium`, remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`,
`frank-secondary`) moved from admin-only to the **user** page with live model `frank-zealot`) moved from admin-only to the **user** page with live model
descriptions and a `docs` link to `/docs#models`. descriptions and a `docs` link to `/docs#models`.
**Sections + WIP split (2026-09-21):** both pages now group their panels **Sections + WIP split (2026-09-21):** both pages now group their panels
@@ -48,23 +48,23 @@ exact bug bit the screenshots on 2026-10-08). The docs page gained a full
(`/docs#api-access`): per-user OpenWebUI API keys (User menu → Settings → (`/docs#api-access`): per-user OpenWebUI API keys (User menu → Settings →
Account → API keys), the base URL `https://ai.franksdank.cloud/api/v1`, and Account → API keys), the base URL `https://ai.franksdank.cloud/api/v1`, and
copy-paste configs for **Opencode** (`~/.config/opencode/opencode.json`) and copy-paste configs for **Opencode** (`~/.config/opencode/opencode.json`) and
**Pi** (`~/.pi/agent/models.json`) pointing at `frank-secondary`. Shipped **Pi** (`~/.pi/agent/models.json`) pointing at `frank-zealot`. Shipped
together with the OpenWebUI side: `features.api_keys=true` for regular users together with the OpenWebUI side: `features.api_keys=true` for regular users
and the restored `frank-secondary` model row + public `read` access grant and the restored `frank-zealot` model row + public `read` access grant
without which the picker showed "No results found" for every family account without which the picker showed "No results found" for every family account
(see [[OpenWebUI]]). (see [[OpenWebUI]]).
**Capability equivalence notes (2026-10-08):** each fleet card (and the **Capability equivalence notes (2026-10-08):** each fleet card (and the
`/docs#models` section) carries a `details.equiv` line — frank-large `/docs#models` section) carries a `details.equiv` line — frank-dragoon
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier "equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
intelligence)", frank-primary "equivalent to Claude Opus 4.8 / OpenAI-Terra intelligence)", frank-archon "equivalent to Claude Opus 4.8 / OpenAI-Terra
(same era)", frank-secondary "equivalent to last year's frontier level (same era)", frank-zealot "equivalent to last year's frontier level
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic); (2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
pinned in `homepage_config.bats`. pinned in `homepage_config.bats`.
**Protoss rename + Model Information component (2026-10-08):** the model **Protoss rename + Model Information component (2026-10-08):** the model
cards became the StarCraft trio — `frank-large` → **frank-dragoon** cards became the StarCraft trio — `frank-dragoon` → **frank-dragoon**
(iGPU), `frank-primary` → **frank-archon** (Dankserver), `frank-secondary` (iGPU), `frank-archon` → **frank-archon** (Dankserver), `frank-zealot`
→ **frank-zealot** (3090). The endpoint/API ids did NOT change in this wave; only the → **frank-zealot** (3090). The endpoint/API ids did NOT change in this wave; only the
family-facing names. The three cards fused into one **Model Information** family-facing names. The three cards fused into one **Model Information**
component (`app.js` `renderModelInfo()`, `.model-info`/`.mi-*` styles): component (`app.js` `renderModelInfo()`, `.model-info`/`.mi-*` styles):
@@ -201,8 +201,8 @@ the pixels.
**Model ids official (2026-10-08):** the card ids `frank-zealot`, **Model ids official (2026-10-08):** the card ids `frank-zealot`,
`frank-dragoon`, `frank-archon` are now the harness's official model ids `frank-dragoon`, `frank-archon` are now the harness's official model ids
in `models.yml`/`config.yml` (see [[OMP Harness]]); the in `models.yml`/`config.yml` (see [[OMP Harness]]); the
systemd services keep the `frank-primary`/`frank-large`/ systemd services keep the `frank-archon`/`frank-dragoon`/
`frank-secondary` names. Homepage, docs page and harness now agree on `frank-zealot` names. Homepage, docs page and harness now agree on
one name per model. one name per model.
**Api ids renamed on the wire (2026-10-09):** the promise in **Api ids renamed on the wire (2026-10-09):** the promise in
@@ -316,7 +316,7 @@ flowchart TD
## Service classification ## Service classification
**User services** (root page), in section order: **User services** (root page), in section order:
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-large, frank-primary, frank-secondary — status + docs link, no Open link). - **AI** — OpenWebUI, and the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot — status + docs link, no Open link).
- **Web Apps** — SearXNG, Gitea, Copyparty, Marreta. - **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr. - **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
@@ -38,7 +38,7 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
| `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works | | `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
| `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` | | `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
| `frank-medium` | `192.168.0.235:7770` | the LLM fleet — `vllm:` series; a different machine, so the LAN address works | | `frank-medium` | `192.168.0.235:7770` | the LLM fleet — `vllm:` series; a different machine, so the LAN address works |
| `frank-large`, `frank-secondary` | `host.containers.internal:7770`, `:7771` | the LLM fleet — `llamacpp:` series; same-host net=host binds, so via the gateway | | `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet — `llamacpp:` series; same-host net=host binds, so via the gateway |
**Three jobs read down, and they are not outages** (audited 2026-09-16): **Three jobs read down, and they are not outages** (audited 2026-09-16):
@@ -115,7 +115,7 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
- Internal order is fixed (ngram types first, then drafters); CLI order is - Internal order is fixed (ngram types first, then drafters); CLI order is
irrelevant. irrelevant.
- **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod` - **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
on the qwen35 hybrid (frank-large era) measured −15 % prose / −7 % on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
block-copy (2026-10-03): adding the ngram layer logs `recurrent state block-copy (2026-10-03): adding the ngram layer logs `recurrent state
rollback is not compatible with 'ngram-mod' - disabling rollback support`, rollback is not compatible with 'ngram-mod' - disabling rollback support`,
and MTP verification depends on that rollback, so acceptance drops on ALL and MTP verification depends on that rollback, so acceptance drops on ALL
@@ -215,9 +215,9 @@ The DFlash draft is not a standalone model. It must pair with its target model.
## Deployment on Dankserver (NVIDIA RTX 3090) ## Deployment on Dankserver (NVIDIA RTX 3090)
The Dankserver llama.cpp deployment (`frank-secondary`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-secondary` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only. > **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions: The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
@@ -250,7 +250,7 @@ docker run --rm --name vllm-qwen36-dflash \
## Deployment on Dankclaw (AMD Strix Halo gfx1151) ## Deployment on Dankclaw (AMD Strix Halo gfx1151)
`frank-large` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]]. `frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only. History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
@@ -315,7 +315,7 @@ not switch without re-measuring".
The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type, The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
slot split, quant rung — not just spec flags) is the transient systemd slot split, quant rung — not just spec flags) is the transient systemd
drop-in procedure in [[frank-large-benchmark]] and [[frank-secondary-benchmark]]. drop-in procedure in [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]].
## Is it worth it? (arithmetic before deployment) ## Is it worth it? (arithmetic before deployment)
@@ -419,4 +419,4 @@ measurement that would settle it.
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in - [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic - [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands - [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
- [[frank-large-benchmark]] and [[frank-secondary-benchmark]] — the single-GPU env drop-in A/B vehicle - [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — the single-GPU env drop-in A/B vehicle
@@ -22,7 +22,7 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
(b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx (b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not 302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
hold on the 8060S. See [[frank-large-benchmark]]. hold on the 8060S. See [[frank-dragoon-benchmark]].
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
does not exist on this machine (checked 2026-08-15). GPU DPM is already does not exist on this machine (checked 2026-08-15). GPU DPM is already
`performance`, forced level `auto`. This item is N/A. `performance`, forced level `auto`. This item is N/A.
@@ -48,7 +48,7 @@ lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
different, driver-reported figure that has always been larger than the kernel different, driver-reported figure that has always been larger than the kernel
cap. cap.
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
limits documented in [[frank-large-benchmark]]. limits documented in [[frank-dragoon-benchmark]].
## 1. Self-Hosted Agentic Tools ## 1. Self-Hosted Agentic Tools
@@ -106,7 +106,7 @@ All require root and a reboot for the modprobe change.
## Sources ## Sources
- [[frank-large-benchmark]] — measured backend/spec numbers - [[frank-dragoon-benchmark]] — measured backend/spec numbers
- [[deployment]] — deploy and update process - [[deployment]] — deploy and update process
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS) - AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`, - llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
@@ -115,7 +115,7 @@ All require root and a reboot for the modprobe change.
## See Also ## See Also
- [[frank-large-benchmark]] - [[frank-dragoon-benchmark]]
- [[DFlash DSpark Speculative Decoding]] - [[DFlash DSpark Speculative Decoding]]
- [[deployment]] - [[deployment]]
- [[index]] - [[index]]
@@ -46,7 +46,7 @@ Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host run
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-09-11** with the `amd_iommu=off` decision) | | Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-09-11** with the `amd_iommu=off` decision) |
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] | | Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
The NPU is retired (2026-09-11). The `frank-secondary` name now serves Gemma4-12B QAT on the RTX 3090 via llama.cpp (see [[frank-secondary-gemma4-12b]]). The NPU is retired (2026-09-11). The `frank-zealot` name now serves Gemma4-12B QAT on the RTX 3090 via llama.cpp (see [[frank-zealot-gemma4-12b]]).
### ROCm ### ROCm
@@ -101,7 +101,7 @@ cable between claw and server is required for 10G.
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM. - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget. - The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active service is `frank-large` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-large-gufo-qwen38-27b]]. - The active service is `frank-dragoon` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-dragoon-gufo-qwen38-27b]].
- The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path. - The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18. - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 1.1 TB free disk space is ample for model storage. - 1.1 TB free disk space is ample for model storage.
@@ -17,7 +17,7 @@ timestamp: 2026-09-20
Dankserver is a media/library server and the `frank-medium` LLM host. The Dankserver is a media/library server and the `frank-medium` LLM host. The
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
FreeToken/CUDA `frank-secondary` era was removed on 2026-09-15 and replaced with FreeToken/CUDA `frank-zealot` era was removed on 2026-09-15 and replaced with
two AMD Radeon AI PRO R9700 cards. two AMD Radeon AI PRO R9700 cards.
## Hardware Specifications ## Hardware Specifications
@@ -107,8 +107,8 @@ hijack `libhsa-runtime64` were purged on 2026-09-15 — see
The active inference service is `frank-medium.service`: mainline llama-server on The active inference service is `frank-medium.service`: mainline llama-server on
port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to
ROCm1. See [[frank-medium-llamacpp-dflash2]]. The former ROCm1. See [[frank-medium-llamacpp-dflash2]]. The former
`frank-secondary.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap; `frank-zealot.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap;
its unit is archived in `~/.AGENTTRASH/20260915-frank-secondary-retire/host-units/`, its unit is archived in `~/.AGENTTRASH/20260915-frank-zealot-retire/host-units/`,
and the vllm-radiance replica trio in `~/.AGENTTRASH/20260916-vllm-radiance-retire/`. and the vllm-radiance replica trio in `~/.AGENTTRASH/20260916-vllm-radiance-retire/`.
See [[deployment]]. See [[deployment]].
@@ -13,12 +13,12 @@ timestamp: 2026-09-03
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology. > **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
> See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration. > See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
> The `frank-secondary` name and port 52625 then belonged to the vLLM > The `frank-zealot` name and port 52625 then belonged to the vLLM
> service on Dankclaw (nameistoken Quark W8A8); this page is archive. > service on Dankclaw (nameistoken Quark W8A8); this page is archive.
# FastFlowLM NPU Inference # FastFlowLM NPU Inference
FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-secondary` (port 52625) serves in production. See Promotion. FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-zealot` (port 52625) serves in production. See Promotion.
## Verdict (2026-08-16) ## Verdict (2026-08-16)
@@ -167,7 +167,7 @@ Proposed role: a small, always-on, low-power endpoint for triage, summarization,
- `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`. - `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
- `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below). - `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below).
- `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot. - `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
- The `llama-server` (frank-large, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16. - The `llama-server` (frank-dragoon, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU) ## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
@@ -185,7 +185,7 @@ Long-context prefill is not yet measured (runs 1-3 used under 200 prompt tokens)
## Model Bakeoff (measured 2026-08-16, 4 models) ## Model Bakeoff (measured 2026-08-16, 4 models)
All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-large) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom). All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-dragoon) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) | | Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------| |-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
@@ -203,21 +203,21 @@ Notes:
## Promotion (2026-08-16) ## Promotion (2026-08-16)
`frank-secondary` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-secondary.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-secondary.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch. `frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
### Boot policy (2026-08-17) ### Boot policy (2026-08-17)
The `frank-secondary` unit is disabled for boot. Only the PP pair (`frank-large` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw: The `frank-zealot` unit is disabled for boot. Only the PP pair (`frank-dragoon` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw:
``` ```
sudo ln -sf /home/dank/vllmScripts/scripts/frank-secondary.service /etc/systemd/system/frank-secondary.service sudo ln -sf /home/dank/vllmScripts/scripts/frank-zealot.service /etc/systemd/system/frank-zealot.service
sudo systemctl daemon-reload sudo systemctl daemon-reload
sudo systemctl enable frank-secondary sudo systemctl enable frank-zealot
``` ```
Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom). Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
The PP split was renamed `frank-pp-head` → `frank-large` (2026-08-16, commit `48142f5`). The PP split was renamed `frank-pp-head` → `frank-dragoon` (2026-08-16, commit `48142f5`).
Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it. Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it.
@@ -353,7 +353,7 @@ installed) against `/opt/fastflowlm/share/flm/xclbins/` (available).
Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window. Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`. 1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`.
2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-secondary` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-secondary' is not supported.` Each such request also triggered the unknown-tag fallback below. 2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-zealot` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-zealot' is not supported.` Each such request also triggered the unknown-tag fallback below.
3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load. 3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load.
Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16. Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16.
@@ -390,7 +390,7 @@ pi sessions failed against this endpoint with `Stream ended without finish_reaso
- **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step. - **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step.
- **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this. - **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this.
- **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags. - **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags.
- **NPU production role (2026-08-16).** `frank-secondary` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded. - **NPU production role (2026-08-16).** `frank-zealot` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded.
## Citations ## Citations
@@ -23,7 +23,7 @@ read the returned content:
```python ```python
import json, urllib.request import json, urllib.request
req = urllib.request.Request("http://<host>:<port>/v1/chat/completions", req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
json.dumps({"model": "frank-secondary", "messages": [{"role": "user", json.dumps({"model": "frank-zealot", "messages": [{"role": "user",
"content": "Hi"}]}).encode(), {"Content-Type": "application/json"}) "content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req, timeout=600))) print(json.load(urllib.request.urlopen(req, timeout=600)))
``` ```
@@ -119,5 +119,5 @@ re-download.
- [[Llama Cpp Runtimes]] — which fork ships which custom types - [[Llama Cpp Runtimes]] — which fork ships which custom types
- [[Llama Cpp Code Diff]] — the enum tables and code-level differences - [[Llama Cpp Code Diff]] — the enum tables and code-level differences
- [[frank-large-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production - [[frank-dragoon-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production
- [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads
@@ -13,7 +13,7 @@ Fleet law first: **STOP the production service before any GPU experiment**
GPU and corrupts results; an unisolated GPU experiment has precedent for GPU and corrupts results; an unisolated GPU experiment has precedent for
wedging a host until power cycle. Reference harnesses: wedging a host until power cycle. Reference harnesses:
`scripts/bench-batch-ubatch.sh`, `scripts/autoresearch.sh`, `scripts/bench-batch-ubatch.sh`, `scripts/autoresearch.sh`,
`scripts/autoresearch/frank-primary-arms.sh`. `scripts/autoresearch/frank-archon-arms.sh`.
## 1. The window structure (one config per arm, a driver owns the window) ## 1. The window structure (one config per arm, a driver owns the window)
@@ -27,7 +27,7 @@ wedging a host until power cycle. Reference harnesses:
with `container state improper`. with `container state improper`.
2. **Boot the arm as the service, not a scratch port** (no scratch ports on 2. **Boot the arm as the service, not a scratch port** (no scratch ports on
LLM hosts — one card, one server). The serve script takes its config via LLM hosts — one card, one server). The serve script takes its config via
env-file override (`FRANK_SECONDARY_ENV_FILE`, `FRANK_PRIMARY_AR_ENV_FILE`) env-file override (`FRANK_ZEALOT_ENV_FILE`, `FRANK_ARCHON_AR_ENV_FILE`)
pointing at a staged copy; the checkout is never mutated. pointing at a staged copy; the checkout is never mutated.
3. **Fast-fail the health wait**: poll `podman ps | grep -qx <name>` inside 3. **Fast-fail the health wait**: poll `podman ps | grep -qx <name>` inside
the loop — a refused boot dies in one poll instead of 20 min. Capture the the loop — a refused boot dies in one poll instead of 20 min. Capture the
@@ -245,17 +245,17 @@ nearly shipped the wrong setting.
the comment); verify multi-line command construction with `bash -n` AND a the comment); verify multi-line command construction with `bash -n` AND a
render test. render test.
## 8. Results location and recovery (frank-primary arms) ## 8. Results location and recovery (frank-archon arms)
| copy | path | | copy | path |
|---|---| |---|---|
| per-arm evidence | `$ARMS_DIR` = `~/.jobs/<run-stamp>/` (`.rc`, `.done` beside it) | | per-arm evidence | `$ARMS_DIR` = `~/.jobs/<run-stamp>/` (`.rc`, `.done` beside it) |
| summary | `$ARMS_DIR/summary.tsv` = `arm rc decode_tps decode_tps_t0.7 kv_tokens boot_s` + `<arm>.out/.env/.overlay/.rc/.metrics` | | summary | `$ARMS_DIR/summary.tsv` = `arm rc decode_tps decode_tps_t0.7 kv_tokens boot_s` + `<arm>.out/.env/.overlay/.rc/.metrics` |
| server-side last window | `/home/dank/.autoresearch-frank-primary/run.json` | | server-side last window | `/home/dank/.autoresearch-frank-archon/run.json` |
| durable memo | `.../last/run.json` + `last/hash` (promoted only after bench AND verified restore) | | durable memo | `.../last/run.json` + `last/hash` (promoted only after bench AND verified restore) |
| fallback | `~/.omp/agent/sessions/<ws>/<session>/<N>.{shake,bash}.log` (permanent) | | fallback | `~/.omp/agent/sessions/<ws>/<session>/<N>.{shake,bash}.log` (permanent) |
`/tmp/frank-primary-arms-*` and `/tmp/armstub/` are stub self-tests — `/tmp/frank-archon-arms-*` and `/tmp/armstub/` are stub self-tests —
constants 99.9/88.8 identical across arms by construction; **identical constants 99.9/88.8 identical across arms by construction; **identical
numbers across arms == stub or bug, never a measurement.** After compaction, numbers across arms == stub or bug, never a measurement.** After compaction,
re-harvest with `grep -rhoE 'decode_tps[a-z0-9_.]*=[0-9.]+'` over the session re-harvest with `grep -rhoE 'decode_tps[a-z0-9_.]*=[0-9.]+'` over the session
@@ -161,8 +161,8 @@ of the same RAM, not extra memory.
## 7. IBS model-access attribution (AMD CPUs) ## 7. IBS model-access attribution (AMD CPUs)
Real CPU access patterns for mmap'd weights, live, no stop (established on Real CPU access patterns for mmap'd weights, live, no stop (established on
frank-primary, 2026-09-26, ~2.7M attributed samples in 75 s). Reference: frank-archon, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
`scripts/frank-primary-expert-activity.py` (self-contained dump + sample + `scripts/frank-archon-expert-activity.py` (self-contained dump + sample +
analyze; `--analyze-only`). analyze; `--analyze-only`).
1. GGUF header-only scan (ms for a 50 GiB part) → tensor extents. **Extents 1. GGUF header-only scan (ms for a 50 GiB part) → tensor extents. **Extents
@@ -80,7 +80,7 @@ co-tenants.
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned — - **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
"move layers to the card" converts reclaimable cache into device memory and "move layers to the card" converts reclaimable cache into device memory and
frees nothing for co-tenants. A served model's footprint ≠ weight size frees nothing for co-tenants. A served model's footprint ≠ weight size
(frank-secondary: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity (frank-zealot: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV = subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k. 28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
- **Context lengths: derive, never remember.** `llama-bench` computes - **Context lengths: derive, never remember.** `llama-bench` computes
@@ -157,7 +157,7 @@ in the timings. Context cap for full-model residency on that shape stayed
## Related ## Related
- [[GGUF Compatibility]] — what a payload is before it is priced - [[GGUF Compatibility]] — what a payload is before it is priced
- [[frank-secondary-llama-mimo-q6k-2x200704]] — the current two-boot priced shape - [[frank-zealot-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured - [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp - [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
@@ -8,8 +8,8 @@ timestamp: 2026-10-05T00:00:00-07:00
# Model Swap and Rollback # Model Swap and Rollback
Applies to every endpoint swap on the fleet (frank-secondary / frank-large / Applies to every endpoint swap on the fleet (frank-zealot / frank-dragoon /
frank-primary). Repo law: all config flows through git; the live models are frank-archon). Repo law: all config flows through git; the live models are
host state under `/home/dank/models/<name>/`. host state under `/home/dank/models/<name>/`.
## 1. Fit preflight — never deploy a swap without confirming it FITS ## 1. Fit preflight — never deploy a swap without confirming it FITS
@@ -26,7 +26,7 @@ conclusions.
| Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` | | Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` |
| FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` | | FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` |
| KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed | | KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed |
| ubatch | 256 | `UBATCH_SIZE` in `frank-large.env` | | ubatch | 256 | `UBATCH_SIZE` in `frank-dragoon.env` |
| MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` | | MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` |
| Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s | | Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s |
| Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends | | Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends |
@@ -20,7 +20,7 @@ morning-of runbook. Research background:
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68) ## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
This split is the production `frank-large` service. Current This split is the production `frank-dragoon` service. Current
state, verified against the live journal and output smoke tests state, verified against the live journal and output smoke tests
(2026-08-17, load 02:06): (2026-08-17, load 02:06):
@@ -41,17 +41,17 @@ state, verified against the live journal and output smoke tests
Vulkan). Vulkan).
- Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but - Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
stable; all allocations are upfront). stable; all allocations are upfront).
- Unit `frank-large.service`, launched through - Unit `frank-dragoon.service`, launched through
`serve-pp-head.sh` + `frank-large.env`. `serve-pp-head.sh` + `frank-dragoon.env`.
## Service enablement (2026-08-17) ## Service enablement (2026-08-17)
The PP pair is the only inference service enabled for boot. Both units The PP pair is the only inference service enabled for boot. Both units
(`frank-large` on Dankclaw, `frank-pp-worker` on Dankserver) are (`frank-dragoon` on Dankclaw, `frank-pp-worker` on Dankserver) are
enabled. The NPU unit (`frank-secondary` on Dankclaw) and all retired enabled. The NPU unit (`frank-zealot` on Dankclaw) and all retired
units are disabled. Restart order: worker first, then head. The Dankserver units are disabled. Restart order: worker first, then head. The Dankserver
helper `frank-restart-small.sh` is obsolete (it restarts the removed helper `frank-restart-small.sh` is obsolete (it restarts the removed
`frank-secondary` unit); restart the worker with `frank-zealot` unit); restart the worker with
`sudo systemctl restart frank-pp-worker`. The Dankclaw helper `sudo systemctl restart frank-pp-worker`. The Dankclaw helper
`frank-restart-large.sh` (sync + cache drop + restart) is correct for the `frank-restart-large.sh` (sync + cache drop + restart) is correct for the
head. `deploy.sh dankserver` is broken until retargeted. See head. `deploy.sh dankserver` is broken until retargeted. See
@@ -173,9 +173,9 @@ llama-server b10433 ggml-rpc-server b10433
|------|---------|---------| |------|---------|---------|
| `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) | | `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
| `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build | | `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
| `scripts/frank-large.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename | | `scripts/frank-dragoon.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
| `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") | | `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
| `scripts/frank-large.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` | | `scripts/frank-dragoon.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
| `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) | | `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
| `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` | | `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
@@ -229,7 +229,7 @@ FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh stop-pp
FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
``` ```
Config variants: edit `scripts/frank-large.env` on the claw (git commit + Config variants: edit `scripts/frank-dragoon.env` on the claw (git commit +
pull if the bench runs from the repo), then re-run `start` + `bench`: pull if the bench runs from the repo), then re-run `start` + `bench`:
- No spec baseline: `SPEC_NONE=1` - No spec baseline: `SPEC_NONE=1`
@@ -65,6 +65,6 @@ graph LR
## See Also ## See Also
- [[DFlash DSpark Speculative Decoding]] - [[DFlash DSpark Speculative Decoding]]
- [[frank-secondary-benchmark]] - [[frank-zealot-benchmark]]
- [[Dankserver System Specs]] - [[Dankserver System Specs]]
- [[Dankclaw System Specs]] - [[Dankclaw System Specs]]
@@ -78,12 +78,12 @@ Laws that steer everything here:
variants **[verified (tool existence: heretic; rest survey)]**. variants **[verified (tool existence: heretic; rest survey)]**.
- Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm - Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
weights, so it composes with any expert quantization. weights, so it composes with any expert quantization.
- Fleet status: frank-large runs an abliterated base **by Frank's choice** - Fleet status: frank-dragoon runs an abliterated base **by Frank's choice**
(AGENTS: censored-base refusals were the accepted trade of the 27B era); (AGENTS: censored-base refusals were the accepted trade of the 27B era);
frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8. frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.
### HauhauCS QX_K_P "Perfect" quants ### HauhauCS QX_K_P "Perfect" quants
- Our frank-secondary publisher. Method, from the publisher's own cards - Our frank-zealot publisher. Method, from the publisher's own cards
**[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"), **[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
(2) quantize with **imatrix calibration on the ABLITERATED weights**, (2) quantize with **imatrix calibration on the ABLITERATED weights**,
(3) K_P = a **per-model, per-tensor mixed-precision profile** — (3) K_P = a **per-model, per-tensor mixed-precision profile** —
@@ -106,7 +106,7 @@ Laws that steer everything here:
## Per-card playbooks ## Per-card playbooks
### RTX 3090 — the unlocked card (frank-secondary runs llama.cpp BY CHOICE) ### RTX 3090 — the unlocked card (frank-zealot runs llama.cpp BY CHOICE)
The 3090 is the only fleet device with no format lock: radiance needs The 3090 is the only fleet device with no format lock: radiance needs
`.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other `.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other
engine. sm_86 + CUDA is the industry's default target, so the runtime engine. sm_86 + CUDA is the industry's default target, so the runtime
@@ -179,7 +179,7 @@ Q8_K_P 40.6 GiB + mmproj 0.84) — Q4_K_P IS the protected-tensor blend.
The self-made delta: calibration on OUR corpora, our hot-expert law, The self-made delta: calibration on OUR corpora, our hot-expert law,
and a KLD gate scored at the serving shape. and a KLD gate scored at the serving shape.
### R9700 — frank-primary (radiance) ### R9700 — frank-archon (radiance)
The full self-quant pipeline already exists end to end: fetch BF16 → The full self-quant pipeline already exists end to end: fetch BF16 →
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap, record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,
`RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) → `RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) →
@@ -219,7 +219,7 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
until proven), and does its MTP cast to the fp8/u2 encodings like until proven), and does its MTP cast to the fp8/u2 encodings like
qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model. qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.
### Strix Halo — frank-large (halogen) and friends ### Strix Halo — frank-dragoon (halogen) and friends
- `.hgn`: **closed production door.** No public quantizer exists - `.hgn`: **closed production door.** No public quantizer exists
(verified absence across peonist-ai repos); the format's lock is the (verified absence across peonist-ai repos); the format's lock is the
engine's lock. We consume vendor artifacts; the repo-side tooling is engine's lock. We consume vendor artifacts; the repo-side tooling is
@@ -255,6 +255,6 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
`scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`, `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
`scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]], `scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
[[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]], [[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
[[frank-primary-radiance-swift15-w4dn8]], [[frank-archon-radiance-swift15-w4dn8]],
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]], [[frank-zealot-llama-qwen35-9b-q6k-2x200704]],
[[frank-large-flash-server-v2-swift15]], [[GGUF Compatibility]]. [[frank-dragoon-flash-server-v2-swift15]], [[GGUF Compatibility]].
@@ -247,4 +247,4 @@ see [[dankserver-qwen38-fp8-expert-offload-tuning]] for why it was rejected);
measured campaigns measured campaigns
- [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it - [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
- [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive - [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
- [[frank-large-benchmark]] and [[frank-secondary-benchmark]] — window discipline on live hosts - [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — window discipline on live hosts

Some files were not shown because too many files have changed in this diff Show More