llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot
The 2026-10-09 wire aliases become the service names everywhere: systemd units, serve scripts, env files and FRANK_* variables, deploy-plan/deploy-remote service keys, container names, Prometheus job names + alert rules, Grafana dashboard queries, bats seams, runbook filenames, homepage/AGENTS lore, and historical prose. Rollback tags renamed in lockstep. Residue in the tree: zero. Retired frank-medium/small/npu names are history and stay. Known casualty: scripts/patches/*.patch context lines were rewritten; those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
@@ -1,4 +1,4 @@
|
|||||||
# Tunables for frank-large (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
|
# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
|
||||||
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
|
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
|
||||||
# Strix Halo gfx1151, unified memory).
|
# Strix Halo gfx1151, unified memory).
|
||||||
#
|
#
|
||||||
@@ -56,7 +56,7 @@ CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
|||||||
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
||||||
|
|
||||||
# Alias + network.
|
# Alias + network.
|
||||||
ALIAS=frank-large
|
ALIAS=frank-dragoon
|
||||||
BIND_ADDR=0.0.0.0
|
BIND_ADDR=0.0.0.0
|
||||||
LISTEN_PORT=7770
|
LISTEN_PORT=7770
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-large)
|
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -7,8 +7,8 @@ Wants=network-online.target
|
|||||||
Type=simple
|
Type=simple
|
||||||
User=dank
|
User=dank
|
||||||
Group=dank
|
Group=dank
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
@@ -6,7 +6,7 @@
|
|||||||
# is reachable on this port regardless of the boot tag. The NPU is a
|
# is reachable on this port regardless of the boot tag. The NPU is a
|
||||||
# single-lane device: generations serialize FIFO, and the first request after a
|
# single-lane device: generations serialize FIFO, and the first request after a
|
||||||
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
||||||
# pool shared with frank-large (iGPU).
|
# pool shared with frank-dragoon (iGPU).
|
||||||
FLM_MODEL=nuextract3:4b
|
FLM_MODEL=nuextract3:4b
|
||||||
FLM_HOST=0.0.0.0
|
FLM_HOST=0.0.0.0
|
||||||
FLM_PORT=7771
|
FLM_PORT=7771
|
||||||
|
|||||||
+3
-3
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
|
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
|
||||||
# frank-large via the kingjones30/ROCmFPX llama.cpp runtime on
|
# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
|
||||||
# Dankclaw (Strix Halo gfx1151, unified memory).
|
# Dankclaw (Strix Halo gfx1151, unified memory).
|
||||||
#
|
#
|
||||||
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -44,7 +44,7 @@ fi
|
|||||||
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
|
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
|
||||||
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
||||||
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
||||||
: "${ALIAS:=frank-large}"
|
: "${ALIAS:=frank-dragoon}"
|
||||||
: "${BIND_ADDR:=0.0.0.0}"
|
: "${BIND_ADDR:=0.0.0.0}"
|
||||||
: "${LISTEN_PORT:=7770}"
|
: "${LISTEN_PORT:=7770}"
|
||||||
: "${CTX_SIZE:=401408}"
|
: "${CTX_SIZE:=401408}"
|
||||||
@@ -4,7 +4,7 @@
|
|||||||
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
|
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
|
||||||
# single-lane device: generations serialize in FIFO order, so this is a
|
# single-lane device: generations serialize in FIFO order, so this is a
|
||||||
# background/triage endpoint, not an interactive multi-user chat. The NPU
|
# background/triage endpoint, not an interactive multi-user chat. The NPU
|
||||||
# draws from the unified RAM pool shared with frank-large (the iGPU
|
# draws from the unified RAM pool shared with frank-dragoon (the iGPU
|
||||||
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
|
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
|
||||||
# for the memory budget and the model-selection tradeoff.
|
# for the memory budget and the model-selection tradeoff.
|
||||||
#
|
#
|
||||||
|
|||||||
@@ -9,11 +9,11 @@ services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
|
|||||||
merge absorbed the former `vllmScripts` repo, history preserved).
|
merge absorbed the former `vllmScripts` repo, history preserved).
|
||||||
|
|
||||||
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
|
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
|
||||||
Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
|
Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
|
||||||
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
||||||
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
||||||
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (flash-server
|
Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
|
||||||
container on the gfx1151 iGPU) and `frank-secondary` (llama.cpp CUDA on the
|
container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
|
||||||
RTX 3090). Caddy on
|
RTX 3090). Caddy on
|
||||||
Dankclaw reverse-proxies both hosts.
|
Dankclaw reverse-proxies both hosts.
|
||||||
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
||||||
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
|
|
||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
||||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||||
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
the [runbook index](#runbook-index). The full swap history lives in
|
the [runbook index](#runbook-index). The full swap history lives in
|
||||||
@@ -66,14 +66,14 @@ All edits flow through git. Never edit files on a remote machine directly.
|
|||||||
./scripts/deploy.sh dankclaw
|
./scripts/deploy.sh dankclaw
|
||||||
./scripts/deploy.sh dankserver
|
./scripts/deploy.sh dankserver
|
||||||
./scripts/deploy.sh dankclaw --service searxng
|
./scripts/deploy.sh dankclaw --service searxng
|
||||||
./scripts/deploy.sh dankserver --service frank-primary
|
./scripts/deploy.sh dankserver --service frank-archon
|
||||||
./scripts/deploy.sh dankclaw --dry-run
|
./scripts/deploy.sh dankclaw --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
The planner maps changed paths to services; the remote runner restarts only
|
The planner maps changed paths to services; the remote runner restarts only
|
||||||
the selected service and records its revision outside the checkout.
|
the selected service and records its revision outside the checkout.
|
||||||
Documentation-only changes are a no-op. LLM services are selected only by
|
Documentation-only changes are a no-op. LLM services are selected only by
|
||||||
their own files — every endpoint (`frank-large`/`frank-secondary`/`frank-primary`)
|
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
||||||
by its unit/env/serve trio in `scripts/` — so
|
by its unit/env/serve trio in `scripts/` — so
|
||||||
unrelated
|
unrelated
|
||||||
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
|
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
|
||||||
@@ -85,7 +85,7 @@ restarts each service whose marker is behind HEAD *and* whose paths changed
|
|||||||
since that marker — so a backlog accumulated by service-scoped deploys fires
|
since that marker — so a backlog accumulated by service-scoped deploys fires
|
||||||
all at once, and one failing health gate aborts the run with every later
|
all at once, and one failing health gate aborts the run with every later
|
||||||
service left pending. Name the service unless a drain is what you want
|
service left pending. Name the service unless a drain is what you want
|
||||||
(`./scripts/deploy.sh dankserver --service frank-primary`).
|
(`./scripts/deploy.sh dankserver --service frank-archon`).
|
||||||
|
|
||||||
### Deployment completeness (MANDATORY)
|
### Deployment completeness (MANDATORY)
|
||||||
|
|
||||||
@@ -229,17 +229,17 @@ archive.
|
|||||||
**STOP the production service BEFORE benchmarking** — these servers serve
|
**STOP the production service BEFORE benchmarking** — these servers serve
|
||||||
real users, and a bench beside a live service starves the GPU and corrupts
|
real users, and a bench beside a live service starves the GPU and corrupts
|
||||||
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
||||||
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-secondary were measured. Rolling your own bench
|
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-dragoon and frank-zealot were measured. Rolling your own bench
|
||||||
requires asking first.
|
requires asking first.
|
||||||
Scratch ports are not used on LLM hosts — a
|
Scratch ports are not used on LLM hosts — a
|
||||||
test process cannot share the single GPU anyway; validate against the service
|
test process cannot share the single GPU anyway; validate against the service
|
||||||
itself.
|
itself.
|
||||||
|
|
||||||
(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
|
(A gufo-era law; frank-dragoon ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
|
||||||
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
|
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
|
||||||
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
|
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
|
||||||
settings, and will invert the ranking.**
|
settings, and will invert the ranking.**
|
||||||
Measured 2026-09-28 on frank-large at depth 32768: its corpus
|
Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus
|
||||||
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
|
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
|
||||||
draft widths, i.e. the same highly-predictable text every run, and 93 % of
|
draft widths, i.e. the same highly-predictable text every run, and 93 % of
|
||||||
every proposal accepted. It reported 29–37 t/s decode where real traffic
|
every proposal accepted. It reported 29–37 t/s decode where real traffic
|
||||||
@@ -264,12 +264,12 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
|
|||||||
merely wedged kernel usually still answers ARP.
|
merely wedged kernel usually still answers ARP.
|
||||||
|
|
||||||
1. Stop and mask (self-healing would restart it):
|
1. Stop and mask (self-healing would restart it):
|
||||||
- Dankclaw `frank-large` (root unit, halogen podman on the iGPU):
|
- Dankclaw `frank-dragoon` (root unit, halogen podman on the iGPU):
|
||||||
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
|
`sudo systemctl stop frank-dragoon && sudo systemctl mask frank-dragoon`
|
||||||
- Dankclaw `frank-secondary` (root unit, llama.cpp on the 3090):
|
- Dankclaw `frank-zealot` (root unit, llama.cpp on the 3090):
|
||||||
`sudo systemctl stop frank-secondary && sudo systemctl mask frank-secondary`
|
`sudo systemctl stop frank-zealot && sudo systemctl mask frank-zealot`
|
||||||
- Dankserver `frank-primary` (root unit, radiance on both R9700s, TP=2):
|
- Dankserver `frank-archon` (root unit, radiance on both R9700s, TP=2):
|
||||||
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
|
`sudo systemctl stop frank-archon && sudo systemctl mask frank-archon`
|
||||||
2. Bench.
|
2. Bench.
|
||||||
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
|
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
|
||||||
`/etc/systemd/system/` is itself a symlink into the checkout — mask
|
`/etc/systemd/system/` is itself a symlink into the checkout — mask
|
||||||
@@ -287,14 +287,15 @@ The harness reaches the fleet through `.omp/agent/models.yml` in the
|
|||||||
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
||||||
where the harness runs; there is no copy on the servers. Each provider is one
|
where the harness runs; there is no copy on the servers. Each provider is one
|
||||||
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
||||||
the StarCraft unit names (2026-10-08) and since 2026-10-09 they are also the
|
the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
|
||||||
ids on the wire:** `frank-archon` → `192.168.0.235:7770`
|
since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
|
||||||
(`FRANK_PRIMARY_SERVED_NAME`), `frank-dragoon` → `192.168.0.143:7770`
|
`192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
|
||||||
(`HALOGEN_MODEL_ID`), `frank-zealot` → `192.168.0.143:7771` (`ALIAS`). The
|
`192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
|
||||||
SERVICES, units, scripts, env files and runbooks keep the old names
|
`192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
|
||||||
(`frank-primary`, `frank-large`, `frank-secondary`); only the served alias,
|
scripts, env files, containers, deploy services and runbooks to match —
|
||||||
the `models.yml`/`config.yml` ids, the homepage cards, the OpenWebUI grants
|
the three former service names are fully retired and exist nowhere in
|
||||||
and the Hermes routing were renamed. Verify a rename on `/v1/models`, never
|
the repo; historical text in runbooks uses the current names. Verify a
|
||||||
|
rename on `/v1/models`, never
|
||||||
on a chat response: radiance echoes the **requested** model name, so a 200
|
on a chat response: radiance echoes the **requested** model name, so a 200
|
||||||
proves nothing about the served id (llama.cpp and halogen ignore it).
|
proves nothing about the served id (llama.cpp and halogen ignore it).
|
||||||
|
|
||||||
@@ -309,16 +310,16 @@ provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
|
|||||||
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
|
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
|
||||||
a role pointing at a provider the file does not define fails at call time. As
|
a role pointing at a provider the file does not define fails at call time. As
|
||||||
of 2026-10-06 **all three endpoints accept images** (`input: [text,
|
of 2026-10-06 **all three endpoints accept images** (`input: [text,
|
||||||
image]`) — `frank-large` returned to vision with the flash-server v2 swap
|
image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
|
||||||
(the v2 vision tower loads beside the checkpoint; the text-only halogen
|
(the v2 vision tower loads beside the checkpoint; the text-only halogen
|
||||||
27B window closed 2026-10-06) — while
|
27B window closed 2026-10-06) — while
|
||||||
`frank-primary` takes them natively (radiance runs the checkpoint's
|
`frank-archon` takes them natively (radiance runs the checkpoint's
|
||||||
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
||||||
are the engine core; `--mm-max-patches` available),
|
are the engine core; `--mm-max-patches` available),
|
||||||
and `frank-secondary` through
|
and `frank-zealot` through
|
||||||
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
|
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
|
||||||
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
|
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
|
||||||
Gemma4 era used). `frank-secondary`'s `contextWindow` is
|
Gemma4 era used). `frank-zealot`'s `contextWindow` is
|
||||||
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
|
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
|
||||||
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
|
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
|
||||||
the harness must compact before llama.cpp answers a longer prompt with a 400.
|
the harness must compact before llama.cpp answers a longer prompt with a 400.
|
||||||
@@ -333,20 +334,20 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
|||||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||||
|
|
||||||
### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
||||||
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
||||||
driven by `scripts/serve-frank-large.sh` under a **root systemd unit**
|
driven by `scripts/serve-frank-dragoon.sh` under a **root systemd unit**
|
||||||
(`scripts/frank-large.service`, `User=dank`, `Restart=always`,
|
(`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
|
||||||
`ExecStop=podman stop -t 60 frank-large-halogen`, the script runs `podman
|
`ExecStop=podman stop -t 60 frank-dragoon-halogen`, the script runs `podman
|
||||||
run` FOREGROUND — the container's exit code is the service's exit so
|
run` FOREGROUND — the container's exit code is the service's exit so
|
||||||
`Restart=always` stays real, and the engine ledger lands on disk) on port
|
`Restart=always` stays real, and the engine ledger lands on disk) on port
|
||||||
7770 from the Strix Halo iGPU. Engine stdout goes to
|
7770 from the Strix Halo iGPU. Engine stdout goes to
|
||||||
`/home/dank/logs/frank_large.log` — **the boot ledger (weights resident, KV
|
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
|
||||||
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
||||||
Cutover 2026-10-06 from the halogen 27B-dense era
|
Cutover 2026-10-06 from the halogen 27B-dense era
|
||||||
([[frank-large-halogen27b-dflash2]], retired the same window).
|
([[frank-dragoon-halogen27b-dflash2]], retired the same window).
|
||||||
|
|
||||||
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
||||||
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
||||||
@@ -368,10 +369,10 @@ sidecar is deliberately not fetched).
|
|||||||
**Product facts vs the 27B era, both regressions FIXED:** the base is
|
**Product facts vs the 27B era, both regressions FIXED:** the base is
|
||||||
**abliterated** (the 27B was a censored-base finetune; refusals were an
|
**abliterated** (the 27B was a censored-base finetune; refusals were an
|
||||||
accepted trade there) and **vision is ON** (the 27B engine silently
|
accepted trade there) and **vision is ON** (the 27B engine silently
|
||||||
ignored its encoder — frank-secondary owned images for exactly one day at this
|
ignored its encoder — frank-zealot owned images for exactly one day at this
|
||||||
endpoint).
|
endpoint).
|
||||||
|
|
||||||
**Engine laws that shape the config** (all in `scripts/frank-large.env`
|
**Engine laws that shape the config** (all in `scripts/frank-dragoon.env`
|
||||||
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
||||||
policy disables the MTP head for whole requests after low acceptance and
|
policy disables the MTP head for whole requests after low acceptance and
|
||||||
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
||||||
@@ -413,7 +414,7 @@ page cache beside the checkpoint — the card's quickstart arithmetic does
|
|||||||
not bill it as resident, but READ the boot ledger and re-size if the
|
not bill it as resident, but READ the boot ledger and re-size if the
|
||||||
engine prints a clamp.
|
engine prints a clamp.
|
||||||
|
|
||||||
Fetch: `scripts/download-frank-large-flashnext.sh` into
|
Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
|
||||||
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
||||||
sha256 + `--verify-only`, no token on the command line; resume goes
|
sha256 + `--verify-only`, no token on the command line; resume goes
|
||||||
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
||||||
@@ -422,18 +423,18 @@ n-gram + vision tower, flat tokenizer with all six files as regular
|
|||||||
files) — the 2026-10-01 crash-loop lesson.
|
files) — the 2026-10-01 crash-loop lesson.
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md`.
|
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
|
||||||
Rollback: revert the swap commit and redeploy — the 27B payload
|
Rollback: revert the swap commit and redeploy — the 27B payload
|
||||||
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
||||||
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
||||||
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||||
|
|
||||||
### frank-secondary — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
|
### frank-zealot — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
|
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
|
||||||
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
||||||
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
||||||
(`scripts/frank-secondary.service`, `relink_unit`ed into
|
(`scripts/frank-zealot.service`, `relink_unit`ed into
|
||||||
`/etc/systemd/system/`), serving
|
`/etc/systemd/system/`), serving
|
||||||
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
||||||
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
||||||
@@ -489,7 +490,7 @@ tokens). With `--parallel 2` a compaction request no longer serializes behind
|
|||||||
a live generation (the Gemma4 era's `--parallel 1` law).
|
a live generation (the Gemma4 era's `--parallel 1` law).
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md`.
|
`serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md`.
|
||||||
Rollback: revert the swap commit — the Gemma4 payload
|
Rollback: revert the swap commit — the Gemma4 payload
|
||||||
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
||||||
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
||||||
@@ -497,7 +498,7 @@ Rollback: revert the swap commit — the Gemma4 payload
|
|||||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||||
|
|
||||||
|
|
||||||
### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
||||||
|
|
||||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||||
@@ -556,7 +557,7 @@ pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
|||||||
the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against
|
the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against
|
||||||
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
||||||
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
||||||
script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to
|
script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=140000`) refuses to
|
||||||
start until both knobs are raised in the stop window (sysfs write first,
|
start until both knobs are raised in the stop window (sysfs write first,
|
||||||
kernel cmdline + reboot only if read-only).
|
kernel cmdline + reboot only if read-only).
|
||||||
|
|
||||||
@@ -570,11 +571,11 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
|||||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||||
|
|
||||||
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`.
|
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
||||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
|
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
||||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
|
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
||||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
|
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
||||||
|
|
||||||
## Key Lessons Learned
|
## Key Lessons Learned
|
||||||
Durable operational lessons from the fleet's history. Full detail in the
|
Durable operational lessons from the fleet's history. Full detail in the
|
||||||
@@ -736,11 +737,11 @@ linked runbooks.
|
|||||||
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
||||||
of the old one, or live pages must land in RAM that `MemAvailable`
|
of the old one, or live pages must land in RAM that `MemAvailable`
|
||||||
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
||||||
the co-tenants, not frank-large — pinned weights don't swap, and paging
|
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
|
||||||
the KV pool is worse than the OOM it prevents.
|
the KV pool is worse than the OOM it prevents.
|
||||||
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
||||||
`swapon --show` before reasoning about headroom.
|
`swapon --show` before reasoning about headroom.
|
||||||
- **Silent RAM corruption outranks every software theory.** frank-primary's
|
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
||||||
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
||||||
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
||||||
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
||||||
@@ -763,7 +764,7 @@ linked runbooks.
|
|||||||
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
||||||
podman to runc without any raise). Upstream halogen's own run lines pass
|
podman to runc without any raise). Upstream halogen's own run lines pass
|
||||||
NO `--ulimit` at all — copy vendor run lines before adding flags
|
NO `--ulimit` at all — copy vendor run lines before adding flags
|
||||||
(measured 2026-10-05 on the frank-large cutover).
|
(measured 2026-10-05 on the frank-dragoon cutover).
|
||||||
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
||||||
`third_party/composable_kernel/include/ck/config.h` generated manually
|
`third_party/composable_kernel/include/ck/config.h` generated manually
|
||||||
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
||||||
@@ -789,7 +790,7 @@ linked runbooks.
|
|||||||
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
||||||
Record the child pid and kill it in a `finally`, then prove `free` came back
|
Record the child pid and kill it in a `finally`, then prove `free` came back
|
||||||
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
||||||
which also hits frank-secondary.
|
which also hits frank-zealot.
|
||||||
|
|
||||||
**llama.cpp serving**
|
**llama.cpp serving**
|
||||||
|
|
||||||
@@ -813,7 +814,7 @@ linked runbooks.
|
|||||||
builds its context against whatever model the draft context holds. Two
|
builds its context against whatever model the draft context holds. Two
|
||||||
context-backed types cannot share one process, and MTP's head only loads
|
context-backed types cannot share one process, and MTP's head only loads
|
||||||
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
||||||
recurrent model:** `draft-mtp,ngram-mod` on frank-large's qwen35 hybrid
|
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
|
||||||
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
||||||
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
||||||
disabling rollback support`, and MTP verification depends on that rollback,
|
disabling rollback support`, and MTP verification depends on that rollback,
|
||||||
@@ -855,7 +856,7 @@ linked runbooks.
|
|||||||
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
|
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
|
||||||
model — the sweep's or the seam's — can see it.)
|
model — the sweep's or the seam's — can see it.)
|
||||||
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
|
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
|
||||||
frank-secondary ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
|
frank-zealot ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
|
||||||
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
|
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
|
||||||
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
|
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
|
||||||
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
|
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
|
||||||
@@ -864,9 +865,9 @@ linked runbooks.
|
|||||||
than the ubatch cannot see it. The logical `--batch-size` was again free. And
|
than the ubatch cannot see it. The logical `--batch-size` was again free. And
|
||||||
the predecessor's "ubatch 2048 failed the scratch boot" was a shape
|
the predecessor's "ubatch 2048 failed the scratch boot" was a shape
|
||||||
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
|
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
|
||||||
it by restarting the SERVICE through a transient `FRANK_SECONDARY_ENV_FILE`
|
it by restarting the SERVICE through a transient `FRANK_ZEALOT_ENV_FILE`
|
||||||
drop-in, never a scratch port — one card holds one server. See
|
drop-in, never a scratch port — one card holds one server. See
|
||||||
[[frank-secondary-llama-mimo-q6k-2x200704]].
|
[[frank-zealot-llama-mimo-q6k-2x200704]].
|
||||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
||||||
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
|
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
|
||||||
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
|
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
|
||||||
@@ -888,7 +889,7 @@ linked runbooks.
|
|||||||
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
|
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
|
||||||
(`400 exceed_context_size_error: request (184001 tokens) exceeds the
|
(`400 exceed_context_size_error: request (184001 tokens) exceeds the
|
||||||
available context size (180224 tokens)`) even when the pool has room. So
|
available context size (180224 tokens)`) even when the pool has room. So
|
||||||
frank-secondary's 2 → 3 reslice moved each conversation from the model's native
|
frank-zealot's 2 → 3 reslice moved each conversation from the model's native
|
||||||
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
|
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
|
||||||
the harness would discover the ceiling as a 400 on a long conversation. Side
|
the harness would discover the ceiling as a 400 on a long conversation. Side
|
||||||
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
|
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
|
||||||
@@ -899,9 +900,9 @@ linked runbooks.
|
|||||||
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
|
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
|
||||||
itself is a further measured −10 %, weight-bandwidth, not slot geometry).
|
itself is a further measured −10 %, weight-bandwidth, not slot geometry).
|
||||||
All moves measured; both slot directions cost or buy window, never pool.
|
All moves measured; both slot directions cost or buy window, never pool.
|
||||||
See [[frank-secondary-llama-mimo-q6k-2x200704]].
|
See [[frank-zealot-llama-mimo-q6k-2x200704]].
|
||||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
||||||
costs ~1.5x its file size.** The frank-secondary 3090 fit was solved from two
|
costs ~1.5x its file size.** The frank-zealot 3090 fit was solved from two
|
||||||
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
|
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
|
||||||
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
|
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
|
||||||
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
|
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
|
||||||
@@ -909,24 +910,24 @@ linked runbooks.
|
|||||||
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
|
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
|
||||||
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
|
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
|
||||||
solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||||
`PARALLEL × 4096`. Recorded in `scripts/frank-secondary.env` so the next resize
|
`PARALLEL × 4096`. Recorded in `scripts/frank-zealot.env` so the next resize
|
||||||
starts from measurement.
|
starts from measurement.
|
||||||
- **`images` and `image_url` are NOT interchangeable: only the content part
|
- **`images` and `image_url` are NOT interchangeable: only the content part
|
||||||
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
||||||
model still invents an answer if the client used the legacy top-level
|
model still invents an answer if the client used the legacy top-level
|
||||||
`images` field on `/v1/chat/completions` — frank-secondary answered "Brown." for
|
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
|
||||||
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
||||||
failure is silent because a hallucinated color is a plausible answer. Probe
|
failure is silent because a hallucinated color is a plausible answer. Probe
|
||||||
vision with a solid color and expect an exact word.
|
vision with a solid color and expect an exact word.
|
||||||
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
||||||
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
||||||
then served frank-large: every tool-using agent on that endpoint was broken
|
then served frank-dragoon: every tool-using agent on that endpoint was broken
|
||||||
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
||||||
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
||||||
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
||||||
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
||||||
option and the same ~12x prefill collapse described in the ROCm note above.
|
option and the same ~12x prefill collapse described in the ROCm note above.
|
||||||
f16 KV needs nothing; frank-secondary ran f16 for that reason until
|
f16 KV needs nothing; frank-zealot ran f16 for that reason until
|
||||||
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
||||||
endpoint could take q8_0 KV.
|
endpoint could take q8_0 KV.
|
||||||
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
||||||
@@ -944,7 +945,7 @@ linked runbooks.
|
|||||||
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
||||||
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
||||||
A role whose level is unmapped fails on its FIRST request, and the failure
|
A role whose level is unmapped fails on its FIRST request, and the failure
|
||||||
names Jinja, not the model: every subagent on `frank-secondary` died with
|
names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||||
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
||||||
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
||||||
differ.
|
differ.
|
||||||
@@ -964,7 +965,7 @@ linked runbooks.
|
|||||||
times (`RuntimeError: No HIP GPUs are available`).
|
times (`RuntimeError: No HIP GPUs are available`).
|
||||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
||||||
ports off the gluetun service.
|
ports off the gluetun service.
|
||||||
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-large,
|
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
|
||||||
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
|
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
|
||||||
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
|
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
|
||||||
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
||||||
@@ -1018,9 +1019,9 @@ Active:
|
|||||||
|
|
||||||
| Service | Runbook |
|
| Service | Runbook |
|
||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-secondary (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1028,41 +1029,41 @@ Retired (rollback targets and history):
|
|||||||
|
|
||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-secondary`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-secondary-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
|
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-zealot-llama-mimo-q6k-2x200704.md` |
|
||||||
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||||
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-secondary`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-zealot`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-dragoon-rocmfpx-q8agent-mtp.md` |
|
||||||
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
|
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-dragoon-gufo-qwen38-27b.md` |
|
||||||
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
|
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-dragoon`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-dragoon-vllm-mimo-v26-bf16.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-secondary`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-secondary-llama-mimo-q6k-2x200704.md`) | `frank-secondary-llama-mimo-q6k.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-zealot`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-zealot-llama-mimo-q6k-2x200704.md`) | `frank-zealot-llama-mimo-q6k.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-secondary-llama-mimo-q5km.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-zealot-llama-mimo-q5km.md` |
|
||||||
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-secondary`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-secondary-sglang-mimo-v26.md` |
|
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-zealot`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-zealot-sglang-mimo-v26.md` |
|
||||||
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
|
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-archon`)|Dankserver|2026-09-30 → 10-01|`frank-archon-glm53-flash-w4a16.md`|
|
||||||
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
|
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-21 → 09-28 | `frank-dragoon-unsloth-qwen38-27b-q8xl.md` |
|
||||||
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
|
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-archon`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-archon-r9v-radiance-qwen38-uncensored.md` |
|
||||||
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` |
|
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-archon`) | Dankserver | 2026-09-26 → 09-27 | `frank-archon-llamacpp-qwen38-flash-next-q8.md` |
|
||||||
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
|
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-dragoon`) | Dankclaw | 2026-09-15 → 09-16 | `frank-dragoon-ik-uncensored.md` |
|
||||||
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
|
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
|
||||||
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-primary`) | Dankserver | 2026-09-26 (same-day swap) | `frank-primary-lucebox-ds4.md` |
|
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-archon`) | Dankserver | 2026-09-26 (same-day swap) | `frank-archon-lucebox-ds4.md` |
|
||||||
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
|
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
|
||||||
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
|
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-dragoon`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
|
||||||
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-large`) | Dankclaw | 2026-09-19 → 09-21 | `frank-large-ciru-orca.md` |
|
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-dragoon`) | Dankclaw | 2026-09-19 → 09-21 | `frank-dragoon-ciru-orca.md` |
|
||||||
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-unsloth-qwen38-flash-next.md` |
|
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-unsloth-qwen38-flash-next.md` |
|
||||||
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-kairic-edge-qwen38-27b.md` |
|
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-kairic-edge-qwen38-27b.md` |
|
||||||
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-secondary`) | Dankclaw | 2026-09-16 → 09-17 | `frank-secondary-qwen38-gsq-rco.md` |
|
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-zealot`) | Dankclaw | 2026-09-16 → 09-17 | `frank-zealot-qwen38-gsq-rco.md` |
|
||||||
| vLLM Gemma4-12B w4a16 (`frank-secondary`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-secondary-vllm-gemma4.md` |
|
| vLLM Gemma4-12B w4a16 (`frank-zealot`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-zealot-vllm-gemma4.md` |
|
||||||
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-secondary`) | Dankclaw | 2026-09-17 → 10-01 | `frank-secondary-gemma4-12b.md` |
|
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-zealot`) | Dankclaw | 2026-09-17 → 10-01 | `frank-zealot-gemma4-12b.md` |
|
||||||
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
|
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
|
||||||
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
|
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
|
||||||
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
|
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
|
||||||
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
|
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
|
||||||
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
|
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
|
||||||
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
|
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
|
||||||
| FreeToken (official + Huihui FP8, `frank-secondary`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
|
| FreeToken (official + Huihui FP8, `frank-zealot`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
|
||||||
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
|
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
|
||||||
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
|
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
|
||||||
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
|
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ one deploy script.
|
|||||||
| Host | User | IP | Podman stack | LLM service | LLM port |
|
| Host | User | IP | Podman stack | LLM service | LLM port |
|
||||||
|------|------|----|--------------|-------------|----------|
|
|------|------|----|--------------|-------------|----------|
|
||||||
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
|
||||||
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
|
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
|
||||||
|
|
||||||
## Active Services
|
## Active Services
|
||||||
|
|
||||||
@@ -30,7 +30,7 @@ one deploy script.
|
|||||||
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
|
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
|
||||||
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
|
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
|
||||||
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
|
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
|
||||||
| frank-large | dankclaw | — (bare endpoint) | 7770 |
|
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
|
||||||
|
|
||||||
## Structure
|
## Structure
|
||||||
|
|
||||||
|
|||||||
+9
-9
@@ -1,26 +1,26 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# autoresearch.sh — canonical frank-primary benchmark entry point (dev box).
|
# autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
|
||||||
#
|
#
|
||||||
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
|
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
|
||||||
# there, and the benchmark must own those cards, so the whole stop -> boot
|
# there, and the benchmark must own those cards, so the whole stop -> boot
|
||||||
# scratch -> bench -> restore sequence runs ON dankserver (see
|
# scratch -> bench -> restore sequence runs ON dankserver (see
|
||||||
# scripts/autoresearch-frank-primary.sh). This wrapper:
|
# scripts/autoresearch-frank-archon.sh). This wrapper:
|
||||||
# 1. copies the candidate config (the repo's scripts/frank-primary.env, the
|
# 1. copies the candidate config (the repo's scripts/frank-archon.env, the
|
||||||
# file the autoresearch loop edits) and the bench script to dankserver,
|
# file the autoresearch loop edits) and the bench script to dankserver,
|
||||||
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
|
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
|
||||||
# exit code back here verbatim.
|
# exit code back here verbatim.
|
||||||
# It deliberately does NOT push to origin: the scp dirties dankserver's
|
# It deliberately does NOT push to origin: the scp dirties dankserver's
|
||||||
# working tree, and the loop's uncommitted candidate config must not be
|
# working tree, and the loop's uncommitted candidate config must not be
|
||||||
# pushed. Only this file and scripts/autoresearch-frank-primary.sh are
|
# pushed. Only this file and scripts/autoresearch-frank-archon.sh are
|
||||||
# committed; the candidate env rides over ssh each run.
|
# committed; the candidate env rides over ssh each run.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REMOTE=dankserver
|
REMOTE=dankserver
|
||||||
REMOTE_REPO=/home/dank/serverAppConfigs
|
REMOTE_REPO=/home/dank/serverAppConfigs
|
||||||
|
|
||||||
scp -q "$DIR/scripts/frank-primary.env" \
|
scp -q "$DIR/scripts/frank-archon.env" \
|
||||||
"$REMOTE:$REMOTE_REPO/scripts/frank-primary.env" || { echo "scp env failed" >&2; exit 1; }
|
"$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
|
||||||
scp -q "$DIR/scripts/autoresearch-frank-primary.sh" \
|
scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
|
||||||
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-primary.sh" || { echo "scp script failed" >&2; exit 1; }
|
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
|
||||||
|
|
||||||
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-primary.sh $REMOTE_REPO/scripts/frank-primary.env"
|
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
|
||||||
|
|||||||
+1
-1
File diff suppressed because one or more lines are too long
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
|
# download-frank-archon-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
|
||||||
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
|
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
|
||||||
# vLLM frank-primary deployment on dankserver.
|
# vLLM frank-archon deployment on dankserver.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary-fp8.sh [dest_dir]
|
# bash scripts/download-frank-archon-fp8.sh [dest_dir]
|
||||||
#
|
#
|
||||||
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
|
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
|
||||||
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
|
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash
|
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-dragoon's DFlash
|
||||||
# speculative-decode arm (vLLM --speculative-config method=dflash). Public
|
# speculative-decode arm (vLLM --speculative-config method=dflash). Public
|
||||||
# repo — no auth header on the command line.
|
# repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# the draft for that target.
|
# the draft for that target.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-dflash.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="z-lab/Qwen3.5-9B-DFlash"
|
REPO="z-lab/Qwen3.5-9B-DFlash"
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large
|
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-dragoon
|
||||||
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
|
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
|
||||||
# command line (a Bearer token in argv is world-readable via /proc).
|
# command line (a Bearer token in argv is world-readable via /proc).
|
||||||
#
|
#
|
||||||
# Per-file FRESH downloads, no resume: the house rule (see
|
# Per-file FRESH downloads, no resume: the house rule (see
|
||||||
# download-frank-secondary.sh); -f/--fail-with-body plus size+sha256 against the
|
# download-frank-zealot.sh); -f/--fail-with-body plus size+sha256 against the
|
||||||
# Hub LFS oid is what makes an access error fail instead of "succeeding".
|
# Hub LFS oid is what makes an access error fail instead of "succeeding".
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-mimo.sh [DEST_DIR] [--verify-only]
|
||||||
#
|
#
|
||||||
# The verified set below is exactly what the 2026-10-01 bench window served:
|
# The verified set below is exactly what the 2026-10-01 bench window served:
|
||||||
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
|
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
|
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
|
||||||
# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the
|
# family's MTP head (15 mtp.* tensors) exists. frank-dragoon runs it as the
|
||||||
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
|
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
|
||||||
# mtp tensors; that is why SGLang's NEXTN on frank-secondary draws 0.00
|
# mtp tensors; that is why SGLang's NEXTN on frank-zealot draws 0.00
|
||||||
# acceptance). Public repo — no auth header on the command line.
|
# acceptance). Public repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
|
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-qwen35.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="Qwen/Qwen3.5-9B"
|
REPO="Qwen/Qwen3.5-9B"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-secondary
|
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-zealot
|
||||||
# (SGLang, RTX 3090). Public repo — no auth header on the command line.
|
# (SGLang, RTX 3090). Public repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
|
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
# the house rule.
|
# the house rule.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
|
||||||
#
|
#
|
||||||
# LFS shards are verified against the Hub LFS sha256; non-LFS text files
|
# LFS shards are verified against the Hub LFS sha256; non-LFS text files
|
||||||
# against their declared byte size. Idempotent: complete files are skipped.
|
# against their declared byte size. Idempotent: complete files are skipped.
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# bench-frank-large.sh — 64k-context benchmark for the frank-default-large stack.
|
# bench-frank-dragoon.sh — 64k-context benchmark for the frank-default-large stack.
|
||||||
#
|
#
|
||||||
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
|
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
|
||||||
# a single backend at 64k context. It starts llama-server on a scratch port,
|
# a single backend at 64k context. It starts llama-server on a scratch port,
|
||||||
@@ -7,15 +7,15 @@
|
|||||||
# prompt, and prints tokens/sec.
|
# prompt, and prints tokens/sec.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# ./scripts/bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]
|
# ./scripts/bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]
|
||||||
#
|
#
|
||||||
# Example:
|
# Example:
|
||||||
# ./scripts/bench-frank-large.sh ROCm0
|
# ./scripts/bench-frank-dragoon.sh ROCm0
|
||||||
# ./scripts/bench-frank-large.sh Vulkan0 --generations 3
|
# ./scripts/bench-frank-dragoon.sh Vulkan0 --generations 3
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
DEVICE="${1:?usage: bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]}"
|
DEVICE="${1:?usage: bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]}"
|
||||||
GENERATIONS=1
|
GENERATIONS=1
|
||||||
if [ "${2:-}" = "--generations" ]; then
|
if [ "${2:-}" = "--generations" ]; then
|
||||||
GENERATIONS="${3:?--generations needs a count}"
|
GENERATIONS="${3:?--generations needs a count}"
|
||||||
@@ -41,7 +41,7 @@ SERVICE_HEALTH[dankclaw]="http://127.0.0.1:7770/health"
|
|||||||
# Drop page caches before restart: with the model's pages fully cached, the
|
# Drop page caches before restart: with the model's pages fully cached, the
|
||||||
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
|
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
|
||||||
DROP_CACHES[dankclaw]=1
|
DROP_CACHES[dankclaw]=1
|
||||||
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-large).
|
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-dragoon).
|
||||||
# When present, deploy.sh uses it instead of inline sudo commands.
|
# When present, deploy.sh uses it instead of inline sudo commands.
|
||||||
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
|
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
|
||||||
# Health wait in iterations of 2s. Cold loads after the cache drop measured
|
# Health wait in iterations of 2s. Cold loads after the cache drop measured
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-large-ling).
|
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-dragoon-ling).
|
||||||
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
|
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
|
||||||
# back: systemctl stop frank-large-ling && systemctl start frank-default-large.
|
# back: systemctl stop frank-dragoon-ling && systemctl start frank-default-large.
|
||||||
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
|
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
|
||||||
#
|
#
|
||||||
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
|
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-large-ling TRIAL)
|
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-dragoon-ling TRIAL)
|
||||||
After=network.target
|
After=network.target
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
@@ -8,7 +8,7 @@
|
|||||||
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
|
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
|
||||||
# GGUF (block 42) - no separate --spec-draft-model needed.
|
# GGUF (block 42) - no separate --spec-draft-model needed.
|
||||||
#
|
#
|
||||||
# Tie-back: to restore the Qwen3.6-27B service, stop frank-large-ling and
|
# Tie-back: to restore the Qwen3.6-27B service, stop frank-dragoon-ling and
|
||||||
# start frank-default-large (its env/unit are committed and unchanged).
|
# start frank-default-large (its env/unit are committed and unchanged).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-default-large.ling.env in this directory unless
|
# Tunables come from frank-default-large.ling.env in this directory unless
|
||||||
|
|||||||
@@ -4,11 +4,11 @@
|
|||||||
# Runtime: llama.cpp llama-server (>= b10411).
|
# Runtime: llama.cpp llama-server (>= b10411).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-default-large.env in this directory unless the
|
# Tunables come from frank-default-large.env in this directory unless the
|
||||||
# FRANK_LARGE_ENV_FILE environment variable overrides that path.
|
# FRANK_DRAGOON_ENV_FILE environment variable overrides that path.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
+1
-1
File diff suppressed because one or more lines are too long
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
Decode throughput benchmark for frank-large.
|
Decode throughput benchmark for frank-dragoon.
|
||||||
Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
|
Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
|
||||||
generates 256 tokens, measures generation tok/s.
|
generates 256 tokens, measures generation tok/s.
|
||||||
|
|
||||||
@@ -13,12 +13,12 @@ SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
|
|||||||
PORT = 7770
|
PORT = 7770
|
||||||
N_GEN = 256
|
N_GEN = 256
|
||||||
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
|
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
|
||||||
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-large.env")
|
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
|
||||||
SERVER_DIR = os.path.dirname(SERVE)
|
SERVER_DIR = os.path.dirname(SERVE)
|
||||||
ROCM_PATH = "/opt/rocm"
|
ROCM_PATH = "/opt/rocm"
|
||||||
|
|
||||||
def make_env():
|
def make_env():
|
||||||
"""Build the same env as serve-frank-large.sh."""
|
"""Build the same env as serve-frank-dragoon.sh."""
|
||||||
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
|
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
|
||||||
|
|
||||||
# Source env file for tunables
|
# Source env file for tunables
|
||||||
@@ -59,7 +59,7 @@ def wait_ready():
|
|||||||
def send_request(prompt, max_tokens=N_GEN):
|
def send_request(prompt, max_tokens=N_GEN):
|
||||||
"""Send chat completion, measure generation throughput."""
|
"""Send chat completion, measure generation throughput."""
|
||||||
body = json.dumps({
|
body = json.dumps({
|
||||||
"model": "frank-large",
|
"model": "frank-dragoon",
|
||||||
"messages": [{"role": "user", "content": prompt}],
|
"messages": [{"role": "user", "content": prompt}],
|
||||||
"max_tokens": max_tokens,
|
"max_tokens": max_tokens,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
Benchmark n-gram speculative decoding vs no-spec for frank-large.
|
Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
|
||||||
|
|
||||||
Tests prompt-processing throughput with 16K token prompts (manageable request size).
|
Tests prompt-processing throughput with 16K token prompts (manageable request size).
|
||||||
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
|
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
|
||||||
@@ -103,7 +103,7 @@ def start_server(port, spec_type):
|
|||||||
def run_test(prompt_text, spec_type, port):
|
def run_test(prompt_text, spec_type, port):
|
||||||
"""Send a request and extract prompt processing metrics."""
|
"""Send a request and extract prompt processing metrics."""
|
||||||
data = json.dumps({
|
data = json.dumps({
|
||||||
"model": "frank-large",
|
"model": "frank-dragoon",
|
||||||
"messages": [{"role": "user", "content": prompt_text}],
|
"messages": [{"role": "user", "content": prompt_text}],
|
||||||
"max_tokens": TEST_TOKENS,
|
"max_tokens": TEST_TOKENS,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
@@ -189,7 +189,7 @@ def main():
|
|||||||
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
|
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
|
||||||
|
|
||||||
print("=" * 60)
|
print("=" * 60)
|
||||||
print(" N-gram Mod Benchmark — frank-large")
|
print(" N-gram Mod Benchmark — frank-dragoon")
|
||||||
print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
|
print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
|
||||||
print("=" * 60)
|
print("=" * 60)
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,19 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Benchmark n-gram speculative decoding vs no-spec for frank-large.
|
# Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
|
||||||
#
|
#
|
||||||
# This tests prompt-processing throughput at 64k context with two distinct
|
# This tests prompt-processing throughput at 64k context with two distinct
|
||||||
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
|
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
|
||||||
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
|
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
|
||||||
#
|
#
|
||||||
# Run on Dankclaw after stopping the production service:
|
# Run on Dankclaw after stopping the production service:
|
||||||
# sudo systemctl stop frank-large
|
# sudo systemctl stop frank-dragoon
|
||||||
# ./scripts/bench-ngram-mod.sh
|
# ./scripts/bench-ngram-mod.sh
|
||||||
# sudo systemctl start frank-large
|
# sudo systemctl start frank-dragoon
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a; source "$ENV_FILE"; set +a
|
set -a; source "$ENV_FILE"; set +a
|
||||||
fi
|
fi
|
||||||
@@ -184,7 +184,7 @@ run_single_test() {
|
|||||||
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
|
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d "{
|
-d "{
|
||||||
\"model\": \"frank-large\",
|
\"model\": \"frank-dragoon\",
|
||||||
\"prompt\": \"$prompt_text\",
|
\"prompt\": \"$prompt_text\",
|
||||||
\"max_tokens\": 128,
|
\"max_tokens\": 128,
|
||||||
\"temperature\": 0,
|
\"temperature\": 0,
|
||||||
@@ -220,7 +220,7 @@ run_single_test() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
echo "╔══════════════════════════════════════════════════════════╗"
|
echo "╔══════════════════════════════════════════════════════════╗"
|
||||||
echo "║ N-gram Mod Benchmark — frank-large ║"
|
echo "║ N-gram Mod Benchmark — frank-dragoon ║"
|
||||||
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
|
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
|
||||||
echo "╚══════════════════════════════════════════════════════════╝"
|
echo "╚══════════════════════════════════════════════════════════╝"
|
||||||
echo ""
|
echo ""
|
||||||
|
|||||||
@@ -15,9 +15,9 @@ import time
|
|||||||
import urllib.request
|
import urllib.request
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
SERVICE = "frank-large"
|
SERVICE = "frank-dragoon"
|
||||||
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-large.env"
|
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ALIAS = "frank-large"
|
ALIAS = "frank-dragoon"
|
||||||
DEFAULT_HOST = "localhost"
|
DEFAULT_HOST = "localhost"
|
||||||
DEFAULT_PORT = 7770
|
DEFAULT_PORT = 7770
|
||||||
DEFAULT_MAX_TOKENS = 16384
|
DEFAULT_MAX_TOKENS = 16384
|
||||||
|
|||||||
+2
-2
@@ -1,4 +1,4 @@
|
|||||||
# Tunables for frank-large (mradermacher i1-Q4_K_M of
|
# Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
|
||||||
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
|
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
|
||||||
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
|
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
|
||||||
#
|
#
|
||||||
@@ -65,7 +65,7 @@ MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K
|
|||||||
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
||||||
|
|
||||||
# Alias + network.
|
# Alias + network.
|
||||||
ALIAS=frank-large
|
ALIAS=frank-dragoon
|
||||||
BIND_ADDR=0.0.0.0
|
BIND_ADDR=0.0.0.0
|
||||||
LISTEN_PORT=7770
|
LISTEN_PORT=7770
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-large)
|
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -12,8 +12,8 @@ Group=dank
|
|||||||
# budget still needs room. Parity with the halogen quadlet's
|
# budget still needs room. Parity with the halogen quadlet's
|
||||||
# Ulimit=memlock=-1:-1.
|
# Ulimit=memlock=-1:-1.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-large
|
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-dragoon
|
||||||
# via ik_llama.cpp on Dankclaw.
|
# via ik_llama.cpp on Dankclaw.
|
||||||
#
|
#
|
||||||
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
|
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
|
||||||
@@ -26,7 +26,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -66,7 +66,7 @@ fi
|
|||||||
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
|
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
|
||||||
# sets it. Refuse to run without it.
|
# sets it. Refuse to run without it.
|
||||||
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
|
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
|
||||||
echo "frank-large: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
|
echo "frank-dragoon: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -112,7 +112,7 @@ fi
|
|||||||
: "${SPEC_DRAFT_POLL_BATCH:=1}"
|
: "${SPEC_DRAFT_POLL_BATCH:=1}"
|
||||||
|
|
||||||
# Text only: the server must not try to autoload the vision tower. Vision
|
# Text only: the server must not try to autoload the vision tower. Vision
|
||||||
# stays on frank-large (Dankclaw), which is where the fleet routes images.
|
# stays on frank-dragoon (Dankclaw), which is where the fleet routes images.
|
||||||
NO_MMPROJ="${NO_MMPROJ:-1}"
|
NO_MMPROJ="${NO_MMPROJ:-1}"
|
||||||
|
|
||||||
export HIP_VISIBLE_DEVICES
|
export HIP_VISIBLE_DEVICES
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""autoresearch-bench.py — deterministic greedy battery for frank-large.
|
"""autoresearch-bench.py — deterministic greedy battery for frank-dragoon.
|
||||||
|
|
||||||
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
|
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
|
||||||
Fixed embedded corpus, temperature 0, streaming so decode speed is measured
|
Fixed embedded corpus, temperature 0, streaming so decode speed is measured
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# autoresearch-frank-primary.sh — RUNS ON DANKSERVER, driven over ssh by the
|
# autoresearch-frank-archon.sh — RUNS ON DANKSERVER, driven over ssh by the
|
||||||
# dev-box autoresearch.sh. It is the canonical frank-primary benchmark entry
|
# dev-box autoresearch.sh. It is the canonical frank-archon benchmark entry
|
||||||
# point for the autoresearch loop.
|
# point for the autoresearch loop.
|
||||||
#
|
#
|
||||||
# Method (mandated by the Benchmarking Policy): the production service serves
|
# Method (mandated by the Benchmarking Policy): the production service serves
|
||||||
# real users and shares the two R9700s, so a bench beside it starves the GPUs
|
# real users and shares the two R9700s, so a bench beside it starves the GPUs
|
||||||
# and corrupts results. This script therefore:
|
# and corrupts results. This script therefore:
|
||||||
# 1. stops the production frank-primary unit (root, relink symlink: stop,
|
# 1. stops the production frank-archon unit (root, relink symlink: stop,
|
||||||
# NOT mask — mask refuses to replace a symlinked unit file),
|
# NOT mask — mask refuses to replace a symlinked unit file),
|
||||||
# 2. boots a scratch radiance container named `arbench` on the SAME two
|
# 2. boots a scratch radiance container named `arbench` on the SAME two
|
||||||
# cards with the candidate engine argv (the env file the loop edits),
|
# cards with the candidate engine argv (the env file the loop edits),
|
||||||
@@ -18,7 +18,7 @@
|
|||||||
# prefill and decode rates for that single request.
|
# prefill and decode rates for that single request.
|
||||||
# 5. prints METRIC lines and tears the scratch down,
|
# 5. prints METRIC lines and tears the scratch down,
|
||||||
# 6. restores production (systemctl start), via a trap so a failed bench
|
# 6. restores production (systemctl start), via a trap so a failed bench
|
||||||
# never leaves frank-primary down (the stale-probe-tail lesson).
|
# never leaves frank-archon down (the stale-probe-tail lesson).
|
||||||
#
|
#
|
||||||
# Determinism: the prompt is built from a fixed, sorted corpus of repo files
|
# Determinism: the prompt is built from a fixed, sorted corpus of repo files
|
||||||
# with a fixed random seed, so prompt_n is stable across runs. Sampling is
|
# with a fixed random seed, so prompt_n is stable across runs. Sampling is
|
||||||
@@ -32,10 +32,10 @@
|
|||||||
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
|
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
|
||||||
# differences: the container name (arbench, so prod's identity is untouched)
|
# differences: the container name (arbench, so prod's identity is untouched)
|
||||||
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
|
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
|
||||||
# prefill is cold — the prod tier at kvcache/frank-primary is left intact.
|
# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-primary.env}"
|
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
|
||||||
SCRATCH=arbench
|
SCRATCH=arbench
|
||||||
PORT=7770
|
PORT=7770
|
||||||
HEALTH="http://127.0.0.1:$PORT/health"
|
HEALTH="http://127.0.0.1:$PORT/health"
|
||||||
@@ -45,7 +45,7 @@ LOG=/home/dank/logs/arbench.log
|
|||||||
|
|
||||||
restore() {
|
restore() {
|
||||||
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
||||||
sudo systemctl start frank-primary >/dev/null 2>&1 || true
|
sudo systemctl start frank-archon >/dev/null 2>&1 || true
|
||||||
}
|
}
|
||||||
trap restore EXIT
|
trap restore EXIT
|
||||||
|
|
||||||
@@ -57,13 +57,13 @@ fi
|
|||||||
set -a; . "$ENV_FILE"; set +a
|
set -a; . "$ENV_FILE"; set +a
|
||||||
|
|
||||||
# --- stop production; the scratch needs both cards to itself ----------------
|
# --- stop production; the scratch needs both cards to itself ----------------
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
|
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
|
||||||
FRESH_KV="$FRANK_PRIMARY_STATE_DIR/kvcache/$SCRATCH"
|
FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
|
||||||
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
|
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
|
|
||||||
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
||||||
podman run -d --name "$SCRATCH" \
|
podman run -d --name "$SCRATCH" \
|
||||||
@@ -72,31 +72,31 @@ podman run -d --name "$SCRATCH" \
|
|||||||
--security-opt seccomp=unconfined \
|
--security-opt seccomp=unconfined \
|
||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device /dev/kfd --device /dev/dri \
|
--device /dev/kfd --device /dev/dri \
|
||||||
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR":/data \
|
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
"$FRANK_PRIMARY_IMAGE" \
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
|
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
--host :: --port "$PORT" \
|
--host :: --port "$PORT" \
|
||||||
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
|
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
|
||||||
--override-chat-template /templates/template.jinja \
|
--override-chat-template /templates/template.jinja \
|
||||||
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
|
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
|
||||||
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
|
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
|
||||||
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
|
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
|
||||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
||||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
||||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
||||||
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
||||||
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
|
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
||||||
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
|
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
|
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
|
||||||
|
|
||||||
+32
-32
@@ -1,19 +1,19 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# frank-primary-arms.sh — run a SEQUENCE of frank-primary bench arms and keep
|
# frank-archon-arms.sh — run a SEQUENCE of frank-archon bench arms and keep
|
||||||
# every arm's evidence. Runs on the operator box; each arm is one full window
|
# every arm's evidence. Runs on the operator box; each arm is one full window
|
||||||
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
|
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
|
||||||
# restore production and verify), so the fleet is only ever down for one arm at
|
# restore production and verify), so the fleet is only ever down for one arm at
|
||||||
# a time and the production unit is back up between arms.
|
# a time and the production unit is back up between arms.
|
||||||
#
|
#
|
||||||
# An arm is the repo's own frank-primary.env plus an OVERLAY of engine knobs
|
# An arm is the repo's own frank-archon.env plus an OVERLAY of engine knobs
|
||||||
# appended after it (later assignment wins when the serve script sources it),
|
# appended after it (later assignment wins when the serve script sources it),
|
||||||
# staged through FRANK_PRIMARY_AR_ENV_FILE so the checkout is never mutated and
|
# staged through FRANK_ARCHON_AR_ENV_FILE so the checkout is never mutated and
|
||||||
# each arm gets its own fingerprint (the memo can never answer one arm with
|
# each arm gets its own fingerprint (the memo can never answer one arm with
|
||||||
# another arm's numbers).
|
# another arm's numbers).
|
||||||
#
|
#
|
||||||
# Usage: scripts/autoresearch/frank-primary-arms.sh [arm ...]
|
# Usage: scripts/autoresearch/frank-archon-arms.sh [arm ...]
|
||||||
# scripts/autoresearch/frank-primary-arms.sh --list
|
# scripts/autoresearch/frank-archon-arms.sh --list
|
||||||
# Output, under /tmp/frank-primary-arms-<UTC stamp>/ (override with ARMS_DIR):
|
# Output, under /tmp/frank-archon-arms-<UTC stamp>/ (override with ARMS_DIR):
|
||||||
#
|
#
|
||||||
# <arm>.out full autoresearch output (metrics + engine evidence)
|
# <arm>.out full autoresearch output (metrics + engine evidence)
|
||||||
# <arm>.env the exact staged env
|
# <arm>.env the exact staged env
|
||||||
@@ -26,8 +26,8 @@ set -uo pipefail
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
|
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
|
||||||
BASE_ENV="$REPO/scripts/frank-primary.env"
|
BASE_ENV="$REPO/scripts/frank-archon.env"
|
||||||
ARMS_DIR="${ARMS_DIR:-/tmp/frank-primary-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
|
ARMS_DIR="${ARMS_DIR:-/tmp/frank-archon-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
|
||||||
|
|
||||||
# --- arms ------------------------------------------------------------------
|
# --- arms ------------------------------------------------------------------
|
||||||
# baseline: the shipped shape E, measured with THIS harness so the comparison is
|
# baseline: the shipped shape E, measured with THIS harness so the comparison is
|
||||||
@@ -45,18 +45,18 @@ LADDER3='{"cudagraph_capture_sizes":[4,8,12],"max_cudagraph_capture_size":12,"in
|
|||||||
# the engine as {temperature:0.7}.
|
# the engine as {temperature:0.7}.
|
||||||
GEN07='{"temperature":0.7}'
|
GEN07='{"temperature":0.7}'
|
||||||
OVER_B=(
|
OVER_B=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
)
|
)
|
||||||
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
|
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
|
||||||
# OFF. It is the fallback if B dies at graph capture, and it prices the arena
|
# OFF. It is the fallback if B dies at graph capture, and it prices the arena
|
||||||
# that precise memory buys (7,522 slots -> ~6,496).
|
# that precise memory buys (7,522 slots -> ~6,496).
|
||||||
OVER_B2=(
|
OVER_B2=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
'FRANK_PRIMARY_PRECISE_MEMORY='
|
'FRANK_ARCHON_PRECISE_MEMORY='
|
||||||
)
|
)
|
||||||
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
|
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
|
||||||
# 0.95. This is also the direct test of the theory that pinning the capture
|
# 0.95. This is also the direct test of the theory that pinning the capture
|
||||||
@@ -64,12 +64,12 @@ OVER_B2=(
|
|||||||
# the constant 3, which is NOT in the default ladder; 4 is).
|
# the constant 3, which is NOT in the default ladder; 4 is).
|
||||||
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
|
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
|
||||||
OVER_C=(
|
OVER_C=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER16'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER16'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
'FRANK_PRIMARY_SEQS=16'
|
'FRANK_ARCHON_SEQS=16'
|
||||||
'FRANK_PRIMARY_MNBT=4096'
|
'FRANK_ARCHON_MNBT=4096'
|
||||||
'FRANK_PRIMARY_UTIL=0.95'
|
'FRANK_ARCHON_UTIL=0.95'
|
||||||
)
|
)
|
||||||
|
|
||||||
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
|
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
|
||||||
@@ -83,11 +83,11 @@ OVER_C=(
|
|||||||
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
|
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
|
||||||
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
|
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
|
||||||
# batch shape.
|
# batch shape.
|
||||||
KV401K='FRANK_PRIMARY_KV_BYTES=4050000000'
|
KV401K='FRANK_ARCHON_KV_BYTES=4050000000'
|
||||||
OVER_D=(
|
OVER_D=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
"$KV401K"
|
"$KV401K"
|
||||||
)
|
)
|
||||||
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
|
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
|
||||||
@@ -97,12 +97,12 @@ OVER_D=(
|
|||||||
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
|
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
|
||||||
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
|
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
|
||||||
OVER_E=(
|
OVER_E=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
"$KV401K"
|
"$KV401K"
|
||||||
'FRANK_PRIMARY_MNBT=4096'
|
'FRANK_ARCHON_MNBT=4096'
|
||||||
'FRANK_PRIMARY_UTIL=0.99'
|
'FRANK_ARCHON_UTIL=0.99'
|
||||||
)
|
)
|
||||||
|
|
||||||
arm_overlay() {
|
arm_overlay() {
|
||||||
@@ -142,7 +142,7 @@ for arm in "${ARMS[@]}"; do
|
|||||||
# holds JSON must still be one valid JSON object after the file is SOURCED
|
# holds JSON must still be one valid JSON object after the file is SOURCED
|
||||||
# (that is the step that silently eats unquoted braces).
|
# (that is the step that silently eats unquoted braces).
|
||||||
if ! bash -c 'set -a; . "$1" >/dev/null; set +a
|
if ! bash -c 'set -a; . "$1" >/dev/null; set +a
|
||||||
for k in FRANK_PRIMARY_SPEC FRANK_PRIMARY_COMPCONFIG FRANK_PRIMARY_GEN_CONFIG; do
|
for k in FRANK_ARCHON_SPEC FRANK_ARCHON_COMPCONFIG FRANK_ARCHON_GEN_CONFIG; do
|
||||||
v="${!k:-}"
|
v="${!k:-}"
|
||||||
[ -n "$v" ] || continue
|
[ -n "$v" ] || continue
|
||||||
case "$v" in
|
case "$v" in
|
||||||
@@ -158,7 +158,7 @@ for arm in "${ARMS[@]}"; do
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
|
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
|
||||||
FRANK_PRIMARY_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
|
FRANK_ARCHON_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
|
||||||
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
|
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
|
||||||
rc=$?
|
rc=$?
|
||||||
echo "$rc" >"$ARMS_DIR/$arm.rc"
|
echo "$rc" >"$ARMS_DIR/$arm.rc"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""frank-primary decode measurement — runs INSIDE the engine container.
|
"""frank-archon decode measurement — runs INSIDE the engine container.
|
||||||
|
|
||||||
The method is the fleet's established one (the 2026-09-29 / 2026-10-01
|
The method is the fleet's established one (the 2026-09-29 / 2026-10-01
|
||||||
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
|
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
|
||||||
@@ -28,7 +28,7 @@ Workload (fixed, deterministic, offline):
|
|||||||
|
|
||||||
Env:
|
Env:
|
||||||
AR_BASE endpoint (default http://127.0.0.1:7770)
|
AR_BASE endpoint (default http://127.0.0.1:7770)
|
||||||
AR_MODEL served name (default frank-primary)
|
AR_MODEL served name (default frank-archon)
|
||||||
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
|
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
|
||||||
AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
|
AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
|
||||||
|
|
||||||
+14
-14
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# frank-primary-window.sh — one full bench window for frank-primary, run ON
|
# frank-archon-window.sh — one full bench window for frank-archon, run ON
|
||||||
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
|
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
|
||||||
#
|
#
|
||||||
# Discipline (AGENTS.md Benchmarking Policy): the production unit is
|
# Discipline (AGENTS.md Benchmarking Policy): the production unit is
|
||||||
@@ -12,9 +12,9 @@
|
|||||||
#
|
#
|
||||||
# Staging layout (created by the orchestrator, ephemeral — never a config
|
# Staging layout (created by the orchestrator, ephemeral — never a config
|
||||||
# source; the repo checkout is):
|
# source; the repo checkout is):
|
||||||
# /home/dank/.autoresearch-frank-primary/
|
# /home/dank/.autoresearch-frank-archon/
|
||||||
# serve.sh staged candidate serve script
|
# serve.sh staged candidate serve script
|
||||||
# frank-primary.env staged candidate env (the candidate knobs)
|
# frank-archon.env staged candidate env (the candidate knobs)
|
||||||
# measure.py the in-container measure client
|
# measure.py the in-container measure client
|
||||||
# hash candidate fingerprint
|
# hash candidate fingerprint
|
||||||
# run.json result (metrics + boot + kv + fingerprint)
|
# run.json result (metrics + boot + kv + fingerprint)
|
||||||
@@ -27,16 +27,16 @@
|
|||||||
# outcome; a failed restore forces exit 1).
|
# outcome; a failed restore forces exit 1).
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}"
|
STAGE="${FRANK_ARCHON_AR_STAGE:-/home/dank/.autoresearch-frank-archon}"
|
||||||
UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}"
|
UNIT="${FRANK_ARCHON_AR_UNIT:-frank-archon}"
|
||||||
CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}"
|
CTR="${FRANK_ARCHON_AR_CTR:-frank-archon-qwen}"
|
||||||
HEALTH_URL="http://127.0.0.1:7770/health"
|
HEALTH_URL="http://127.0.0.1:7770/health"
|
||||||
MODELS_URL="http://127.0.0.1:7770/v1/models"
|
MODELS_URL="http://127.0.0.1:7770/v1/models"
|
||||||
BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}"
|
BOOT_TIMEOUT="${FRANK_ARCHON_AR_BOOT_TIMEOUT:-1200}"
|
||||||
RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}"
|
RESTORE_TIMEOUT="${FRANK_ARCHON_AR_RESTORE_TIMEOUT:-1200}"
|
||||||
MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}"
|
MEASURE_TIMEOUT="${FRANK_ARCHON_AR_MEASURE_TIMEOUT:-1800}"
|
||||||
SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)"
|
SERVED="$(sed -n 's/^FRANK_ARCHON_SERVED_NAME=//p' "$STAGE/frank-archon.env" 2>/dev/null | head -1)"
|
||||||
SERVED="${SERVED:-frank-primary}"
|
SERVED="${SERVED:-frank-archon}"
|
||||||
|
|
||||||
RESTORED=0
|
RESTORED=0
|
||||||
|
|
||||||
@@ -72,7 +72,7 @@ capture_bench_failure() {
|
|||||||
echo "=== staged serve.out (tail 60) ==="
|
echo "=== staged serve.out (tail 60) ==="
|
||||||
tail -60 "$STAGE/serve.out" 2>/dev/null || true
|
tail -60 "$STAGE/serve.out" 2>/dev/null || true
|
||||||
echo "=== engine log (tail 60) ==="
|
echo "=== engine log (tail 60) ==="
|
||||||
tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true
|
tail -60 /home/dank/logs/frank_archon.log 2>/dev/null || true
|
||||||
} >"$STAGE/fail.log" 2>&1 || true
|
} >"$STAGE/fail.log" 2>&1 || true
|
||||||
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
|
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
|
||||||
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
|
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
|
||||||
@@ -114,7 +114,7 @@ trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
local f
|
local f
|
||||||
for f in serve.sh frank-primary.env measure.py hash; do
|
for f in serve.sh frank-archon.env measure.py hash; do
|
||||||
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
|
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
|
||||||
done
|
done
|
||||||
mkdir -p "$STAGE/last"
|
mkdir -p "$STAGE/last"
|
||||||
@@ -133,7 +133,7 @@ main() {
|
|||||||
|
|
||||||
t0="$(date +%s)"
|
t0="$(date +%s)"
|
||||||
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
|
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
|
||||||
FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \
|
FRANK_ARCHON_ENV_FILE="$STAGE/frank-archon.env" \
|
||||||
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
|
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
|
||||||
echo "$!" >"$STAGE/serve.pid"
|
echo "$!" >"$STAGE/serve.pid"
|
||||||
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Decode throughput benchmark for frank-large.
|
# Decode throughput benchmark for frank-dragoon.
|
||||||
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
|
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -75,7 +75,7 @@ print(json.dumps(' '.join(words)))
|
|||||||
t0=$(date +%s%N)
|
t0=$(date +%s%N)
|
||||||
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
|
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d "{\"model\":\"frank-large\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
|
-d "{\"model\":\"frank-dragoon\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
|
||||||
t1=$(date +%s%N)
|
t1=$(date +%s%N)
|
||||||
|
|
||||||
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
|
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
HTTP-based prefill-throughput sweep for frank-large.
|
HTTP-based prefill-throughput sweep for frank-dragoon.
|
||||||
Starts the server with a given --threads-batch value, sends a 64K-prompt via
|
Starts the server with a given --threads-batch value, sends a 64K-prompt via
|
||||||
the chat API (max_tokens=0), measures prompt_tokens/wall_time.
|
the chat API (max_tokens=0), measures prompt_tokens/wall_time.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Threads-batch sweep for frank-large on dankclaw
|
# Threads-batch sweep for frank-dragoon on dankclaw
|
||||||
# Benches prefill throughput at 65536 prompt tokens across threads-batch values.
|
# Benches prefill throughput at 65536 prompt tokens across threads-batch values.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
@@ -10,7 +10,7 @@ TB_LIST=(16 32 48 64 80 96 128)
|
|||||||
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
|
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
|
||||||
|
|
||||||
echo "========================================"
|
echo "========================================"
|
||||||
echo "THREADS-BATCH SWEEP — frank-large"
|
echo "THREADS-BATCH SWEEP — frank-dragoon"
|
||||||
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
|
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
|
||||||
echo "Threads-batch values: ${TB_LIST[*]}"
|
echo "Threads-batch values: ${TB_LIST[*]}"
|
||||||
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
|
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Benchmark ubatch with llama-bench for frank-large.
|
# Benchmark ubatch with llama-bench for frank-dragoon.
|
||||||
#
|
#
|
||||||
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
||||||
# process, an 8K prompt workload, and equal batch/ubatch values. No server
|
# process, an 8K prompt workload, and equal batch/ubatch values. No server
|
||||||
# process, HTTP requests, or log scraping are used.
|
# process, HTTP requests, or log scraping are used.
|
||||||
#
|
#
|
||||||
# Run on Dankclaw after stopping frank-large:
|
# Run on Dankclaw after stopping frank-dragoon:
|
||||||
# sudo systemctl stop frank-large
|
# sudo systemctl stop frank-dragoon
|
||||||
# ./scripts/bench-ubatch.sh
|
# ./scripts/bench-ubatch.sh
|
||||||
# sudo systemctl start frank-large
|
# sudo systemctl start frank-dragoon
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# build-frank-primary-rad.sh — build frank-primary's .rad container from the
|
# build-frank-archon-rad.sh — build frank-archon's .rad container from the
|
||||||
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
|
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
|
||||||
#
|
#
|
||||||
# bash scripts/build-frank-primary-rad.sh # convert
|
# bash scripts/build-frank-archon-rad.sh # convert
|
||||||
# bash scripts/build-frank-primary-rad.sh --plan-only # declare, no I/O
|
# bash scripts/build-frank-archon-rad.sh --plan-only # declare, no I/O
|
||||||
# bash scripts/build-frank-primary-rad.sh --restage # redo staging
|
# bash scripts/build-frank-archon-rad.sh --restage # redo staging
|
||||||
# bash scripts/build-frank-primary-rad.sh --force # rebuild existing
|
# bash scripts/build-frank-archon-rad.sh --force # rebuild existing
|
||||||
#
|
#
|
||||||
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
|
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
|
||||||
# architecture plugin by the checkpoint's quantisation DESCRIPTOR —
|
# architecture plugin by the checkpoint's quantisation DESCRIPTOR —
|
||||||
@@ -55,8 +55,8 @@ OUT="$OUT_DIR/qwen4exp-fp8-native.rad"
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$SCRIPT_DIR/frank-primary.env"
|
source "$SCRIPT_DIR/frank-archon.env"
|
||||||
IMAGE="$FRANK_PRIMARY_IMAGE"
|
IMAGE="$FRANK_ARCHON_IMAGE"
|
||||||
|
|
||||||
plan_only=0 restage=0 force=0
|
plan_only=0 restage=0 force=0
|
||||||
for a in "$@"; do
|
for a in "$@"; do
|
||||||
@@ -69,11 +69,11 @@ for a in "$@"; do
|
|||||||
done
|
done
|
||||||
|
|
||||||
if [ ! -f "$SRC/config.json" ]; then
|
if [ ! -f "$SRC/config.json" ]; then
|
||||||
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-primary.sh --fetch)" >&2
|
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-archon.sh --fetch)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "verifying raw checkpoint first..."
|
echo "verifying raw checkpoint first..."
|
||||||
bash "$SCRIPT_DIR/download-frank-primary.sh"
|
bash "$SCRIPT_DIR/download-frank-archon.sh"
|
||||||
|
|
||||||
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
|
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
|
||||||
echo "container already exists: $OUT"
|
echo "container already exists: $OUT"
|
||||||
@@ -83,7 +83,7 @@ fi
|
|||||||
|
|
||||||
# ---- staging ----------------------------------------------------------------
|
# ---- staging ----------------------------------------------------------------
|
||||||
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
|
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
|
||||||
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-stage"
|
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-stage"
|
||||||
mkdir -p "$TRASH"
|
mkdir -p "$TRASH"
|
||||||
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
|
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
|
||||||
echo "old staging dir moved to $TRASH"
|
echo "old staging dir moved to $TRASH"
|
||||||
@@ -138,7 +138,7 @@ CONVERT_ARGS=(/models/stage-qwen4exp-fp8 -o /models/rad/qwen4exp-fp8-native.rad
|
|||||||
# exactly and the plugin's one-element scale plane is produced.
|
# exactly and the plugin's one-element scale plane is produced.
|
||||||
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
|
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
|
||||||
[ "$force" -eq 1 ] && [ -f "$OUT" ] && {
|
[ "$force" -eq 1 ] && [ -f "$OUT" ] && {
|
||||||
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-rad"
|
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-rad"
|
||||||
mkdir -p "$TRASH"
|
mkdir -p "$TRASH"
|
||||||
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
|
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
|
||||||
echo "previous container moved to $TRASH"
|
echo "previous container moved to $TRASH"
|
||||||
@@ -158,8 +158,8 @@ echo "rad-info summary:"
|
|||||||
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
|
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
|
||||||
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
|
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
|
||||||
MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
|
MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
|
||||||
echo "container: $OUT (${MIB} MiB; env floor $FRANK_PRIMARY_RAD_MIN_MIB MiB)"
|
echo "container: $OUT (${MIB} MiB; env floor $FRANK_ARCHON_RAD_MIN_MIB MiB)"
|
||||||
if [ "$MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
|
if [ "$MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
|
||||||
echo "ERROR: container below the env floor — the convert was truncated" >&2
|
echo "ERROR: container below the env floor — the convert was truncated" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Build the gufo runtime that serves frank-large on Dankclaw.
|
# Build the gufo runtime that serves frank-dragoon on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/build-gufo.sh # build + install the pinned tag
|
# bash scripts/build-gufo.sh # build + install the pinned tag
|
||||||
# bash scripts/build-gufo.sh --verify # only check the installed tree
|
# bash scripts/build-gufo.sh --verify # only check the installed tree
|
||||||
@@ -93,4 +93,4 @@ cmake --install build/release
|
|||||||
|
|
||||||
echo "== verify =="
|
echo "== verify =="
|
||||||
"$PREFIX/bin/gufo" diagnose
|
"$PREFIX/bin/gufo" diagnose
|
||||||
echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
|
echo "frank-dragoon runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Build the ROCmFPX runtime that serves frank-large on Dankclaw.
|
# Build the ROCmFPX runtime that serves frank-dragoon on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
|
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
|
||||||
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree
|
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree
|
||||||
#
|
#
|
||||||
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
|
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
|
||||||
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's
|
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-dragoon's
|
||||||
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
|
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
|
||||||
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
|
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
|
||||||
# stock llama.cpp, gufo and every other loader on the host reject the file.
|
# stock llama.cpp, gufo and every other loader on the host reject the file.
|
||||||
@@ -30,7 +30,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
|
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
|
||||||
# The commit frank-large was qualified with (measured 2026-10-02).
|
# The commit frank-dragoon was qualified with (measured 2026-10-02).
|
||||||
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
|
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
|
||||||
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
|
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
|
||||||
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
|
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
|
||||||
@@ -86,4 +86,4 @@ cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
|
|||||||
|
|
||||||
echo "== verify =="
|
echo "== verify =="
|
||||||
"$BUILD_DIR/bin/llama-server" --version | head -1
|
"$BUILD_DIR/bin/llama-server" --version | head -1
|
||||||
echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
|
echo "frank-dragoon runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
|
||||||
|
|||||||
@@ -57,9 +57,9 @@ else
|
|||||||
echo " none"
|
echo " none"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
section "frank-primary unit"
|
section "frank-archon unit"
|
||||||
printf ' active: %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)"
|
printf ' active: %s\n' "$(systemctl is-active frank-archon 2>/dev/null | head -1)"
|
||||||
printf ' enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)"
|
printf ' enabled: %s\n' "$(systemctl is-enabled frank-archon 2>/dev/null | head -1)"
|
||||||
|
|
||||||
section "glm serve tail"
|
section "glm serve tail"
|
||||||
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
|
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
|
||||||
|
|||||||
+23
-23
@@ -61,9 +61,9 @@ SERVICE_ORDER = (
|
|||||||
"sonarr",
|
"sonarr",
|
||||||
"lidarr",
|
"lidarr",
|
||||||
"readarr",
|
"readarr",
|
||||||
"frank-large",
|
"frank-dragoon",
|
||||||
"frank-secondary",
|
"frank-zealot",
|
||||||
"frank-primary",
|
"frank-archon",
|
||||||
)
|
)
|
||||||
|
|
||||||
DANKCLAW_SERVICES = {
|
DANKCLAW_SERVICES = {
|
||||||
@@ -93,8 +93,8 @@ DANKCLAW_SERVICES = {
|
|||||||
"authentik-outpost-ldap",
|
"authentik-outpost-ldap",
|
||||||
"caddy",
|
"caddy",
|
||||||
"homepage",
|
"homepage",
|
||||||
"frank-large",
|
"frank-dragoon",
|
||||||
"frank-secondary",
|
"frank-zealot",
|
||||||
}
|
}
|
||||||
|
|
||||||
DANKSERVER_SERVICES = {
|
DANKSERVER_SERVICES = {
|
||||||
@@ -113,7 +113,7 @@ DANKSERVER_SERVICES = {
|
|||||||
"sonarr",
|
"sonarr",
|
||||||
"lidarr",
|
"lidarr",
|
||||||
"readarr",
|
"readarr",
|
||||||
"frank-primary",
|
"frank-archon",
|
||||||
"node-exporter",
|
"node-exporter",
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -316,7 +316,7 @@ def services_for_path(path: str, target: str) -> set[str]:
|
|||||||
if len(parts) >= 2 and parts[1] == "open-webui":
|
if len(parts) >= 2 and parts[1] == "open-webui":
|
||||||
_add(services, "open-webui")
|
_add(services, "open-webui")
|
||||||
return services
|
return services
|
||||||
# frank-large left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
||||||
# build, root unit): its files live in scripts/ and map there.
|
# build, root unit): its files live in scripts/ and map there.
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
if len(parts) >= 2 and parts[1] == "observability":
|
||||||
# The five observability containers became Quadlet units on
|
# The five observability containers became Quadlet units on
|
||||||
@@ -371,10 +371,10 @@ def services_for_path(path: str, target: str) -> set[str]:
|
|||||||
if name == "node-exporter":
|
if name == "node-exporter":
|
||||||
_add(services, "node-exporter")
|
_add(services, "node-exporter")
|
||||||
return services
|
return services
|
||||||
# frank-primary's quadlet-era directory left the tree on 2026-09-26
|
# frank-archon's quadlet-era directory left the tree on 2026-09-26
|
||||||
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
|
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
|
||||||
# the trio in scripts/ now maps it). Nothing lives under
|
# the trio in scripts/ now maps it). Nothing lives under
|
||||||
# dankserver/frank-primary/ except dated trash, and trash plans
|
# dankserver/frank-archon/ except dated trash, and trash plans
|
||||||
# nothing (the retire rule above the service blocks).
|
# nothing (the retire rule above the service blocks).
|
||||||
return services
|
return services
|
||||||
|
|
||||||
@@ -400,32 +400,32 @@ def services_for_path(path: str, target: str) -> set[str]:
|
|||||||
# change to it redeploying alertmanager is the only path that
|
# change to it redeploying alertmanager is the only path that
|
||||||
# reaches it on the host.
|
# reaches it on the host.
|
||||||
_add(services, "alertmanager")
|
_add(services, "alertmanager")
|
||||||
if name in {"frank-secondary.env", "frank-secondary.service", "serve-frank-secondary.sh"}:
|
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
|
||||||
# llama.cpp frank-secondary (2026-09-16) keeps its unit, env and serve
|
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
|
||||||
# files here; each one selects frank-secondary only. The ik_llama.cpp
|
# files here; each one selects frank-zealot only. The ik_llama.cpp
|
||||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
||||||
# which is archive material and plans nothing.
|
# which is archive material and plans nothing.
|
||||||
_add(services, "frank-secondary")
|
_add(services, "frank-zealot")
|
||||||
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
|
if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
|
||||||
# CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and
|
# CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
|
||||||
# serve files here the same way: a root unit on a host-built
|
# serve files here the same way: a root unit on a host-built
|
||||||
# runtime. The download script fetches the payload, which is
|
# runtime. The download script fetches the payload, which is
|
||||||
# host-side state, so it selects nothing.
|
# host-side state, so it selects nothing.
|
||||||
_add(services, "frank-large")
|
_add(services, "frank-dragoon")
|
||||||
if name == "qwen-sharp-v22.4.0.jinja":
|
if name == "qwen-sharp-v22.4.0.jinja":
|
||||||
# The Qwen-Sharp chat template frank-large loads via
|
# The Qwen-Sharp chat template frank-dragoon loads via
|
||||||
# --chat-template-file. It lived under dankserver/frank-medium/ until
|
# --chat-template-file. It lived under dankserver/frank-medium/ until
|
||||||
# the 2026-09-26 cutover retired that directory; editing it changes
|
# the 2026-09-26 cutover retired that directory; editing it changes
|
||||||
# what the live frank-large endpoint renders, so it plans
|
# what the live frank-dragoon endpoint renders, so it plans
|
||||||
# frank-large.
|
# frank-dragoon.
|
||||||
_add(services, "frank-large")
|
_add(services, "frank-dragoon")
|
||||||
if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}:
|
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
|
||||||
# llama.cpp frank-primary (2026-09-26, replacing the Lucebox
|
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
|
||||||
# DS4.1 quadlet) keeps its unit, env and serve files here the
|
# DS4.1 quadlet) keeps its unit, env and serve files here the
|
||||||
# same way: a root unit on the unsloth b11030-mix prebuilt
|
# same way: a root unit on the unsloth b11030-mix prebuilt
|
||||||
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
||||||
# the payload, which is host-side state, so it selects nothing.
|
# the payload, which is host-side state, so it selects nothing.
|
||||||
_add(services, "frank-primary")
|
_add(services, "frank-archon")
|
||||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
# Deployment/test/benchmark scripts and other repo tooling do not
|
||||||
# restart a runtime service.
|
# restart a runtime service.
|
||||||
return services
|
return services
|
||||||
|
|||||||
+113
-113
@@ -90,12 +90,12 @@ service_is_allowed() {
|
|||||||
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
||||||
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
||||||
dankclaw:frank-large|dankclaw:frank-secondary|\
|
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
||||||
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
|
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
|
||||||
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
|
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
|
||||||
dankserver:frank-primary|dankserver:node-exporter|\
|
dankserver:frank-archon|dankserver:node-exporter|\
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
||||||
frankbook-pro:readarr)
|
frankbook-pro:readarr)
|
||||||
@@ -192,7 +192,7 @@ add_pending_services() {
|
|||||||
continue
|
continue
|
||||||
fi
|
fi
|
||||||
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
|
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
|
||||||
if [[ "$TARGET" == dankserver && "$service" == frank-large ]]; then continue; fi
|
if [[ "$TARGET" == dankserver && "$service" == frank-dragoon ]]; then continue; fi
|
||||||
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
|
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
|
||||||
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
|
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
|
||||||
echo "WARN: unable to compare state for $service; retaining it as pending" >&2
|
echo "WARN: unable to compare state for $service; retaining it as pending" >&2
|
||||||
@@ -1092,21 +1092,21 @@ deploy_anubis() {
|
|||||||
wait_container_healthy anubis 60 || return 1
|
wait_container_healthy anubis 60 || return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
# frank-large is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
|
# frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
|
||||||
# through scripts/frank-large.env) serving the Quat3rnion ABLITERATED
|
# through scripts/frank-dragoon.env) serving the Quat3rnion ABLITERATED
|
||||||
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
|
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
|
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
|
||||||
# resident) + the REQUIRED n-gram lookup table + the vision tower —
|
# resident) + the REQUIRED n-gram lookup table + the vision tower —
|
||||||
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
|
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
|
||||||
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script
|
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script
|
||||||
# drives `podman run` directly (the frank-primary pattern): relink_unit
|
# drives `podman run` directly (the frank-archon pattern): relink_unit
|
||||||
# points /etc/systemd/system/frank-large.service at the repo file, boot
|
# points /etc/systemd/system/frank-dragoon.service at the repo file, boot
|
||||||
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
|
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
|
||||||
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
|
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
|
||||||
# rollback = revert the swap commit's trio; the 27B payload stays on
|
# rollback = revert the swap commit's trio; the 27B payload stays on
|
||||||
# disk (~/models/halogen-qwen3.8-27b).
|
# disk (~/models/halogen-qwen3.8-27b).
|
||||||
#
|
#
|
||||||
# The gates mirror serve-frank-large.sh's predicates EXACTLY (the
|
# The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the
|
||||||
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
|
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
|
||||||
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
|
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
|
||||||
# byte count, the tokenizer directory holding every named file as a
|
# byte count, the tokenizer directory holding every named file as a
|
||||||
@@ -1114,23 +1114,23 @@ deploy_anubis() {
|
|||||||
# artifact would leave a restart-looping unit and a dark endpoint. The
|
# artifact would leave a restart-looping unit and a dark endpoint. The
|
||||||
# port gate is the same as every cutover since 2026-09-16: nothing
|
# port gate is the same as every cutover since 2026-09-16: nothing
|
||||||
# starts until 7770 is released.
|
# starts until 7770 is released.
|
||||||
LLM_LARGE_IMAGE="${LLM_LARGE_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
||||||
LLM_LARGE_CKPT="${LLM_LARGE_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
|
LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
|
||||||
LLM_LARGE_CKPT_BYTES="${LLM_LARGE_CKPT_BYTES:-66687678464}"
|
LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678464}"
|
||||||
LLM_LARGE_NGRAM="${LLM_LARGE_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
|
LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
|
||||||
LLM_LARGE_NGRAM_BYTES="${LLM_LARGE_NGRAM_BYTES:-51200246144}"
|
LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}"
|
||||||
LLM_LARGE_VISION="${LLM_LARGE_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
|
LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
|
||||||
LLM_LARGE_VISION_BYTES="${LLM_LARGE_VISION_BYTES:-897916416}"
|
LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
|
||||||
LLM_LARGE_TOK_DIR="${LLM_LARGE_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
|
LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
|
||||||
LLM_LARGE_TOK_FILES="${LLM_LARGE_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
||||||
|
|
||||||
retire_halogen_large_quadlet() {
|
retire_halogen_dragoon_quadlet() {
|
||||||
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
|
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
|
||||||
systemctl --user stop frank-large.service 2>/dev/null || true
|
systemctl --user stop frank-dragoon.service 2>/dev/null || true
|
||||||
if [[ -e "$quadlet_unit_dir/frank-large.container" ]]; then
|
if [[ -e "$quadlet_unit_dir/frank-dragoon.container" ]]; then
|
||||||
mkdir -p "$trash"
|
mkdir -p "$trash"
|
||||||
mv -- "$quadlet_unit_dir/frank-large.container" "$trash/frank-large.container"
|
mv -- "$quadlet_unit_dir/frank-dragoon.container" "$trash/frank-dragoon.container"
|
||||||
echo "retired quadlet frank-large.container -> $trash"
|
echo "retired quadlet frank-dragoon.container -> $trash"
|
||||||
fi
|
fi
|
||||||
systemctl --user daemon-reload
|
systemctl --user daemon-reload
|
||||||
}
|
}
|
||||||
@@ -1154,165 +1154,165 @@ wait_port_free() {
|
|||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
deploy_llm_large() {
|
deploy_llm_dragoon() {
|
||||||
# Refuse rather than restart-loop; every gate below has a twin in
|
# Refuse rather than restart-loop; every gate below has a twin in
|
||||||
# serve-frank-large.sh and they must agree (2026-10-01 lesson).
|
# serve-frank-dragoon.sh and they must agree (2026-10-01 lesson).
|
||||||
# Capture podman output before grepping: `podman images | grep -q`
|
# Capture podman output before grepping: `podman images | grep -q`
|
||||||
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
|
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
|
||||||
if ! grep -qF "${LLM_LARGE_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
|
if ! grep -qF "${LLM_DRAGOON_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
|
||||||
echo "ERROR: halogen image absent: $LLM_LARGE_IMAGE" >&2
|
echo "ERROR: halogen image absent: $LLM_DRAGOON_IMAGE" >&2
|
||||||
echo "frank-large: pull it: podman pull ${LLM_LARGE_IMAGE%@*}" >&2
|
echo "frank-dragoon: pull it: podman pull ${LLM_DRAGOON_IMAGE%@*}" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
if [[ ! -f "$LLM_LARGE_CKPT" ]]; then
|
if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
|
||||||
echo "ERROR: halogen checkpoint absent at $LLM_LARGE_CKPT" >&2
|
echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
if [[ "$(stat -c %s "$LLM_LARGE_CKPT")" != "$LLM_LARGE_CKPT_BYTES" ]]; then
|
if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
|
||||||
echo "ERROR: halogen checkpoint is not $LLM_LARGE_CKPT_BYTES bytes (partial download?)" >&2
|
echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
|
||||||
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
|
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
for gate in "$LLM_LARGE_NGRAM:$LLM_LARGE_NGRAM_BYTES:n-gram table" \
|
for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
|
||||||
"$LLM_LARGE_VISION:$LLM_LARGE_VISION_BYTES:vision tower"; do
|
"$LLM_DRAGOON_VISION:$LLM_DRAGOON_VISION_BYTES:vision tower"; do
|
||||||
IFS=: read -r gpath gbytes gname <<<"$gate"
|
IFS=: read -r gpath gbytes gname <<<"$gate"
|
||||||
if [[ ! -f "$gpath" ]]; then
|
if [[ ! -f "$gpath" ]]; then
|
||||||
echo "ERROR: halogen $gname absent at $gpath" >&2
|
echo "ERROR: halogen $gname absent at $gpath" >&2
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
|
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
|
||||||
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
|
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
|
||||||
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
|
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
if [[ -L "$LLM_LARGE_TOK_DIR" || ! -d "$LLM_LARGE_TOK_DIR" ]]; then
|
if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
|
||||||
echo "ERROR: tokenizer dir missing or a symlink at $LLM_LARGE_TOK_DIR" >&2
|
echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
for tf in $LLM_LARGE_TOK_FILES; do
|
for tf in $LLM_DRAGOON_TOK_FILES; do
|
||||||
if [[ ! -f "$LLM_LARGE_TOK_DIR/$tf" || -L "$LLM_LARGE_TOK_DIR/$tf" ]]; then
|
if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
|
||||||
echo "ERROR: tokenizer file missing or a symlink: $LLM_LARGE_TOK_DIR/$tf" >&2
|
echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
retire_halogen_large_quadlet
|
retire_halogen_dragoon_quadlet
|
||||||
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
||||||
# the container) before waiting on the port. The stray sweep is keyed
|
# the container) before waiting on the port. The stray sweep is keyed
|
||||||
# on the FULL container name, which appears in the argv of a manual
|
# on the FULL container name, which appears in the argv of a manual
|
||||||
# `podman run --name frank-large-halogen` / `podman wait` but NOT in
|
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
|
||||||
# this deployer's own command line (`bash deploy-remote.sh
|
# this deployer's own command line (`bash deploy-remote.sh
|
||||||
# frank-large`) — so it cannot shoot itself in the foot (the fleet
|
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
|
||||||
# pkill law), and it can never match frank-secondary's llama-server.
|
# pkill law), and it can never match frank-zealot's llama-server.
|
||||||
run_privileged systemctl stop frank-large.service 2>/dev/null || true
|
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
||||||
pkill -f 'frank-large-halogen' 2>/dev/null || true
|
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
||||||
podman rm -f frank-large frank-large-halogen >/dev/null 2>&1 || true
|
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
||||||
# Nothing may start until the predecessor has released the port
|
# Nothing may start until the predecessor has released the port
|
||||||
# (2026-09-16: "Deactivated successfully" was not "port free").
|
# (2026-09-16: "Deactivated successfully" was not "port free").
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-large
|
relink_unit frank-dragoon
|
||||||
run_privileged systemctl daemon-reload
|
run_privileged systemctl daemon-reload
|
||||||
run_privileged systemctl enable frank-large.service
|
run_privileged systemctl enable frank-dragoon.service
|
||||||
run_privileged systemctl restart frank-large.service
|
run_privileged systemctl restart frank-dragoon.service
|
||||||
wait_for_health frank-large http://127.0.0.1:7770/health 1200
|
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_secondary — frank-secondary is llama.cpp (master build, CUDA sm_86) on
|
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
|
||||||
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
||||||
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
|
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
|
||||||
# no generated unit to avoid. relink_unit points
|
# no generated unit to avoid. relink_unit points
|
||||||
# /etc/systemd/system/frank-secondary.service at the repo file and boot
|
# /etc/systemd/system/frank-zealot.service at the repo file and boot
|
||||||
# persistence is that unit's [Install] section.
|
# persistence is that unit's [Install] section.
|
||||||
#
|
#
|
||||||
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
|
# It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
|
||||||
# not shared: frank-large's engine runs on the gfx1151 iGPU.
|
# not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
|
||||||
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
||||||
LLM_SECONDARY_RUNTIME="${LLM_SECONDARY_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
||||||
# The projector IS the vision capability here: the model is natively
|
# The projector IS the vision capability here: the model is natively
|
||||||
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
||||||
# its own BF16 projector in the same repo, so a deploy that starts without
|
# its own BF16 projector in the same repo, so a deploy that starts without
|
||||||
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
||||||
# frank-large's.
|
# frank-dragoon's.
|
||||||
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
||||||
|
|
||||||
deploy_llm_secondary() {
|
deploy_llm_zealot() {
|
||||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why.
|
||||||
if [[ ! -x "$LLM_SECONDARY_RUNTIME" ]]; then
|
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
|
||||||
echo "ERROR: llama.cpp runtime absent at $LLM_SECONDARY_RUNTIME" >&2
|
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
|
||||||
echo "frank-secondary: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
|
echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
|
||||||
echo "frank-secondary: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
|
echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
|
||||||
echo "frank-secondary: && cmake --build build-cuda --target llama-server) first." >&2
|
echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
for f in "$LLM_SECONDARY_MODEL_FILE" "$LLM_SECONDARY_MMPROJ"; do
|
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
|
||||||
if [[ ! -f "$f" ]]; then
|
if [[ ! -f "$f" ]]; then
|
||||||
echo "ERROR: frank-secondary payload file absent at $f" >&2
|
echo "ERROR: frank-zealot payload file absent at $f" >&2
|
||||||
echo "frank-secondary: run scripts/download-frank-secondary.sh first;" >&2
|
echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
|
||||||
echo "frank-secondary: refusing to start a unit that cannot load." >&2
|
echo "frank-zealot: refusing to start a unit that cannot load." >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
relink_unit frank-secondary
|
relink_unit frank-zealot
|
||||||
run_privileged systemctl daemon-reload
|
run_privileged systemctl daemon-reload
|
||||||
run_privileged systemctl enable frank-secondary.service
|
run_privileged systemctl enable frank-zealot.service
|
||||||
run_privileged systemctl restart frank-secondary.service
|
run_privileged systemctl restart frank-zealot.service
|
||||||
wait_for_health frank-secondary http://127.0.0.1:7771/health 900 || return 1
|
wait_for_health frank-zealot http://127.0.0.1:7771/health 900 || return 1
|
||||||
# False-green hardening (frank-large's lesson): the port answering
|
# False-green hardening (frank-dragoon's lesson): the port answering
|
||||||
# proves nothing about WHO answers — the unit must be active with a
|
# proves nothing about WHO answers — the unit must be active with a
|
||||||
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
|
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
|
||||||
local main_pid
|
local main_pid
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-secondary.service)"
|
main_pid="$(run_privileged systemctl show -p MainPID --value frank-zealot.service)"
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
||||||
echo "FAIL: :7771 answered but frank-secondary.service has no MainPID — a stray process holds the port" >&2
|
echo "FAIL: :7771 answered but frank-zealot.service has no MainPID — a stray process holds the port" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
echo "frank-secondary: health answered by unit MainPID $main_pid"
|
echo "frank-zealot: health answered by unit MainPID $main_pid"
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_primary — frank-primary is the radiance engine (StillDeadcode,
|
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
||||||
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
||||||
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
||||||
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
||||||
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
||||||
# the image entrypoint; the image lives in dank's rootless podman storage).
|
# the image entrypoint; the image lives in dank's rootless podman storage).
|
||||||
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
||||||
# frank-primary-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
||||||
# fork (tag frank-primary-vllm-shapeG-20261003). Same port and alias; the
|
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
||||||
# harness models.yml id is unchanged.
|
# harness models.yml id is unchanged.
|
||||||
#
|
#
|
||||||
# Gates: the image must exist under its digest ref in dank's rootless
|
# Gates: the image must exist under its digest ref in dank's rootless
|
||||||
# storage (FRANK_PRIMARY_IMAGE), the .rad container file must exist at the
|
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
||||||
# env's size floor (FRANK_PRIMARY_MODEL_FILE >= FRANK_PRIMARY_RAD_MIN_MIB —
|
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
||||||
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
||||||
# must be complete. Per-file sha256 verification is the fetch script's job
|
# must be complete. Per-file sha256 verification is the fetch script's job
|
||||||
# at download time and rad-convert/rad-info's job at build time, never
|
# at download time and rad-convert/rad-info's job at build time, never
|
||||||
# re-hashed here.
|
# re-hashed here.
|
||||||
LLM_PRIMARY_CONTAINER=frank-primary
|
LLM_ARCHON_CONTAINER=frank-archon
|
||||||
LLM_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
|
LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
|
||||||
|
|
||||||
deploy_llm_primary() {
|
deploy_llm_archon() {
|
||||||
# Refuse rather than restart-loop: a unit whose image or payload is
|
# Refuse rather than restart-loop: a unit whose image or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why.
|
||||||
local env_file="/home/dank/serverAppConfigs/scripts/frank-primary.env"
|
local env_file="/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
||||||
local image model_dir model_file rad_min mib
|
local image model_dir model_file rad_min mib
|
||||||
if [[ ! -f "$env_file" ]]; then
|
if [[ ! -f "$env_file" ]]; then
|
||||||
echo "ERROR: frank-primary env file absent: $env_file" >&2
|
echo "ERROR: frank-archon env file absent: $env_file" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
image="$(sed -n 's/^FRANK_PRIMARY_IMAGE=//p' "$env_file" | tail -1)"
|
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
|
||||||
model_dir="$(sed -n 's/^FRANK_PRIMARY_MODEL_DIR=//p' "$env_file" | tail -1)"
|
model_dir="$(sed -n 's/^FRANK_ARCHON_MODEL_DIR=//p' "$env_file" | tail -1)"
|
||||||
model_file="$(sed -n 's/^FRANK_PRIMARY_MODEL_FILE=//p' "$env_file" | tail -1)"
|
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
|
||||||
rad_min="$(sed -n 's/^FRANK_PRIMARY_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
||||||
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
||||||
echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
echo "ERROR: frank-archon.env missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
# The image gate is the digest ref itself: a name@sha256 ref exists in
|
# The image gate is the digest ref itself: a name@sha256 ref exists in
|
||||||
@@ -1320,18 +1320,18 @@ deploy_llm_primary() {
|
|||||||
# pipefail/grep race (the 2026-09-30 lesson).
|
# pipefail/grep race (the 2026-09-30 lesson).
|
||||||
if ! podman image exists "$image"; then
|
if ! podman image exists "$image"; then
|
||||||
echo "ERROR: runtime image absent from storage: $image" >&2
|
echo "ERROR: runtime image absent from storage: $image" >&2
|
||||||
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest" >&2
|
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
if [[ ! -f "$model_dir/$model_file" ]]; then
|
if [[ ! -f "$model_dir/$model_file" ]]; then
|
||||||
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
||||||
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
|
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
||||||
if [[ "$mib" -lt "$rad_min" ]]; then
|
if [[ "$mib" -lt "$rad_min" ]]; then
|
||||||
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
||||||
echo "frank-primary: rebuild it (scripts/build-frank-primary-rad.sh --force)" >&2
|
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -1343,19 +1343,19 @@ deploy_llm_primary() {
|
|||||||
# The legacy identities (the retired R9V container, the ad hoc
|
# The legacy identities (the retired R9V container, the ad hoc
|
||||||
# experiment container qwen-fp8, the retired glm-probe) hold the same
|
# experiment container qwen-fp8, the retired glm-probe) hold the same
|
||||||
# GPUs and die here too.
|
# GPUs and die here too.
|
||||||
run_privileged systemctl stop frank-primary.service 2>/dev/null || true
|
run_privileged systemctl stop frank-archon.service 2>/dev/null || true
|
||||||
local ctr
|
local ctr
|
||||||
for ctr in $LLM_PRIMARY_LEGACY_CONTAINERS "$LLM_PRIMARY_CONTAINER"; do
|
for ctr in $LLM_ARCHON_LEGACY_CONTAINERS "$LLM_ARCHON_CONTAINER"; do
|
||||||
podman rm -f "$ctr" >/dev/null 2>&1 || true
|
podman rm -f "$ctr" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
# Nothing starts until the port is released: the lucebox cutover
|
# Nothing starts until the port is released: the lucebox cutover
|
||||||
# (2026-09-26) and the frank-large cutover (2026-09-16) both failed
|
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
|
||||||
# exactly here when the next engine bound 7770 a second too early.
|
# exactly here when the next engine bound 7770 a second too early.
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-primary
|
relink_unit frank-archon
|
||||||
run_privileged systemctl daemon-reload
|
run_privileged systemctl daemon-reload
|
||||||
run_privileged systemctl enable frank-primary.service
|
run_privileged systemctl enable frank-archon.service
|
||||||
run_privileged systemctl restart frank-primary.service
|
run_privileged systemctl restart frank-archon.service
|
||||||
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
||||||
# publisher's compose healthcheck allows a 15-minute start_period).
|
# publisher's compose healthcheck allows a 15-minute start_period).
|
||||||
# Parity with every predecessor on this port. The HTTP listener binds
|
# Parity with every predecessor on this port. The HTTP listener binds
|
||||||
@@ -1366,14 +1366,14 @@ deploy_llm_primary() {
|
|||||||
local i ok=0
|
local i ok=0
|
||||||
for ((i=1; i<=900; i++)); do
|
for ((i=1; i<=900; i++)); do
|
||||||
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
|
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
|
||||||
echo "frank-primary engine healthy after approximately $((i * 2))s"
|
echo "frank-archon engine healthy after approximately $((i * 2))s"
|
||||||
ok=1
|
ok=1
|
||||||
break
|
break
|
||||||
fi
|
fi
|
||||||
sleep 2
|
sleep 2
|
||||||
done
|
done
|
||||||
if [[ "$ok" != 1 ]]; then
|
if [[ "$ok" != 1 ]]; then
|
||||||
echo "FAIL: frank-primary /health never answered 200 in 1800s" >&2
|
echo "FAIL: frank-archon /health never answered 200 in 1800s" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
|
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
|
||||||
@@ -1381,12 +1381,12 @@ deploy_llm_primary() {
|
|||||||
# it while the unit is stopped. The deploy is only green if the unit
|
# it while the unit is stopped. The deploy is only green if the unit
|
||||||
# itself is active with a live MainPID behind the health answer.
|
# itself is active with a live MainPID behind the health answer.
|
||||||
local main_pid
|
local main_pid
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-primary.service)"
|
main_pid="$(run_privileged systemctl show -p MainPID --value frank-archon.service)"
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
||||||
echo "FAIL: :7770 answered but frank-primary.service has no MainPID — a stray process holds the port" >&2
|
echo "FAIL: :7770 answered but frank-archon.service has no MainPID — a stray process holds the port" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
echo "frank-primary: health answered by unit MainPID $main_pid"
|
echo "frank-archon: health answered by unit MainPID $main_pid"
|
||||||
}
|
}
|
||||||
|
|
||||||
wait_for_health() {
|
wait_for_health() {
|
||||||
@@ -1573,9 +1573,9 @@ deploy_service() {
|
|||||||
anubis) deploy_anubis ;;
|
anubis) deploy_anubis ;;
|
||||||
seerr) deploy_seerr ;;
|
seerr) deploy_seerr ;;
|
||||||
marreta) deploy_marreta ;;
|
marreta) deploy_marreta ;;
|
||||||
frank-large) deploy_llm_large ;;
|
frank-dragoon) deploy_llm_dragoon ;;
|
||||||
frank-secondary) deploy_llm_secondary ;;
|
frank-zealot) deploy_llm_zealot ;;
|
||||||
frank-primary) deploy_llm_primary ;;
|
frank-archon) deploy_llm_archon ;;
|
||||||
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
|
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
|
||||||
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
|
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
|
||||||
jellyfin) deploy_jellyfin ;;
|
jellyfin) deploy_jellyfin ;;
|
||||||
|
|||||||
+3
-3
@@ -35,7 +35,7 @@ Options:
|
|||||||
Examples:
|
Examples:
|
||||||
./scripts/deploy.sh dankclaw
|
./scripts/deploy.sh dankclaw
|
||||||
./scripts/deploy.sh dankclaw --service searxng
|
./scripts/deploy.sh dankclaw --service searxng
|
||||||
./scripts/deploy.sh dankserver --service frank-primary
|
./scripts/deploy.sh dankserver --service frank-archon
|
||||||
./scripts/deploy.sh dankclaw --dry-run
|
./scripts/deploy.sh dankclaw --dry-run
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
@@ -141,12 +141,12 @@ valid_service() {
|
|||||||
dankclaw:alertmanager|\
|
dankclaw:alertmanager|\
|
||||||
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
||||||
dankclaw:frank-large|dankclaw:frank-secondary|\
|
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
||||||
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
|
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
|
||||||
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
|
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
|
||||||
dankserver:frank-primary|\
|
dankserver:frank-archon|\
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
||||||
frankbook-pro:readarr)
|
frankbook-pro:readarr)
|
||||||
|
|||||||
+4
-4
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary-swift15.sh — payload fetch/verify for the planned
|
# download-frank-archon-swift15.sh — payload fetch/verify for the planned
|
||||||
# frank-primary swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
|
# frank-archon swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
|
||||||
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
|
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
|
||||||
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
|
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
|
||||||
# template + licenses). Key set verified identical to the official
|
# template + licenses). Key set verified identical to the official
|
||||||
@@ -8,8 +8,8 @@
|
|||||||
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
|
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
|
||||||
# Repo is UNGATED — no token needed (never put one on the command line).
|
# Repo is UNGATED — no token needed (never put one on the command line).
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary-swift15.sh --fetch resume/complete
|
# bash scripts/download-frank-archon-swift15.sh --fetch resume/complete
|
||||||
# bash scripts/download-frank-primary-swift15.sh verify only
|
# bash scripts/download-frank-archon-swift15.sh verify only
|
||||||
#
|
#
|
||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary.sh — payload fetch/verify for frank-primary:
|
# download-frank-archon.sh — payload fetch/verify for frank-archon:
|
||||||
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
|
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
|
||||||
# shards + weight-map index + tokenizer + vision preprocessing configs,
|
# shards + weight-map index + tokenizer + vision preprocessing configs,
|
||||||
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
|
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
|
||||||
# FreeToken era (that script lived here; its repo was gated and needed
|
# FreeToken era (that script lived here; its repo was gated and needed
|
||||||
# HF_TOKEN — this one is UNGATED and needs no token at all).
|
# HF_TOKEN — this one is UNGATED and needs no token at all).
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary.sh --fetch resume/complete
|
# bash scripts/download-frank-archon.sh --fetch resume/complete
|
||||||
# bash scripts/download-frank-primary.sh verify only
|
# bash scripts/download-frank-archon.sh verify only
|
||||||
#
|
#
|
||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-large-flashnext.sh — fetch frank-large's flash-server v2
|
# download-frank-dragoon-flashnext.sh — fetch frank-dragoon's flash-server v2
|
||||||
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
|
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
|
||||||
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
# fetched fresh rather than symlinked, per the no-symlink law.
|
# fetched fresh rather than symlinked, per the no-symlink law.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
||||||
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
|
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
|
||||||
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
|
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
|
||||||
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
|
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-large-halogen.sh — fetch frank-large's halogen payload:
|
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
||||||
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
||||||
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
|
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
|
||||||
# secret law, none would ever touch argv).
|
# secret law, none would ever touch argv).
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# existing payload without downloading.
|
# existing payload without downloading.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
||||||
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
|
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
|
||||||
|
|
||||||
FILES=(qwen3.8-27b-p1w4d-d2.hgn
|
FILES=(qwen3.8-27b-p1w4d-d2.hgn
|
||||||
+3
-3
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
|
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
|
||||||
# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token
|
# frank-dragoon (Dankclaw iGPU). Public repo — no auth header, and never a token
|
||||||
# on the command line (it is world-readable in /proc/<pid>/cmdline).
|
# on the command line (it is world-readable in /proc/<pid>/cmdline).
|
||||||
#
|
#
|
||||||
# The payload is the repo's Q6 AGENT rung:
|
# The payload is the repo's Q6 AGENT rung:
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
|
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
|
||||||
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
|
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
|
||||||
#
|
#
|
||||||
# The other two files frank-large needs are NOT re-downloaded:
|
# The other two files frank-dragoon needs are NOT re-downloaded:
|
||||||
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
|
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
|
||||||
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
|
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
|
||||||
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
|
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
|
||||||
@@ -25,7 +25,7 @@
|
|||||||
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
|
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
|
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large
|
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-dragoon
|
||||||
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
|
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
|
||||||
# command line (it is world-readable in /proc/<pid>/cmdline).
|
# command line (it is world-readable in /proc/<pid>/cmdline).
|
||||||
#
|
#
|
||||||
@@ -24,7 +24,7 @@
|
|||||||
# come from the Hub LFS oids (api/models/<repo>/tree/main).
|
# come from the Hub LFS oids (api/models/<repo>/tree/main).
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-kingjones.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
|
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large
|
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-dragoon
|
||||||
# model) on Dankclaw.
|
# model) on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-orca.sh
|
# bash scripts/download-frank-dragoon-orca.sh
|
||||||
# bash scripts/download-frank-large-orca.sh --verify-only
|
# bash scripts/download-frank-dragoon-orca.sh --verify-only
|
||||||
#
|
#
|
||||||
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
|
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
|
||||||
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
|
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
|
||||||
@@ -105,4 +105,4 @@ fi
|
|||||||
echo "verifying assembled files against SHA256SUMS"
|
echo "verifying assembled files against SHA256SUMS"
|
||||||
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
|
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
|
||||||
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
|
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the Q4_K_XL frank-large payload on Dankclaw:
|
# Download the Q4_K_XL frank-dragoon payload on Dankclaw:
|
||||||
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
|
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-q4.sh
|
# bash scripts/download-frank-dragoon-q4.sh
|
||||||
# bash scripts/download-frank-large-q4.sh --verify-only
|
# bash scripts/download-frank-dragoon-q4.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: the file is checked against its declared byte count and sha256
|
# Integrity: the file is checked against its declared byte count and sha256
|
||||||
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
|
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
|
||||||
@@ -21,12 +21,12 @@
|
|||||||
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
|
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
|
||||||
#
|
#
|
||||||
# The speculative drafter is UNCHANGED and still fetched by its own script
|
# The speculative drafter is UNCHANGED and still fetched by its own script
|
||||||
# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not
|
# (scripts/download-frank-dragoon.sh, z-lab's Hub repo). That name must not
|
||||||
# appear here at all — the frank-large seam greps THIS file for it to prove
|
# appear here at all — the frank-dragoon seam greps THIS file for it to prove
|
||||||
# the drafter was not folded into the payload fetch.
|
# the drafter was not folded into the payload fetch.
|
||||||
#
|
#
|
||||||
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
|
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
|
||||||
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it —
|
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-dragoon does not need it —
|
||||||
# spec decode here is the block-diffusion drafter, not MTP.
|
# spec decode here is the block-diffusion drafter, not MTP.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the Swift-1.5 frank-large candidate on Dankclaw:
|
# Download the Swift-1.5 frank-dragoon candidate on Dankclaw:
|
||||||
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
||||||
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-swift.sh
|
# bash scripts/download-frank-dragoon-swift.sh
|
||||||
# bash scripts/download-frank-large-swift.sh --verify-only
|
# bash scripts/download-frank-dragoon-swift.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: the file is checked against its declared byte count and sha256
|
# Integrity: the file is checked against its declared byte count and sha256
|
||||||
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
|
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
|
||||||
@@ -15,9 +15,9 @@
|
|||||||
# Layout after a successful run (MODEL_DIR):
|
# Layout after a successful run (MODEL_DIR):
|
||||||
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
|
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
|
||||||
#
|
#
|
||||||
# This is the CANDIDATE payload, not the deployed one. frank-large still
|
# This is the CANDIDATE payload, not the deployed one. frank-dragoon still
|
||||||
# serves scripts/download-frank-large.sh's output; do not repoint
|
# serves scripts/download-frank-dragoon.sh's output; do not repoint
|
||||||
# frank-large.env at MODEL_DIR until the gufo load test passes.
|
# frank-dragoon.env at MODEL_DIR until the gufo load test passes.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
|
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
|
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
|
||||||
# frank-large model) on Dankclaw.
|
# frank-dragoon model) on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-unsloth.sh
|
# bash scripts/download-frank-dragoon-unsloth.sh
|
||||||
# bash scripts/download-frank-large-unsloth.sh --verify-only
|
# bash scripts/download-frank-dragoon-unsloth.sh --verify-only
|
||||||
#
|
#
|
||||||
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
|
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
|
||||||
# 50 GB). The parts ARE the canonical payload: the serve script points -m at
|
# 50 GB). The parts ARE the canonical payload: the serve script points -m at
|
||||||
@@ -112,4 +112,4 @@ if [[ "$mode" == "download" ]]; then
|
|||||||
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
|
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the frank-large payload on Dankclaw:
|
# Download the frank-dragoon payload on Dankclaw:
|
||||||
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
||||||
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large.sh
|
# bash scripts/download-frank-dragoon.sh
|
||||||
# bash scripts/download-frank-large.sh --verify-only
|
# bash scripts/download-frank-dragoon.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: every file is checked against its declared byte count and sha256
|
# Integrity: every file is checked against its declared byte count and sha256
|
||||||
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
||||||
@@ -86,4 +86,4 @@ if [[ "$mode" == "download" ]]; then
|
|||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
|
# Download the frank-zealot payload for llama.cpp on the RTX 3090:
|
||||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
||||||
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
||||||
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
||||||
@@ -21,7 +21,7 @@
|
|||||||
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST=""
|
DEST=""
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Measure per-block / per-expert expert-routing activity on live frank-primary.
|
"""Measure per-block / per-expert expert-routing activity on live frank-archon.
|
||||||
|
|
||||||
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
|
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
|
||||||
on the llama-server process while concurrent generations of mixed content run.
|
on the llama-server process while concurrent generations of mixed content run.
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
# frank-primary.env — radiance (StillDeadcode, Apache-2.0) serving the
|
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
|
||||||
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
|
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
|
||||||
# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
|
# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
|
||||||
# dankserver :7770.
|
# dankserver :7770.
|
||||||
#
|
#
|
||||||
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
||||||
# pinned host RAM; git tag frank-primary-freetoken-pr587-20261005 is the
|
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
|
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
|
||||||
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
|
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
|
||||||
# frank-primary-vllm-shapeG-20261003).
|
# frank-archon-vllm-shapeG-20261003).
|
||||||
#
|
#
|
||||||
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
|
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
|
||||||
# gfx1201, drives `podman run` directly from the digest-pinned
|
# gfx1201, drives `podman run` directly from the digest-pinned
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
||||||
# containers carry. Per-weight encodings are data, not the descriptor —
|
# containers carry. Per-weight encodings are data, not the descriptor —
|
||||||
# removing the key changes nothing about what the weights hold. Built by
|
# removing the key changes nothing about what the weights hold. Built by
|
||||||
# scripts/build-frank-primary-rad.sh (staging + convert + rad-info).
|
# scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
|
||||||
#
|
#
|
||||||
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
||||||
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
||||||
@@ -67,14 +67,14 @@
|
|||||||
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
|
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
|
||||||
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
|
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
|
||||||
# file already uses the pair, which BOTH pins accept (rollback symmetry).
|
# file already uses the pair, which BOTH pins accept (rollback symmetry).
|
||||||
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||||
# The .rad container FILE (mounted under /models) and the state dir for the
|
# The .rad container FILE (mounted under /models) and the state dir for the
|
||||||
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
||||||
# .rad files; the raw HF download and its staging dir live beside it and
|
# .rad files; the raw HF download and its staging dir live beside it and
|
||||||
# are never mounted.
|
# are never mounted.
|
||||||
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
||||||
FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||||
FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
|
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
||||||
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
||||||
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
||||||
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
|
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
|
||||||
@@ -84,48 +84,48 @@ FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
|
|||||||
# name across systemctl and podman ps. The runtime changes; the name does
|
# name across systemctl and podman ps. The runtime changes; the name does
|
||||||
# not. The legacy list retires every predecessor identity that could hold
|
# not. The legacy list retires every predecessor identity that could hold
|
||||||
# the GPUs or the port.
|
# the GPUs or the port.
|
||||||
FRANK_PRIMARY_CONTAINER=frank-primary
|
FRANK_ARCHON_CONTAINER=frank-archon
|
||||||
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
|
FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
|
||||||
# Served model id — Protoss name (wire rename 2026-10-09). The container,
|
# Served model id — Protoss name (wire rename 2026-10-09). The container,
|
||||||
# unit, scripts and this file keep the frank-primary name above; only the
|
# unit, scripts and this file keep the frank-archon name above; only the
|
||||||
# alias on the wire moved. Radiance validates the requested model name, so
|
# alias on the wire moved. Radiance validates the requested model name, so
|
||||||
# every client (harness, bench scripts, probes) MUST send this value.
|
# every client (harness, bench scripts, probes) MUST send this value.
|
||||||
FRANK_PRIMARY_SERVED_NAME=frank-archon
|
FRANK_ARCHON_SERVED_NAME=frank-archon
|
||||||
FRANK_PRIMARY_PORT=7770
|
FRANK_ARCHON_PORT=7770
|
||||||
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
|
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
|
||||||
# repo's root chat_template.jinja, vendored here; the checkpoint's own
|
# repo's root chat_template.jinja, vendored here; the checkpoint's own
|
||||||
# template is baked into the .rad by rad-convert and this overrides it).
|
# template is baked into the .rad by rad-convert and this overrides it).
|
||||||
# Radiance derives the reasoning/tool-call reply format from it.
|
# Radiance derives the reasoning/tool-call reply format from it.
|
||||||
FRANK_PRIMARY_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
|
FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
|
||||||
|
|
||||||
# --- engine shape (first-boot conservative; the boot ledger arbitrates) --
|
# --- engine shape (first-boot conservative; the boot ledger arbitrates) --
|
||||||
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
|
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
|
||||||
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
|
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
|
||||||
# wire is libr4d's own, not RCCL.
|
# wire is libr4d's own, not RCCL.
|
||||||
FRANK_PRIMARY_TP=2
|
FRANK_ARCHON_TP=2
|
||||||
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
|
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
|
||||||
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
||||||
# engine default.
|
# engine default.
|
||||||
FRANK_PRIMARY_TP_WIRE=wht6
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
|
# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
|
||||||
# start"); raise only after a real-traffic look at slot occupancy and the
|
# start"); raise only after a real-traffic look at slot occupancy and the
|
||||||
# KV pool's headroom line. The recurrent-state cost scales with it (54
|
# KV pool's headroom line. The recurrent-state cost scales with it (54
|
||||||
# MiB/copy/card, 2 copies with speculation).
|
# MiB/copy/card, 2 copies with speculation).
|
||||||
FRANK_PRIMARY_MAX_SEQS=3
|
FRANK_ARCHON_MAX_SEQS=3
|
||||||
# Served window. Native is 262,144; 200,000 is the publisher's production
|
# Served window. Native is 262,144; 200,000 is the publisher's production
|
||||||
# value (compose) and what models.yml's contextWindow must quote — a longer
|
# value (compose) and what models.yml's contextWindow must quote — a longer
|
||||||
# prompt is refused by the engine, never truncated.
|
# prompt is refused by the engine, never truncated.
|
||||||
FRANK_PRIMARY_MAX_MODEL_LEN=200000
|
FRANK_ARCHON_MAX_MODEL_LEN=200000
|
||||||
FRANK_PRIMARY_PLACEMENT=expert_tiered
|
FRANK_ARCHON_PLACEMENT=expert_tiered
|
||||||
# Pinned host pool for the expert rows that do not fit VRAM. Native FP8
|
# Pinned host pool for the expert rows that do not fit VRAM. Native FP8
|
||||||
# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
|
# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
|
||||||
# publisher shape's ~29 GiB — this pool is the cost of native precision.
|
# publisher shape's ~29 GiB — this pool is the cost of native precision.
|
||||||
# The boot ledger prints the pool's coverage ("the pool backs N% of the
|
# The boot ledger prints the pool's coverage ("the pool backs N% of the
|
||||||
# worst case ... M full-length sessions"); resize from that line, not from
|
# worst case ... M full-length sessions"); resize from that line, not from
|
||||||
# arithmetic.
|
# arithmetic.
|
||||||
FRANK_PRIMARY_HOST_POOL_MIB=40960
|
FRANK_ARCHON_HOST_POOL_MIB=40960
|
||||||
# VRAM left unclaimed per card (publisher default 96 MiB).
|
# VRAM left unclaimed per card (publisher default 96 MiB).
|
||||||
FRANK_PRIMARY_GPU_HEADROOM_MIB=96
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
||||||
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
|
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
|
||||||
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
|
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
|
||||||
@@ -133,36 +133,36 @@ FRANK_PRIMARY_GPU_HEADROOM_MIB=96
|
|||||||
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
|
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
|
||||||
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
|
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
|
||||||
# boot ledger arbitrates; re-derive after any payload change.
|
# boot ledger arbitrates; re-derive after any payload change.
|
||||||
FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071
|
FRANK_ARCHON_VRAM_WEIGHTS_MIB=26071
|
||||||
FRANK_PRIMARY_VRAM_KV_MIB=3543
|
FRANK_ARCHON_VRAM_KV_MIB=3543
|
||||||
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
|
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
|
||||||
# 2 KV heads, 256 head_dim, K+V).
|
# 2 KV heads, 256 head_dim, K+V).
|
||||||
FRANK_PRIMARY_KV_DTYPE=fp8
|
FRANK_ARCHON_KV_DTYPE=fp8
|
||||||
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
||||||
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
||||||
# flag states it so the ledger records the intent).
|
# flag states it so the ledger records the intent).
|
||||||
FRANK_PRIMARY_SPEC_TOKENS=2
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
||||||
FRANK_PRIMARY_PREFIX_HOST_MIB=4096
|
FRANK_ARCHON_PREFIX_HOST_MIB=4096
|
||||||
FRANK_PRIMARY_PREFIX_DISK_MIB=65536
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
||||||
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
|
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
|
||||||
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
|
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
|
||||||
FRANK_PRIMARY_NGRAM_PLACEMENT=ram
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||||
# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
||||||
# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
|
# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
|
||||||
# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
|
# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
|
||||||
# plan, so the serve script refuses to start until both are raised. This is
|
# plan, so the serve script refuses to start until both are raised. This is
|
||||||
# the floor it enforces (in MiB), with ~8 GiB of margin.
|
# the floor it enforces (in MiB), with ~8 GiB of margin.
|
||||||
FRANK_PRIMARY_TTM_MIN_MIB=140000
|
FRANK_ARCHON_TTM_MIN_MIB=140000
|
||||||
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
||||||
# multi-flag values as one string; the serve script expands it unquoted on
|
# multi-flag values as one string; the serve script expands it unquoted on
|
||||||
# purpose.
|
# purpose.
|
||||||
FRANK_PRIMARY_EXTRA_ARGS=
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
||||||
# guaranteed long-timeout boot instead of a fast failure). The converted
|
# guaranteed long-timeout boot instead of a fast failure). The converted
|
||||||
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
||||||
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
||||||
# tail) - floor sits just under the payload.
|
# tail) - floor sits just under the payload.
|
||||||
FRANK_PRIMARY_RAD_MIN_MIB=110000
|
FRANK_ARCHON_RAD_MIN_MIB=110000
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# frank-primary.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
||||||
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
||||||
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
||||||
# frank-primary-freetoken-pr587-20261005 is the pre-cutover state; the
|
# frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
|
||||||
# retired unit/env/serve trio and Dockerfile are in that tag).
|
# retired unit/env/serve trio and Dockerfile are in that tag).
|
||||||
#
|
#
|
||||||
# Same port, same served alias, same container identity as the unit name;
|
# Same port, same served alias, same container identity as the unit name;
|
||||||
@@ -19,7 +19,7 @@
|
|||||||
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
||||||
# engine still runs out of dank's rootless podman storage.
|
# engine still runs out of dank's rootless podman storage.
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=frank-primary — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
||||||
@@ -33,14 +33,14 @@ Group=dank
|
|||||||
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
|
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
|
||||||
# PID1 grants infinity here — parity with the fleet's other LLM units.
|
# PID1 grants infinity here — parity with the fleet's other LLM units.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
|
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
|
||||||
# systemd stop: wait alone cannot stop the container child; stop it
|
# systemd stop: wait alone cannot stop the container child; stop it
|
||||||
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
||||||
# holds no queue to drain — in-flight decodes are lost, which a stop
|
# holds no queue to drain — in-flight decodes are lost, which a stop
|
||||||
# during a bench window accepts; radiance takes SIGTERM on the engine
|
# during a bench window accepts; radiance takes SIGTERM on the engine
|
||||||
# process and podman reaps the container).
|
# process and podman reaps the container).
|
||||||
ExecStop=/usr/bin/podman stop -t 120 frank-primary
|
ExecStop=/usr/bin/podman stop -t 120 frank-archon
|
||||||
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
|
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
|
||||||
# publisher's compose healthcheck allows a 15-minute start_period; the
|
# publisher's compose healthcheck allows a 15-minute start_period; the
|
||||||
# vLLM-era budget (1800 s) carried forward.
|
# vLLM-era budget (1800 s) carried forward.
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
# frank-large.env — tunables for the frank-large endpoint on Dankclaw's
|
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
||||||
# Strix Halo iGPU (gfx1151), host port 7770.
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
||||||
#
|
#
|
||||||
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
|
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
|
||||||
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
|
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
|
||||||
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
|
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
|
||||||
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
|
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
|
||||||
# a root unit, driven by scripts/serve-frank-large.sh. This CUTS OVER the
|
# a root unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
|
||||||
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
|
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
|
||||||
# rollback = revert the swap commit's trio; the 27B payload stays on disk at
|
# rollback = revert the swap commit's trio; the 27B payload stays on disk at
|
||||||
# /home/dank/models/halogen-qwen3.8-27b/.
|
# /home/dank/models/halogen-qwen3.8-27b/.
|
||||||
@@ -36,7 +36,7 @@
|
|||||||
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
||||||
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
||||||
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
||||||
# 10 % under-count). Compaction rides on frank-secondary: input ≈
|
# 10 % under-count). Compaction rides on frank-zealot: input ≈
|
||||||
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
||||||
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
||||||
# form of the engine's default, so the seam test can pin the product.
|
# form of the engine's default, so the seam test can pin the product.
|
||||||
@@ -52,7 +52,7 @@
|
|||||||
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||||
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
||||||
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||||
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ)
|
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
|
||||||
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
||||||
# after deploy and re-size if the engine prints a clamp.
|
# after deploy and re-size if the engine prints a clamp.
|
||||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||||
@@ -91,47 +91,47 @@
|
|||||||
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
||||||
# is what was qualified). 0.16.2 = the version the model card names as its
|
# is what was qualified). 0.16.2 = the version the model card names as its
|
||||||
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
||||||
FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||||
# Container identity: kept from the 27B era (unit ExecStop names it).
|
# Container identity: kept from the 27B era (unit ExecStop names it).
|
||||||
FRANK_LARGE_CONTAINER=frank-large-halogen
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
||||||
# Old quadlet-era container identity; the serve script clears stale names.
|
# Old quadlet-era container identity; the serve script clears stale names.
|
||||||
FRANK_LARGE_LEGACY_CONTAINERS=frank-large
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
||||||
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
|
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
|
||||||
# default); the engine port (8730) is never published.
|
# default); the engine port (8730) is never published.
|
||||||
FRANK_LARGE_PUBLISH=7770:8731
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
||||||
|
|
||||||
FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
|
||||||
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
|
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
|
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
|
||||||
# append-only original (byte-identical live tensors, padding dropped:
|
# append-only original (byte-identical live tensors, padding dropped:
|
||||||
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
|
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
|
||||||
FRANK_LARGE_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
|
||||||
FRANK_LARGE_CHECKPOINT_BYTES=66687678464
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678464
|
||||||
FRANK_LARGE_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
|
FRANK_DRAGOON_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
|
||||||
# The n-gram lookup table: REQUIRED (the engine reads it beside the
|
# The n-gram lookup table: REQUIRED (the engine reads it beside the
|
||||||
# checkpoint; the card's quickstart names it explicitly). From
|
# checkpoint; the card's quickstart names it explicitly). From
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
|
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
|
||||||
FRANK_LARGE_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
||||||
FRANK_LARGE_NGRAM_BYTES=51200246144
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
||||||
FRANK_LARGE_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
||||||
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
|
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
|
||||||
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
|
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
|
||||||
FRANK_LARGE_VISION_TOWER=qwen38-flash-next-vision.hgn
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
||||||
FRANK_LARGE_VISION_BYTES=897916416
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
||||||
FRANK_LARGE_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
||||||
# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer,
|
# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
|
||||||
# mounted whole at /tokenizer. The two repos ship the same one; fetched
|
# mounted whole at /tokenizer. The two repos ship the same one; fetched
|
||||||
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
|
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
|
||||||
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
|
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
|
||||||
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
|
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
|
||||||
# cache snapshot symlinks each file into blobs/, and those links dangle
|
# cache snapshot symlinks each file into blobs/, and those links dangle
|
||||||
# inside the container mount.
|
# inside the container mount.
|
||||||
FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
||||||
|
|
||||||
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
||||||
# stays at the image default, which is what was qualified) ------------------
|
# stays at the image default, which is what was qualified) ------------------
|
||||||
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
||||||
# scripts and this file keep the frank-large name. OpenWebUI api_configs
|
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
|
||||||
# model_ids for the :7770 backend MUST name this value.
|
# model_ids for the :7770 backend MUST name this value.
|
||||||
HALOGEN_MODEL_ID=frank-dragoon
|
HALOGEN_MODEL_ID=frank-dragoon
|
||||||
HALOGEN_CTX=262144
|
HALOGEN_CTX=262144
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
|
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -11,11 +11,11 @@ Group=dank
|
|||||||
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
||||||
# other LLM units on this fleet.
|
# other LLM units on this fleet.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
# systemd stop alone cannot stop the container child; stop it explicitly
|
# systemd stop alone cannot stop the container child; stop it explicitly
|
||||||
# (frank-primary pattern). The engine drains on SIGTERM.
|
# (frank-archon pattern). The engine drains on SIGTERM.
|
||||||
ExecStop=/usr/bin/podman stop -t 60 frank-large-halogen
|
ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
|
||||||
TimeoutStartSec=1200
|
TimeoutStartSec=1200
|
||||||
TimeoutStopSec=120
|
TimeoutStopSec=120
|
||||||
# always, not on-failure: an engine that exits 0 after an internal fault
|
# always, not on-failure: an engine that exits 0 after an internal fault
|
||||||
@@ -23,7 +23,7 @@ TimeoutStopSec=120
|
|||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
||||||
# on a box that also hosts frank-secondary, ComfyUI and the Caddy/observability
|
# on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
|
||||||
# stack; make it a late OOM candidate, not the first.
|
# stack; make it a late OOM candidate, not the first.
|
||||||
OOMScoreAdjust=-500
|
OOMScoreAdjust=-500
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# frank-secondary — llama.cpp (master, CUDA sm_86) serving
|
# frank-zealot — llama.cpp (master, CUDA sm_86) serving
|
||||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
||||||
# :7771.
|
# :7771.
|
||||||
#
|
#
|
||||||
@@ -38,7 +38,7 @@ MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensore
|
|||||||
# Served model id — the harness/family name (Protoss rename completed to the
|
# Served model id — the harness/family name (Protoss rename completed to the
|
||||||
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
||||||
# this day). The SERVICE, unit, scripts and this env file keep the
|
# this day). The SERVICE, unit, scripts and this env file keep the
|
||||||
# frank-secondary name; only the alias on the wire changed. Consumers that
|
# frank-zealot name; only the alias on the wire changed. Consumers that
|
||||||
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
||||||
# model_ids) and the harness models.yml.
|
# model_ids) and the harness models.yml.
|
||||||
ALIAS=frank-zealot
|
ALIAS=frank-zealot
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
|
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -10,8 +10,8 @@ Group=dank
|
|||||||
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
||||||
# though the weights themselves are copied into VRAM.
|
# though the weights themselves are copied into VRAM.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_SECONDARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-secondary.env"
|
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-secondary.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
@@ -24,7 +24,7 @@ KLD_OUT=/home/dank/radiance-state/kld-gate.json
|
|||||||
: > "$KLD_OUT"
|
: > "$KLD_OUT"
|
||||||
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
||||||
/home/dank/models/tpl.jinja 2>/dev/null || true
|
/home/dank/models/tpl.jinja 2>/dev/null || true
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
podman run --rm --name "$C" \
|
podman run --rm --name "$C" \
|
||||||
|
|||||||
@@ -44,17 +44,17 @@ JSONL_EXTS = (".jsonl", ".json")
|
|||||||
DOC_DIR = "serverAppConfigs-docs"
|
DOC_DIR = "serverAppConfigs-docs"
|
||||||
|
|
||||||
FLEET_CMDS = [
|
FLEET_CMDS = [
|
||||||
"systemctl status frank-primary --no-pager",
|
"systemctl status frank-archon --no-pager",
|
||||||
"journalctl -u frank-primary -n 100 --no-pager",
|
"journalctl -u frank-archon -n 100 --no-pager",
|
||||||
"podman ps --format '{{.Names}} {{.Status}}'",
|
"podman ps --format '{{.Names}} {{.Status}}'",
|
||||||
"podman logs --tail 50 frank-primary",
|
"podman logs --tail 50 frank-archon",
|
||||||
"curl -s http://192.168.0.235:7770/health",
|
"curl -s http://192.168.0.235:7770/health",
|
||||||
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
|
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
|
||||||
"df -h /home/dank | tail -1",
|
"df -h /home/dank | tail -1",
|
||||||
"free -g",
|
"free -g",
|
||||||
"sudo systemctl restart frank-primary",
|
"sudo systemctl restart frank-archon",
|
||||||
"cd /home/dank/serverAppConfigs && git pull --ff-only",
|
"cd /home/dank/serverAppConfigs && git pull --ff-only",
|
||||||
"./scripts/deploy.sh dankserver --service frank-primary",
|
"./scripts/deploy.sh dankserver --service frank-archon",
|
||||||
"./tests/run.sh 2>&1 | tail -5",
|
"./tests/run.sh 2>&1 | tail -5",
|
||||||
"jq . /home/dank/models/rad/index.json | head -40",
|
"jq . /home/dank/models/rad/index.json | head -40",
|
||||||
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
|
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
#
|
#
|
||||||
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
|
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
|
||||||
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
|
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
|
||||||
# the deployed frank-primary shape. Nothing in scripts/ mounts these files.
|
# the deployed frank-archon shape. Nothing in scripts/ mounts these files.
|
||||||
#
|
#
|
||||||
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
|
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
|
||||||
#
|
#
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
|
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
|
||||||
#
|
#
|
||||||
# STATUS: NOT part of the deployed frank-primary shape. It exists because the
|
# STATUS: NOT part of the deployed frank-archon shape. It exists because the
|
||||||
# fork's plan under-models this model's KV, and it is the enabler for a future
|
# fork's plan under-models this model's KV, and it is the enabler for a future
|
||||||
# MTP attempt (the production GLM shape runs without MTP and boots fine on the
|
# MTP attempt (the production GLM shape runs without MTP and boots fine on the
|
||||||
# plan's own reserve).
|
# plan's own reserve).
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
||||||
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
||||||
# base) as a radiance container for frank-primary: EVERY routed expert -- gate_up AND down --
|
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
||||||
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
||||||
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
||||||
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
||||||
|
|||||||
@@ -30,9 +30,9 @@ KLD_REF_DIR=/home/dank/corpora/kld-ref
|
|||||||
CALIB_DIR=/home/dank/corpora/calib
|
CALIB_DIR=/home/dank/corpora/calib
|
||||||
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
|
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
|
||||||
|
|
||||||
restore() { sudo systemctl start frank-primary >/dev/null 2>&1 || true; }
|
restore() { sudo systemctl start frank-archon >/dev/null 2>&1 || true; }
|
||||||
trap restore EXIT
|
trap restore EXIT
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# serve-frank-primary.sh — frank-primary endpoint: radiance (StillDeadcode,
|
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
||||||
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
||||||
# NATIVE block-FP8 precision from a rad-convert container
|
# NATIVE block-FP8 precision from a rad-convert container
|
||||||
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
||||||
# one card; git tag frank-primary-freetoken-pr587-20261005 is the
|
# one card; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
# pre-cutover state).
|
# pre-cutover state).
|
||||||
#
|
#
|
||||||
# The run line below IS the engine argv: the image's entrypoint is the
|
# The run line below IS the engine argv: the image's entrypoint is the
|
||||||
@@ -16,18 +16,18 @@
|
|||||||
# 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV,
|
# 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV,
|
||||||
# MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM.
|
# MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM.
|
||||||
#
|
#
|
||||||
# The root unit (scripts/frank-primary.service, User=dank,
|
# The root unit (scripts/frank-archon.service, User=dank,
|
||||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools +
|
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools +
|
||||||
# the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned
|
# the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned
|
||||||
# pages, which is ALSO why this script refuses to start when ttm's
|
# pages, which is ALSO why this script refuses to start when ttm's
|
||||||
# pages_limit/pages_max cap is under FRANK_PRIMARY_TTM_MIN_MIB (see below).
|
# pages_limit/pages_max cap is under FRANK_ARCHON_TTM_MIN_MIB (see below).
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}"
|
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
|
||||||
|
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
echo "ERROR: frank-primary env file absent: $ENV_FILE" >&2
|
echo "ERROR: frank-archon env file absent: $ENV_FILE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
set -a
|
set -a
|
||||||
@@ -35,74 +35,74 @@ set -a
|
|||||||
source "$ENV_FILE"
|
source "$ENV_FILE"
|
||||||
set +a
|
set +a
|
||||||
|
|
||||||
: "${FRANK_PRIMARY_IMAGE:?FRANK_PRIMARY_IMAGE must name the radiance image}"
|
: "${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
|
||||||
: "${FRANK_PRIMARY_MODEL_DIR:?FRANK_PRIMARY_MODEL_DIR must name the .rad dir}"
|
: "${FRANK_ARCHON_MODEL_DIR:?FRANK_ARCHON_MODEL_DIR must name the .rad dir}"
|
||||||
: "${FRANK_PRIMARY_MODEL_FILE:?FRANK_PRIMARY_MODEL_FILE must name the .rad file}"
|
: "${FRANK_ARCHON_MODEL_FILE:?FRANK_ARCHON_MODEL_FILE must name the .rad file}"
|
||||||
: "${FRANK_PRIMARY_STATE_DIR:?FRANK_PRIMARY_STATE_DIR must name the /data state dir}"
|
: "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the /data state dir}"
|
||||||
: "${FRANK_PRIMARY_CONTAINER:?FRANK_PRIMARY_CONTAINER must name the container}"
|
: "${FRANK_ARCHON_CONTAINER:?FRANK_ARCHON_CONTAINER must name the container}"
|
||||||
: "${FRANK_PRIMARY_SERVED_NAME:?FRANK_PRIMARY_SERVED_NAME must name the alias}"
|
: "${FRANK_ARCHON_SERVED_NAME:?FRANK_ARCHON_SERVED_NAME must name the alias}"
|
||||||
: "${FRANK_PRIMARY_PORT:?FRANK_PRIMARY_PORT must be the port}"
|
: "${FRANK_ARCHON_PORT:?FRANK_ARCHON_PORT must be the port}"
|
||||||
: "${FRANK_PRIMARY_TEMPLATE:?FRANK_PRIMARY_TEMPLATE must name the chat template}"
|
: "${FRANK_ARCHON_TEMPLATE:?FRANK_ARCHON_TEMPLATE must name the chat template}"
|
||||||
: "${FRANK_PRIMARY_TP:?FRANK_PRIMARY_TP must be set}"
|
: "${FRANK_ARCHON_TP:?FRANK_ARCHON_TP must be set}"
|
||||||
: "${FRANK_PRIMARY_TP_WIRE:?FRANK_PRIMARY_TP_WIRE must be set}"
|
: "${FRANK_ARCHON_TP_WIRE:?FRANK_ARCHON_TP_WIRE must be set}"
|
||||||
: "${FRANK_PRIMARY_MAX_SEQS:?FRANK_PRIMARY_MAX_SEQS must be set}"
|
: "${FRANK_ARCHON_MAX_SEQS:?FRANK_ARCHON_MAX_SEQS must be set}"
|
||||||
: "${FRANK_PRIMARY_MAX_MODEL_LEN:?FRANK_PRIMARY_MAX_MODEL_LEN must be set}"
|
: "${FRANK_ARCHON_MAX_MODEL_LEN:?FRANK_ARCHON_MAX_MODEL_LEN must be set}"
|
||||||
: "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
|
||||||
: "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}"
|
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}"
|
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_VRAM_WEIGHTS_MIB:?FRANK_PRIMARY_VRAM_WEIGHTS_MIB must be set}"
|
: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_VRAM_KV_MIB:?FRANK_PRIMARY_VRAM_KV_MIB must be set}"
|
: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}"
|
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
|
||||||
: "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}"
|
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
|
||||||
: "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}"
|
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
||||||
: "${FRANK_PRIMARY_PREFIX_HOST_MIB:?FRANK_PRIMARY_PREFIX_HOST_MIB must be set}"
|
: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_PREFIX_DISK_MIB:?FRANK_PRIMARY_PREFIX_DISK_MIB must be set}"
|
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_NGRAM_PLACEMENT:?FRANK_PRIMARY_NGRAM_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
||||||
: "${FRANK_PRIMARY_TTM_MIN_MIB:?FRANK_PRIMARY_TTM_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
||||||
: "${FRANK_PRIMARY_RAD_MIN_MIB:?FRANK_PRIMARY_RAD_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
||||||
|
|
||||||
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
||||||
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
||||||
# newer latest does not satisfy it. `podman image exists` takes a
|
# newer latest does not satisfy it. `podman image exists` takes a
|
||||||
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
|
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
|
||||||
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
|
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
|
||||||
# storage (root unit, User=dank — the fleet's frank-primary shape).
|
# storage (root unit, User=dank — the fleet's frank-archon shape).
|
||||||
if ! podman image exists "$FRANK_PRIMARY_IMAGE"; then
|
if ! podman image exists "$FRANK_ARCHON_IMAGE"; then
|
||||||
echo "ERROR: radiance image absent: $FRANK_PRIMARY_IMAGE" >&2
|
echo "ERROR: radiance image absent: $FRANK_ARCHON_IMAGE" >&2
|
||||||
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
|
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# The payload gate: the .rad container file, sized. A truncated convert
|
# The payload gate: the .rad container file, sized. A truncated convert
|
||||||
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
||||||
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
||||||
# (scripts/build-frank-primary-rad.sh built the file; its rad-info output
|
# (scripts/build-frank-archon-rad.sh built the file; its rad-info output
|
||||||
# is the record).
|
# is the record).
|
||||||
RAD="$FRANK_PRIMARY_MODEL_DIR/$FRANK_PRIMARY_MODEL_FILE"
|
RAD="$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE"
|
||||||
if [ ! -f "$RAD" ]; then
|
if [ ! -f "$RAD" ]; then
|
||||||
echo "ERROR: .rad container missing: $RAD" >&2
|
echo "ERROR: .rad container missing: $RAD" >&2
|
||||||
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
|
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
|
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
|
||||||
if [ "$RAD_MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
|
if [ "$RAD_MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
|
||||||
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_PRIMARY_RAD_MIN_MIB" >&2
|
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_ARCHON_RAD_MIN_MIB" >&2
|
||||||
echo "frank-primary: the convert was truncated — rebuild it" >&2
|
echo "frank-archon: the convert was truncated — rebuild it" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if [ ! -f "$FRANK_PRIMARY_TEMPLATE" ]; then
|
if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
|
||||||
echo "ERROR: chat template missing: $FRANK_PRIMARY_TEMPLATE" >&2
|
echo "ERROR: chat template missing: $FRANK_ARCHON_TEMPLATE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/kvcache/frank-primary"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
|
||||||
# The engine's create is NOT recursive: with only the state dir mounted, its
|
# The engine's create is NOT recursive: with only the state dir mounted, its
|
||||||
# "cannot create /data/kvcache/frank-primary: No such file or directory"
|
# "cannot create /data/kvcache/frank-archon: No such file or directory"
|
||||||
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
|
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
|
||||||
# the full path on the host so the tier opens on first boot.
|
# the full path on the host so the tier opens on first boot.
|
||||||
|
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
||||||
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
||||||
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
||||||
@@ -118,7 +118,7 @@ fi
|
|||||||
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
||||||
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
||||||
# takes the min over the knobs that exist). Below FRANK_PRIMARY_TTM_MIN_MIB
|
# takes the min over the knobs that exist). Below FRANK_ARCHON_TTM_MIN_MIB
|
||||||
# the load dies mid-pin with an allocator error instead of this named
|
# the load dies mid-pin with an allocator error instead of this named
|
||||||
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
||||||
# fallback + one reboot only if the file is read-only), see the runbook.
|
# fallback + one reboot only if the file is read-only), see the runbook.
|
||||||
@@ -138,22 +138,22 @@ ttm_cap_mib() {
|
|||||||
echo "${cur:-0}"
|
echo "${cur:-0}"
|
||||||
}
|
}
|
||||||
TTM_CAP="$(ttm_cap_mib)"
|
TTM_CAP="$(ttm_cap_mib)"
|
||||||
if [ "$TTM_CAP" -lt "$FRANK_PRIMARY_TTM_MIN_MIB" ]; then
|
if [ "$TTM_CAP" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_PRIMARY_TTM_MIN_MIB" >&2
|
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
echo "frank-primary: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_PRIMARY_TTM_MIN_MIB in 4 KiB pages" >&2
|
echo "frank-archon: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_ARCHON_TTM_MIN_MIB in 4 KiB pages" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# A crashed/killed unit can leave the container name registered (systemd
|
# A crashed/killed unit can leave the container name registered (systemd
|
||||||
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
||||||
# identities (the FreeToken-era frank-primary-qwen, the retired R9V
|
# identities (the FreeToken-era frank-archon-qwen, the retired R9V
|
||||||
# container and the ad hoc experiment containers all hold this GPU); all
|
# container and the ad hoc experiment containers all hold this GPU); all
|
||||||
# idempotent.
|
# idempotent.
|
||||||
for CTR in $FRANK_PRIMARY_LEGACY_CONTAINERS "$FRANK_PRIMARY_CONTAINER"; do
|
for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
|
||||||
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
|
|
||||||
LOG=/home/dank/logs/frank_primary.log
|
LOG=/home/dank/logs/frank_archon.log
|
||||||
mkdir -p "$(dirname "$LOG")"
|
mkdir -p "$(dirname "$LOG")"
|
||||||
|
|
||||||
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
||||||
@@ -166,41 +166,41 @@ mkdir -p "$(dirname "$LOG")"
|
|||||||
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
||||||
# The template rides a single-file mount; the .rad dir is ro; /data holds
|
# The template rides a single-file mount; the .rad dir is ro; /data holds
|
||||||
# the prefix-cache disk tier.
|
# the prefix-cache disk tier.
|
||||||
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
||||||
--network host \
|
--network host \
|
||||||
--init \
|
--init \
|
||||||
--security-opt seccomp=unconfined \
|
--security-opt seccomp=unconfined \
|
||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device /dev/kfd --device /dev/dri \
|
--device /dev/kfd --device /dev/dri \
|
||||||
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR":/data \
|
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
"$FRANK_PRIMARY_IMAGE" \
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
|
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
--host :: --port "$FRANK_PRIMARY_PORT" \
|
--host :: --port "$FRANK_ARCHON_PORT" \
|
||||||
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
|
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
|
||||||
--override-chat-template /templates/template.jinja \
|
--override-chat-template /templates/template.jinja \
|
||||||
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
|
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
|
||||||
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
|
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
|
||||||
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
|
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
|
||||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
||||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
||||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
||||||
--prefix-cache-dir /data/kvcache/frank-primary \
|
--prefix-cache-dir /data/kvcache/frank-archon \
|
||||||
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
|
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
||||||
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
|
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
|
|
||||||
# From here on, this script's lifetime IS the container's: wait returns the
|
# From here on, this script's lifetime IS the container's: wait returns the
|
||||||
# container's exit code, so a crash is a nonzero ExecStart and the unit's
|
# container's exit code, so a crash is a nonzero ExecStart and the unit's
|
||||||
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
|
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
|
||||||
# podman wait alone cannot stop the container child.)
|
# podman wait alone cannot stop the container child.)
|
||||||
exec podman wait "$FRANK_PRIMARY_CONTAINER"
|
exec podman wait "$FRANK_ARCHON_CONTAINER"
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# serve-frank-large.sh — frank-large endpoint: halogen-flash-server 0.16.2
|
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
|
||||||
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
|
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
|
||||||
# digest-pinned in frank-large.env) serving the Quat3rnion abliterated
|
# digest-pinned in frank-dragoon.env) serving the Quat3rnion abliterated
|
||||||
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
|
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
|
||||||
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
|
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
|
||||||
# halogen 27B-dense shape — revert the swap commit's trio to roll back
|
# halogen 27B-dense shape — revert the swap commit's trio to roll back
|
||||||
# (the 27B payload stays on disk).
|
# (the 27B payload stays on disk).
|
||||||
#
|
#
|
||||||
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-primary
|
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon
|
||||||
# pattern): the image starts engine + OpenAI frontend in one container.
|
# pattern): the image starts engine + OpenAI frontend in one container.
|
||||||
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
|
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
|
||||||
# published as host 7770; the engine's own token port (8730, no auth)
|
# published as host 7770; the engine's own token port (8730, no auth)
|
||||||
@@ -26,13 +26,13 @@
|
|||||||
# Why containers instead of a bare binary: the payload is .hgn and only
|
# Why containers instead of a bare binary: the payload is .hgn and only
|
||||||
# the flash-server line reads it; it ships as a container; the root unit
|
# the flash-server line reads it; it ships as a container; the root unit
|
||||||
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
|
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
|
||||||
# exactly like frank-primary's.
|
# exactly like frank-archon's.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
echo "frank-large: env file not found: $ENV_FILE" >&2
|
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
set -a
|
set -a
|
||||||
@@ -40,17 +40,17 @@ set -a
|
|||||||
source "$ENV_FILE"
|
source "$ENV_FILE"
|
||||||
set +a
|
set +a
|
||||||
|
|
||||||
: "${FRANK_LARGE_IMAGE:?FRANK_LARGE_IMAGE must name the digest-pinned image}"
|
: "${FRANK_DRAGOON_IMAGE:?FRANK_DRAGOON_IMAGE must name the digest-pinned image}"
|
||||||
: "${FRANK_LARGE_CONTAINER:?FRANK_LARGE_CONTAINER must name the container}"
|
: "${FRANK_DRAGOON_CONTAINER:?FRANK_DRAGOON_CONTAINER must name the container}"
|
||||||
: "${FRANK_LARGE_PUBLISH:?FRANK_LARGE_PUBLISH must be host:container}"
|
: "${FRANK_DRAGOON_PUBLISH:?FRANK_DRAGOON_PUBLISH must be host:container}"
|
||||||
: "${FRANK_LARGE_MODEL_DIR:?FRANK_LARGE_MODEL_DIR must name the payload dir}"
|
: "${FRANK_DRAGOON_MODEL_DIR:?FRANK_DRAGOON_MODEL_DIR must name the payload dir}"
|
||||||
: "${FRANK_LARGE_CHECKPOINT:?FRANK_LARGE_CHECKPOINT must name the .hgn inside it}"
|
: "${FRANK_DRAGOON_CHECKPOINT:?FRANK_DRAGOON_CHECKPOINT must name the .hgn inside it}"
|
||||||
: "${FRANK_LARGE_CHECKPOINT_BYTES:?FRANK_LARGE_CHECKPOINT_BYTES must pin the size}"
|
: "${FRANK_DRAGOON_CHECKPOINT_BYTES:?FRANK_DRAGOON_CHECKPOINT_BYTES must pin the size}"
|
||||||
: "${FRANK_LARGE_NGRAM_TABLE:?FRANK_LARGE_NGRAM_TABLE must name the lookup table}"
|
: "${FRANK_DRAGOON_NGRAM_TABLE:?FRANK_DRAGOON_NGRAM_TABLE must name the lookup table}"
|
||||||
: "${FRANK_LARGE_NGRAM_BYTES:?FRANK_LARGE_NGRAM_BYTES must pin the table size}"
|
: "${FRANK_DRAGOON_NGRAM_BYTES:?FRANK_DRAGOON_NGRAM_BYTES must pin the table size}"
|
||||||
: "${FRANK_LARGE_VISION_TOWER:?FRANK_LARGE_VISION_TOWER must name the tower}"
|
: "${FRANK_DRAGOON_VISION_TOWER:?FRANK_DRAGOON_VISION_TOWER must name the tower}"
|
||||||
: "${FRANK_LARGE_VISION_BYTES:?FRANK_LARGE_VISION_BYTES must pin the tower size}"
|
: "${FRANK_DRAGOON_VISION_BYTES:?FRANK_DRAGOON_VISION_BYTES must pin the tower size}"
|
||||||
: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}"
|
: "${FRANK_DRAGOON_TOKENIZER_FILES:?FRANK_DRAGOON_TOKENIZER_FILES must list the tokenizer dir contents}"
|
||||||
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
|
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
|
||||||
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
|
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
|
||||||
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
|
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
|
||||||
@@ -62,9 +62,9 @@ set +a
|
|||||||
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
|
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
|
||||||
# 2026-09-30 deploy refusal).
|
# 2026-09-30 deploy refusal).
|
||||||
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
|
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
|
||||||
if ! grep -qF "${FRANK_LARGE_IMAGE#*@}" <<<"$DIGESTS"; then
|
if ! grep -qF "${FRANK_DRAGOON_IMAGE#*@}" <<<"$DIGESTS"; then
|
||||||
echo "frank-large: image $FRANK_LARGE_IMAGE not in podman storage;" >&2
|
echo "frank-dragoon: image $FRANK_DRAGOON_IMAGE not in podman storage;" >&2
|
||||||
echo " pull it: podman pull ${FRANK_LARGE_IMAGE%@*}" >&2
|
echo " pull it: podman pull ${FRANK_DRAGOON_IMAGE%@*}" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -74,45 +74,45 @@ fi
|
|||||||
ck() {
|
ck() {
|
||||||
# ck <env-name> <path> <expected-bytes>
|
# ck <env-name> <path> <expected-bytes>
|
||||||
if [ ! -f "$2" ]; then
|
if [ ! -f "$2" ]; then
|
||||||
echo "frank-large: $1 $2 missing;" >&2
|
echo "frank-dragoon: $1 $2 missing;" >&2
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
local sz
|
local sz
|
||||||
sz="$(stat -c %s "$2")"
|
sz="$(stat -c %s "$2")"
|
||||||
if [ "$sz" != "$3" ]; then
|
if [ "$sz" != "$3" ]; then
|
||||||
echo "frank-large: $1 is $sz bytes, expected $3 (partial download?)" >&2
|
echo "frank-dragoon: $1 is $sz bytes, expected $3 (partial download?)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
ck checkpoint "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_CHECKPOINT" "$FRANK_LARGE_CHECKPOINT_BYTES"
|
ck checkpoint "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" "$FRANK_DRAGOON_CHECKPOINT_BYTES"
|
||||||
ck n-gram-table "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_NGRAM_TABLE" "$FRANK_LARGE_NGRAM_BYTES"
|
ck n-gram-table "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" "$FRANK_DRAGOON_NGRAM_BYTES"
|
||||||
ck vision-tower "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_VISION_TOWER" "$FRANK_LARGE_VISION_BYTES"
|
ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_DRAGOON_VISION_BYTES"
|
||||||
|
|
||||||
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
||||||
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
||||||
# dangle inside the container mount — so the directory must be real and
|
# dangle inside the container mount — so the directory must be real and
|
||||||
# every named file must be a regular file, not a link.
|
# every named file must be a regular file, not a link.
|
||||||
TOK_DIR="$FRANK_LARGE_MODEL_DIR/tokenizer"
|
TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
|
||||||
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
|
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
|
||||||
echo "frank-large: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
|
echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
for TF in $FRANK_LARGE_TOKENIZER_FILES; do
|
for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
|
||||||
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
|
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
|
||||||
echo "frank-large: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
|
echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
# GPU access is mandatory but the node paths are variables so the seam can
|
# GPU access is mandatory but the node paths are variables so the seam can
|
||||||
# drive the gate on a box without an AMD GPU.
|
# drive the gate on a box without an AMD GPU.
|
||||||
DEV_KFD="${FRANK_LARGE_DEV_KFD:-/dev/kfd}"
|
DEV_KFD="${FRANK_DRAGOON_DEV_KFD:-/dev/kfd}"
|
||||||
DEV_DRI="${FRANK_LARGE_DEV_DRI:-/dev/dri}"
|
DEV_DRI="${FRANK_DRAGOON_DEV_DRI:-/dev/dri}"
|
||||||
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
||||||
echo "frank-large: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
||||||
@@ -125,15 +125,15 @@ fi
|
|||||||
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
|
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
|
||||||
# WAIT for the RAM to clear, so a transient dip boots once instead of
|
# WAIT for the RAM to clear, so a transient dip boots once instead of
|
||||||
# crash-looping; if it never clears, refuse loudly.
|
# crash-looping; if it never clears, refuse loudly.
|
||||||
MEMINFO="${FRANK_LARGE_MEMINFO:-/proc/meminfo}"
|
MEMINFO="${FRANK_DRAGOON_MEMINFO:-/proc/meminfo}"
|
||||||
PIN_KB=$(( FRANK_LARGE_CHECKPOINT_BYTES / 1024 ))
|
PIN_KB=$(( FRANK_DRAGOON_CHECKPOINT_BYTES / 1024 ))
|
||||||
NEED_KB=$(( PIN_KB + 16 * 1048576 ))
|
NEED_KB=$(( PIN_KB + 16 * 1048576 ))
|
||||||
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
|
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
|
||||||
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
|
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
|
||||||
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
||||||
WAIT_S="${FRANK_LARGE_RAM_WAIT:-600}"
|
WAIT_S="${FRANK_DRAGOON_RAM_WAIT:-600}"
|
||||||
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
|
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
|
||||||
echo "frank-large: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
|
echo "frank-dragoon: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
|
||||||
deadline=$(( $(date +%s) + WAIT_S ))
|
deadline=$(( $(date +%s) + WAIT_S ))
|
||||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||||
sleep 5
|
sleep 5
|
||||||
@@ -143,16 +143,16 @@ if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
||||||
echo "frank-large: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
|
echo "frank-dragoon: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "frank-large: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
|
echo "frank-dragoon: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
|
||||||
fi
|
fi
|
||||||
for CTR in ${FRANK_LARGE_LEGACY_CONTAINERS:-} "$FRANK_LARGE_CONTAINER"; do
|
for CTR in ${FRANK_DRAGOON_LEGACY_CONTAINERS:-} "$FRANK_DRAGOON_CONTAINER"; do
|
||||||
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
|
|
||||||
LOG="${FRANK_LARGE_LOG:-/home/dank/logs/frank_large.log}"
|
LOG="${FRANK_DRAGOON_LOG:-/home/dank/logs/frank_dragoon.log}"
|
||||||
mkdir -p "$(dirname "$LOG")"
|
mkdir -p "$(dirname "$LOG")"
|
||||||
# --ipc=host for the udev-granted supplementary group, seccomp unconfined
|
# --ipc=host for the udev-granted supplementary group, seccomp unconfined
|
||||||
# for the ROCm ioctl surface, and unbounded memlock (the unit's
|
# for the ROCm ioctl surface, and unbounded memlock (the unit's
|
||||||
@@ -163,17 +163,17 @@ mkdir -p "$(dirname "$LOG")"
|
|||||||
# and both sibling serve scripts pass no nofile either. No comment lines
|
# and both sibling serve scripts pass no nofile either. No comment lines
|
||||||
# inside the backslash chain — a mid-chain # terminates the continuation
|
# inside the backslash chain — a mid-chain # terminates the continuation
|
||||||
# (the 2026-10-05 exit-127 crash loop on the sibling serve script).
|
# (the 2026-10-05 exit-127 crash loop on the sibling serve script).
|
||||||
podman run --name "$FRANK_LARGE_CONTAINER" \
|
podman run --name "$FRANK_DRAGOON_CONTAINER" \
|
||||||
--publish "$FRANK_LARGE_PUBLISH" \
|
--publish "$FRANK_DRAGOON_PUBLISH" \
|
||||||
--ipc=host \
|
--ipc=host \
|
||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device "$DEV_KFD" --device "$DEV_DRI" \
|
--device "$DEV_KFD" --device "$DEV_DRI" \
|
||||||
--group-add keep-groups \
|
--group-add keep-groups \
|
||||||
--security-opt seccomp=unconfined \
|
--security-opt seccomp=unconfined \
|
||||||
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
|
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
|
||||||
-e HALOGEN_CHECKPOINT="/models/$FRANK_LARGE_CHECKPOINT" \
|
-e HALOGEN_CHECKPOINT="/models/$FRANK_DRAGOON_CHECKPOINT" \
|
||||||
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_LARGE_NGRAM_TABLE" \
|
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_DRAGOON_NGRAM_TABLE" \
|
||||||
-e HALOGEN_VISION_TOWER="/models/$FRANK_LARGE_VISION_TOWER" \
|
-e HALOGEN_VISION_TOWER="/models/$FRANK_DRAGOON_VISION_TOWER" \
|
||||||
-e HALOGEN_TOKENIZER=/tokenizer \
|
-e HALOGEN_TOKENIZER=/tokenizer \
|
||||||
-e HALOGEN_CTX="$HALOGEN_CTX" \
|
-e HALOGEN_CTX="$HALOGEN_CTX" \
|
||||||
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
|
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
|
||||||
@@ -187,9 +187,9 @@ podman run --name "$FRANK_LARGE_CONTAINER" \
|
|||||||
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
|
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
|
||||||
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
|
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
|
||||||
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
|
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
|
||||||
-v "$FRANK_LARGE_MODEL_DIR:/models:ro" \
|
-v "$FRANK_DRAGOON_MODEL_DIR:/models:ro" \
|
||||||
-v "$FRANK_LARGE_MODEL_DIR/tokenizer:/tokenizer:ro" \
|
-v "$FRANK_DRAGOON_MODEL_DIR/tokenizer:/tokenizer:ro" \
|
||||||
"$FRANK_LARGE_IMAGE" \
|
"$FRANK_DRAGOON_IMAGE" \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
|
|
||||||
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's
|
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
||||||
# frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
# frank-zealot on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
||||||
#
|
#
|
||||||
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
||||||
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
||||||
@@ -24,11 +24,11 @@
|
|||||||
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
||||||
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides.
|
# Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_SECONDARY_ENV_FILE:-$SCRIPT_DIR/frank-secondary.env}"
|
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -83,11 +83,11 @@ fi
|
|||||||
# has not finished, and Restart=always would turn it into a crash loop that
|
# has not finished, and Restart=always would turn it into a crash loop that
|
||||||
# reads like a runtime fault.
|
# reads like a runtime fault.
|
||||||
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
|
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
|
||||||
[[ -e "$path" ]] || { echo "frank-secondary: missing $path" >&2; exit 1; }
|
[[ -e "$path" ]] || { echo "frank-zealot: missing $path" >&2; exit 1; }
|
||||||
done
|
done
|
||||||
EXTRA_ARGS=()
|
EXTRA_ARGS=()
|
||||||
if [[ -n "$MMPROJ_FILE" ]]; then
|
if [[ -n "$MMPROJ_FILE" ]]; then
|
||||||
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-secondary: missing $MMPROJ_FILE" >&2; exit 1; }
|
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-zealot: missing $MMPROJ_FILE" >&2; exit 1; }
|
||||||
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -97,7 +97,7 @@ fi
|
|||||||
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
||||||
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
|
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
|
||||||
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
|
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
|
||||||
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-secondary: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
|
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-zealot: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
|
||||||
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
|
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
@@ -108,7 +108,7 @@ nvidia-modprobe -u -c=0 2>/dev/null || true
|
|||||||
|
|
||||||
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
|
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
|
||||||
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
|
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
|
||||||
# the q8_0 KV contract in frank-secondary.env is a real path — without this
|
# the q8_0 KV contract in frank-zealot.env is a real path — without this
|
||||||
# flag or that build option the engine silently falls back and the
|
# flag or that build option the engine silently falls back and the
|
||||||
# contract becomes a lie.
|
# contract becomes a lie.
|
||||||
ARGS=(
|
ARGS=(
|
||||||
+11
-11
@@ -145,12 +145,12 @@ for path in "${CHANGED[@]}"; do
|
|||||||
add_test tests/homepage_caddy.bats
|
add_test tests/homepage_caddy.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
;;
|
;;
|
||||||
# frank-large is a rootless quadlet again since 2026-09-16 (the
|
# frank-dragoon is a rootless quadlet again since 2026-09-16 (the
|
||||||
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
||||||
# podman posture, so it gets its own seam rather than the generic
|
# podman posture, so it gets its own seam rather than the generic
|
||||||
# fallback below.
|
# fallback below.
|
||||||
dankclaw/frank-large/*)
|
dankclaw/frank-dragoon/*)
|
||||||
add_test tests/frank_large_unit.bats
|
add_test tests/frank_dragoon_unit.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/docs_completeness.bats
|
add_test tests/docs_completeness.bats
|
||||||
;;
|
;;
|
||||||
@@ -187,29 +187,29 @@ for path in "${CHANGED[@]}"; do
|
|||||||
# as additions, with "no focused-test mapping".
|
# as additions, with "no focused-test mapping".
|
||||||
.AGENTTRASH/*|*/.AGENTTRASH/*)
|
.AGENTTRASH/*|*/.AGENTTRASH/*)
|
||||||
;;
|
;;
|
||||||
# frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the
|
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
|
||||||
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
|
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
|
||||||
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
|
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
|
||||||
# live in scripts/, which is why this case has to precede the generic
|
# live in scripts/, which is why this case has to precede the generic
|
||||||
# scripts/*.service and scripts/* fallbacks below.
|
# scripts/*.service and scripts/* fallbacks below.
|
||||||
scripts/frank-primary.*|scripts/serve-frank-primary.sh)
|
scripts/frank-archon.*|scripts/serve-frank-archon.sh)
|
||||||
add_test tests/frank_primary_unit.bats
|
add_test tests/frank_archon_unit.bats
|
||||||
add_test tests/llm_units.bats
|
add_test tests/llm_units.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
# frank-large's chat template lives in scripts/ (moved there on
|
# frank-dragoon's chat template lives in scripts/ (moved there on
|
||||||
# 2026-09-26 when its old home was retired): a template edit changes
|
# 2026-09-26 when its old home was retired): a template edit changes
|
||||||
# what the live endpoint renders, so it selects the large unit seam.
|
# what the live endpoint renders, so it selects the large unit seam.
|
||||||
scripts/qwen-sharp-v22.4.0.jinja)
|
scripts/qwen-sharp-v22.4.0.jinja)
|
||||||
add_test tests/frank_large_unit.bats
|
add_test tests/frank_dragoon_unit.bats
|
||||||
;;
|
;;
|
||||||
# frank-secondary is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
# frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
||||||
# binary, so no generated quadlet is involved. Its env, unit and serve
|
# binary, so no generated quadlet is involved. Its env, unit and serve
|
||||||
# script all live in scripts/, which is why this case has to precede
|
# script all live in scripts/, which is why this case has to precede
|
||||||
# the generic scripts/*.service and scripts/* fallbacks below.
|
# the generic scripts/*.service and scripts/* fallbacks below.
|
||||||
scripts/frank-secondary.*|scripts/serve-frank-secondary.sh)
|
scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
|
||||||
add_test tests/frank_secondary_unit.bats
|
add_test tests/frank_zealot_unit.bats
|
||||||
add_test tests/llm_units.bats
|
add_test tests/llm_units.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
|
|||||||
@@ -16,13 +16,13 @@ this file is repo-managed; edits are harness-side only.
|
|||||||
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
||||||
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
|
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
|
||||||
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
|
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
|
||||||
:7771, the 3090 compaction hub) — formerly `frank-primary`, `frank-large`,
|
:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
|
||||||
`frank-secondary`. The SERVICES keep the old names (units, scripts, env
|
`frank-zealot`. The SERVICES keep the old names (units, scripts, env
|
||||||
files, runbook pages); pages written in the old era name the service,
|
files, runbook pages); pages written in the old era name the service,
|
||||||
the harness names the model. The wire now matches: each endpoint's
|
the harness names the model. The wire now matches: each endpoint's
|
||||||
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
|
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
|
||||||
picker/grants, the homepage docs and the raw API all use one id per model
|
picker/grants, the homepage docs and the raw API all use one id per model
|
||||||
(see [[OpenWebUI]] for the grant re-key and the [[frank-primary-radiance-swift15-w4dn8]]
|
(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
|
||||||
note on radiance echoing the requested name).
|
note on radiance echoing the requested name).
|
||||||
|
|
||||||
## Compaction chain arithmetic (verify before ANY window change)
|
## Compaction chain arithmetic (verify before ANY window change)
|
||||||
@@ -45,14 +45,14 @@ The arithmetic (the coupled contract every endpoint's maxTokens comment
|
|||||||
encodes):
|
encodes):
|
||||||
|
|
||||||
1. **Source trigger** = `thresholdPercent × source contextWindow`
|
1. **Source trigger** = `thresholdPercent × source contextWindow`
|
||||||
(0.70 × 200,704 = 140,492 for frank-primary on the current shape)
|
(0.70 × 200,704 = 140,492 for frank-archon on the current shape)
|
||||||
2. **Output budget** = capped at the target model's `maxTokens`
|
2. **Output budget** = capped at the target model's `maxTokens`
|
||||||
(32,768 for frank-secondary; 36,864 frank-primary; 57,344 frank-large — the
|
(32,768 for frank-zealot; 36,864 frank-archon; 57,344 frank-dragoon — the
|
||||||
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
|
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
|
||||||
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
|
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
|
||||||
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
|
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
|
||||||
200,704 (frank-primary); 140,490 + 57,344 = 197,834 ≤ 200,700
|
200,704 (frank-archon); 140,490 + 57,344 = 197,834 ≤ 200,700
|
||||||
(frank-large); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-secondary).
|
(frank-dragoon); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-zealot).
|
||||||
|
|
||||||
Procedure (recompute from the LIVE files every time — windows rotate with
|
Procedure (recompute from the LIVE files every time — windows rotate with
|
||||||
every swap; the numbers above are the 2026-10-05 state):
|
every swap; the numbers above are the 2026-10-05 state):
|
||||||
@@ -112,19 +112,19 @@ Use **wire names**, not OMP camelCase. Two traps:
|
|||||||
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
|
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
|
||||||
*auto-detected* `thinkingFormat` + local baseUrl, **before**
|
*auto-detected* `thinkingFormat` + local baseUrl, **before**
|
||||||
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
|
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
|
||||||
models.yml does **not** enable it. Custom-named models (`frank-large`)
|
models.yml does **not** enable it. Custom-named models (`frank-dragoon`)
|
||||||
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
||||||
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
|
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
|
||||||
also sets it — `Object.assign` is shallow and replaces the whole object,
|
also sets it — `Object.assign` is shallow and replaces the whole object,
|
||||||
silently dropping the `reasoning_effort` kwarg the compat policy set.
|
silently dropping the `reasoning_effort` kwarg the compat policy set.
|
||||||
Top-level `preserve_thinking` is honoured by halogen. (frank-primary's
|
Top-level `preserve_thinking` is honoured by halogen. (frank-archon's
|
||||||
FreeToken entry currently carries a `chat_template_kwargs` extraBody with
|
FreeToken entry currently carries a `chat_template_kwargs` extraBody with
|
||||||
a comment asking to verify-and-prune — the exact trap to check at that
|
a comment asking to verify-and-prune — the exact trap to check at that
|
||||||
endpoint's next swap.)
|
endpoint's next swap.)
|
||||||
|
|
||||||
**Gotcha: silent greedy decode.** halogen has **no server-side sampler
|
**Gotcha: silent greedy decode.** halogen has **no server-side sampler
|
||||||
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
|
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
|
||||||
endpoint decodes greedy while looking configured. This is why frank-large's
|
endpoint decodes greedy while looking configured. This is why frank-dragoon's
|
||||||
entry carries the recipe on the wire and a comment saying "do not drop it".
|
entry carries the recipe on the wire and a comment saying "do not drop it".
|
||||||
|
|
||||||
**Reasoning-effort folding.** Qwen3.8's template accepts only
|
**Reasoning-effort folding.** Qwen3.8's template accepts only
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
|
|||||||
|
|
||||||
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
|
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
|
||||||
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
|
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
|
||||||
a 100+ GiB payload — and frank-primary (dankserver:7770) is also the
|
a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
|
||||||
harness's own model backend, so a blind deploy is a self-inflicted outage for
|
harness's own model backend, so a blind deploy is a self-inflicted outage for
|
||||||
possibly zero behavioral change.
|
possibly zero behavioral change.
|
||||||
|
|
||||||
|
|||||||
@@ -62,8 +62,8 @@ service on the other host.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
|
||||||
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
||||||
| `frank-large` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
||||||
| `frank-secondary` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
||||||
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
|
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
|
||||||
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
||||||
|
|
||||||
|
|||||||
@@ -95,7 +95,7 @@ host-side `workflows/` dir and the repo copy.
|
|||||||
## Recommended additions
|
## Recommended additions
|
||||||
|
|
||||||
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
|
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
|
||||||
frank-secondary's). All repos verified 2026-10-05. None are on disk yet.
|
frank-zealot's). All repos verified 2026-10-05. None are on disk yet.
|
||||||
|
|
||||||
| # | What | Repo | Grab | Size | Why |
|
| # | What | Repo | Grab | Size | Why |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ timestamp: 2026-08-26
|
|||||||
# Dankclaw Stack
|
# Dankclaw Stack
|
||||||
|
|
||||||
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
|
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
|
||||||
stack. Its LLM inference endpoint `frank-large` (`:7770`) is a ROCmFPX
|
stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
|
||||||
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
|
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
|
||||||
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
|
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
|
||||||
FLM v1.0.1) serves sight+sound models. The completion endpoint
|
FLM v1.0.1) serves sight+sound models. The completion endpoint
|
||||||
@@ -30,7 +30,7 @@ graph LR
|
|||||||
S --> V[Gluetun VPN]
|
S --> V[Gluetun VPN]
|
||||||
O --> L[llama.cpp dankserver :7770]
|
O --> L[llama.cpp dankserver :7770]
|
||||||
O --> H[Hermes]
|
O --> H[Hermes]
|
||||||
O --> E[frank-large :7770]
|
O --> E[frank-dragoon :7770]
|
||||||
```
|
```
|
||||||
|
|
||||||
## Services
|
## Services
|
||||||
@@ -48,14 +48,14 @@ graph LR
|
|||||||
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
|
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
|
||||||
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
|
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
|
||||||
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
|
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
|
||||||
| frank-large (host service) | — | `7770` | [[deployment]] |
|
| frank-dragoon (host service) | — | `7770` | [[deployment]] |
|
||||||
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
|
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
|
||||||
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
|
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
|
||||||
|
|
||||||
The LLM rows are bare-metal systemd services, not containers. This
|
The LLM rows are bare-metal systemd services, not containers. This
|
||||||
repo owns their scripts, units, and knowledge base (see
|
repo owns their scripts, units, and knowledge base (see
|
||||||
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
|
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
|
||||||
`frank-large` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
|
`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
|
||||||
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
|
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
|
||||||
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
|
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
|
||||||
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
|
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ is not durable. The tracked, sanitized configuration baseline is
|
|||||||
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
|
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
|
||||||
- The data volume persists skills, memory, and session history.
|
- The data volume persists skills, memory, and session history.
|
||||||
- Hermes has two named local providers: `dankclaw-large` reaches
|
- Hermes has two named local providers: `dankclaw-large` reaches
|
||||||
`http://192.168.0.143:7770/v1` and exposes `frank-large`;
|
`http://192.168.0.143:7770/v1` and exposes `frank-dragoon`;
|
||||||
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
|
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
|
||||||
`frank-medium` (renamed from `frank-small` on 2026-09-15).
|
`frank-medium` (renamed from `frank-small` on 2026-09-15).
|
||||||
- The live `config.yaml` contains secrets and user state and is not committed;
|
- The live `config.yaml` contains secrets and user state and is not committed;
|
||||||
@@ -64,7 +64,7 @@ is not durable. The tracked, sanitized configuration baseline is
|
|||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- Hermes is under active development; pin the image tag for reproducibility.
|
- Hermes is under active development; pin the image tag for reproducibility.
|
||||||
- Model choice drives quality; the local `frank-large` (Dankclaw, 7770) is the intended backend.
|
- Model choice drives quality; the local `frank-dragoon` (Dankclaw, 7770) is the intended backend.
|
||||||
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
|
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
|
||||||
|
|
||||||
## Browser automation (Chrome DevTools Protocol daemon)
|
## Browser automation (Chrome DevTools Protocol daemon)
|
||||||
@@ -189,7 +189,7 @@ What actually breaks, and the fix:
|
|||||||
auxiliary:
|
auxiliary:
|
||||||
title_generation:
|
title_generation:
|
||||||
provider: dankserver-small # names track the live config; diff first
|
provider: dankserver-small # names track the live config; diff first
|
||||||
model: frank-secondary
|
model: frank-zealot
|
||||||
api_mode: codex_responses # real per-task key
|
api_mode: codex_responses # real per-task key
|
||||||
extra_body:
|
extra_body:
|
||||||
reasoning:
|
reasoning:
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ https://github.com/open-webui/open-webui
|
|||||||
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
|
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
|
||||||
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
|
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
|
||||||
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
|
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
|
||||||
| Backends | Picker exposes exactly one model — `frank-secondary` (host.containers.internal:7771). The `frank-large` and `frank-primary` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
|
| Backends | Picker exposes exactly one model — `frank-zealot` (host.containers.internal:7771). The `frank-dragoon` and `frank-archon` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
|
||||||
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
|
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
|
||||||
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
|
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
|
||||||
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
|
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
|
||||||
@@ -34,15 +34,15 @@ https://github.com/open-webui/open-webui
|
|||||||
The picker is built from the `model` table in `webui.db` (one row per model)
|
The picker is built from the `model` table in `webui.db` (one row per model)
|
||||||
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
|
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
|
||||||
connection discovery — so the picker shows exactly what is seeded there.
|
connection discovery — so the picker shows exactly what is seeded there.
|
||||||
As of 2026-10-07 the picker exposes **exactly one model: `frank-secondary`**.
|
As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`**.
|
||||||
|
|
||||||
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-large` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
|
||||||
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-primary` | radiance TP=2 (2× R9700) | Dankserver |
|
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
|
||||||
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-secondary` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
||||||
|
|
||||||
The `model` table carries exactly one decoration row, `frank-secondary`
|
The `model` table carries exactly one decoration row, `frank-zealot`
|
||||||
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
|
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
|
||||||
Q6_K + BF16 vision mmproj).
|
Q6_K + BF16 vision mmproj).
|
||||||
|
|
||||||
@@ -62,7 +62,7 @@ Two gotchas make the wiring non-obvious:
|
|||||||
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
|
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
|
||||||
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
|
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
|
||||||
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
|
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
|
||||||
the llama.cpp endpoints (frank-large, frank-secondary). The fix is to set
|
the llama.cpp endpoints (frank-dragoon, frank-zealot). The fix is to set
|
||||||
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
|
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
|
||||||
builds the model list directly and never fetches `/v1/models`. frank-medium
|
builds the model list directly and never fetches `/v1/models`. frank-medium
|
||||||
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
|
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
|
||||||
@@ -116,7 +116,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
|||||||
models. Response key is `items`, not `data`.
|
models. Response key is `items`, not `data`.
|
||||||
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
|
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
|
||||||
with exactly one word: pong") → each returns content `pong`. Reasoning
|
with exactly one word: pong") → each returns content `pong`. Reasoning
|
||||||
models need a generous budget: frank-secondary (Qwen3.5 template) returns
|
models need a generous budget: frank-zealot (Qwen3.5 template) returns
|
||||||
empty content + `finish_reason: "length"` at `max_tokens 200` (the
|
empty content + `finish_reason: "length"` at `max_tokens 200` (the
|
||||||
thinking block eats it) and answers `pong` at 1500.
|
thinking block eats it) and answers `pong` at 1500.
|
||||||
3. A 500 "Server Connection Error" → `podman logs open-webui` for
|
3. A 500 "Server Connection Error" → `podman logs open-webui` for
|
||||||
@@ -191,8 +191,8 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
|
|||||||
run with host `python3` over ssh against
|
run with host `python3` over ssh against
|
||||||
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
|
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
|
||||||
Dankclaw; table is `user`, singular; no `active` column). No per-user model
|
Dankclaw; table is `user`, singular; no `active` column). No per-user model
|
||||||
grant is needed: the picker model `frank-secondary` carries the global
|
grant is needed: the picker model `frank-zealot` carries the global
|
||||||
`access_grant` row `(model, frank-secondary, user, *, read)`, so any
|
`access_grant` row `(model, frank-zealot, user, *, read)`, so any
|
||||||
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
|
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
|
||||||
not exist until the account signs in (2026-10-09: five provisioned accounts
|
not exist until the account signs in (2026-10-09: five provisioned accounts
|
||||||
had no row).
|
had no row).
|
||||||
@@ -271,18 +271,18 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
|
|||||||
|
|
||||||
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
|
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
|
||||||
|
|
||||||
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-large`/`frank-medium`/`frank-secondary` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
|
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-dragoon`/`frank-medium`/`frank-zealot` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
|
||||||
|
|
||||||
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
|
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
|
||||||
|
|
||||||
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
|
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
|
||||||
|
|
||||||
- **2026-10-07 — picker narrowed to `frank-secondary` only.** Per Frank: the only model accessible in OpenWebUI is `frank-secondary`. Rebuilt the `model` table to a single `frank-secondary` row (the old rows `frank-large`/`frank-primary`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-large) and 1 (frank-primary) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-secondary` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-secondary"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
|
- **2026-10-07 — picker narrowed to `frank-zealot` only.** Per Frank: the only model accessible in OpenWebUI is `frank-zealot`. Rebuilt the `model` table to a single `frank-zealot` row (the old rows `frank-dragoon`/`frank-archon`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-dragoon) and 1 (frank-archon) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-zealot` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-zealot"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
|
||||||
|
|
||||||
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
|
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
|
||||||
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
||||||
id-keyed objects had to move in the same window as the alias flip:
|
id-keyed objects had to move in the same window as the alias flip:
|
||||||
`model.id`+`name` (`frank-secondary` → `frank-zealot`, `params`
|
`model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
|
||||||
`{"context":{"length":200704}}` and the `meta.description` kept),
|
`{"context":{"length":200704}}` and the `meta.description` kept),
|
||||||
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
||||||
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
||||||
|
|||||||
@@ -33,10 +33,10 @@ it is not invisible.
|
|||||||
|
|
||||||
## Why 64 GiB
|
## Why 64 GiB
|
||||||
|
|
||||||
Dankclaw is shared. `frank-large` alone holds **103.5 GiB of the 121.2 GiB**
|
Dankclaw is shared. `frank-dragoon` alone holds **103.5 GiB of the 121.2 GiB**
|
||||||
host (`68.0 weights + 14.4 KV pool + 21.1 working`), leaving roughly
|
host (`68.0 weights + 14.4 KV pool + 21.1 working`), leaving roughly
|
||||||
**7.1 GiB** for 28 co-tenant containers whose working set is ~4.3 GiB. The
|
**7.1 GiB** for 28 co-tenant containers whose working set is ~4.3 GiB. The
|
||||||
engine itself warns that when that runs out, frank-large and its neighbours
|
engine itself warns that when that runs out, frank-dragoon and its neighbours
|
||||||
"can stall for minutes at a time at full CPU with no disk activity and no
|
"can stall for minutes at a time at full CPU with no disk activity and no
|
||||||
output". Swap is the valve that turns an OOM kill into slow paging.
|
output". Swap is the valve that turns an OOM kill into slow paging.
|
||||||
|
|
||||||
@@ -96,20 +96,20 @@ Then confirm the inference service is unaffected — a completion, not just
|
|||||||
```bash
|
```bash
|
||||||
curl -s http://127.0.0.1:7770/v1/chat/completions \
|
curl -s http://127.0.0.1:7770/v1/chat/completions \
|
||||||
-H 'Content-Type: application/json' \
|
-H 'Content-Type: application/json' \
|
||||||
-d '{"model":"frank-large","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}'
|
-d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Reply with exactly: SWAP64-OK"}],"max_tokens":48,"reasoning_effort":"low"}'
|
||||||
```
|
```
|
||||||
|
|
||||||
As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up,
|
As run 2026-09-12: `finish_reason: stop`, `SWAP64-OK`, 28 containers up,
|
||||||
frank-large `health=200` throughout.
|
frank-dragoon `health=200` throughout.
|
||||||
|
|
||||||
## Caveats
|
## Caveats
|
||||||
|
|
||||||
- **Swap cannot help the pinned weights.** frank-large's ~68 GiB of weights are
|
- **Swap cannot help the pinned weights.** frank-dragoon's ~68 GiB of weights are
|
||||||
held deliberately; paging them is a stall, not a relief. More swap does not
|
held deliberately; paging them is a stall, not a relief. More swap does not
|
||||||
make the doubled KV pool affordable — see the reserve/pool ceiling in
|
make the doubled KV pool affordable — see the reserve/pool ceiling in
|
||||||
[[halogen-flash-server]].
|
[[halogen-flash-server]].
|
||||||
- **`vm.swappiness=60` is untested against this workload.** Under real
|
- **`vm.swappiness=60` is untested against this workload.** Under real
|
||||||
pressure the kernel will happily page anonymous memory, and frank-large's KV
|
pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
|
||||||
pool and working set *are* anonymous. Swapping the KV pool would be far worse
|
pool and working set *are* anonymous. Swapping the KV pool would be far worse
|
||||||
than the OOM kill it prevents. If the box ever starts thrashing, lowering
|
than the OOM kill it prevents. If the box ever starts thrashing, lowering
|
||||||
swappiness (or capping the engine's cgroup) is the knob, not more swap.
|
swappiness (or capping the engine's cgroup) is the knob, not more swap.
|
||||||
|
|||||||
@@ -22,8 +22,8 @@ The homepage is live on Dankclaw. `./scripts/deploy.sh dankclaw` copies the stat
|
|||||||
**Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
|
**Docs page shipped (2026-09-17):** `/docs` serves a static user documentation
|
||||||
page (`shared/homepage/docs.html`) — access walkthrough, the running-model
|
page (`shared/homepage/docs.html`) — access walkthrough, the running-model
|
||||||
guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
|
guide, media how-tos, and troubleshooting. Per-service `/docs/<svc>/` pages
|
||||||
remain a separate thread. The three LLM cards (`frank-large`, `frank-medium`,
|
remain a separate thread. The three LLM cards (`frank-dragoon`, `frank-medium`,
|
||||||
`frank-secondary`) moved from admin-only to the **user** page with live model
|
`frank-zealot`) moved from admin-only to the **user** page with live model
|
||||||
descriptions and a `docs` link to `/docs#models`.
|
descriptions and a `docs` link to `/docs#models`.
|
||||||
|
|
||||||
**Sections + WIP split (2026-09-21):** both pages now group their panels
|
**Sections + WIP split (2026-09-21):** both pages now group their panels
|
||||||
@@ -48,23 +48,23 @@ exact bug bit the screenshots on 2026-10-08). The docs page gained a full
|
|||||||
(`/docs#api-access`): per-user OpenWebUI API keys (User menu → Settings →
|
(`/docs#api-access`): per-user OpenWebUI API keys (User menu → Settings →
|
||||||
Account → API keys), the base URL `https://ai.franksdank.cloud/api/v1`, and
|
Account → API keys), the base URL `https://ai.franksdank.cloud/api/v1`, and
|
||||||
copy-paste configs for **Opencode** (`~/.config/opencode/opencode.json`) and
|
copy-paste configs for **Opencode** (`~/.config/opencode/opencode.json`) and
|
||||||
**Pi** (`~/.pi/agent/models.json`) pointing at `frank-secondary`. Shipped
|
**Pi** (`~/.pi/agent/models.json`) pointing at `frank-zealot`. Shipped
|
||||||
together with the OpenWebUI side: `features.api_keys=true` for regular users
|
together with the OpenWebUI side: `features.api_keys=true` for regular users
|
||||||
and the restored `frank-secondary` model row + public `read` access grant
|
and the restored `frank-zealot` model row + public `read` access grant
|
||||||
without which the picker showed "No results found" for every family account
|
without which the picker showed "No results found" for every family account
|
||||||
(see [[OpenWebUI]]).
|
(see [[OpenWebUI]]).
|
||||||
|
|
||||||
**Capability equivalence notes (2026-10-08):** each fleet card (and the
|
**Capability equivalence notes (2026-10-08):** each fleet card (and the
|
||||||
`/docs#models` section) carries a `details.equiv` line — frank-large
|
`/docs#models` section) carries a `details.equiv` line — frank-dragoon
|
||||||
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
|
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
|
||||||
intelligence)", frank-primary "equivalent to Claude Opus 4.8 / OpenAI-Terra
|
intelligence)", frank-archon "equivalent to Claude Opus 4.8 / OpenAI-Terra
|
||||||
(same era)", frank-secondary "equivalent to last year's frontier level
|
(same era)", frank-zealot "equivalent to last year's frontier level
|
||||||
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
|
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
|
||||||
pinned in `homepage_config.bats`.
|
pinned in `homepage_config.bats`.
|
||||||
|
|
||||||
**Protoss rename + Model Information component (2026-10-08):** the model
|
**Protoss rename + Model Information component (2026-10-08):** the model
|
||||||
cards became the StarCraft trio — `frank-large` → **frank-dragoon**
|
cards became the StarCraft trio — `frank-dragoon` → **frank-dragoon**
|
||||||
(iGPU), `frank-primary` → **frank-archon** (Dankserver), `frank-secondary`
|
(iGPU), `frank-archon` → **frank-archon** (Dankserver), `frank-zealot`
|
||||||
→ **frank-zealot** (3090). The endpoint/API ids did NOT change in this wave; only the
|
→ **frank-zealot** (3090). The endpoint/API ids did NOT change in this wave; only the
|
||||||
family-facing names. The three cards fused into one **Model Information**
|
family-facing names. The three cards fused into one **Model Information**
|
||||||
component (`app.js` `renderModelInfo()`, `.model-info`/`.mi-*` styles):
|
component (`app.js` `renderModelInfo()`, `.model-info`/`.mi-*` styles):
|
||||||
@@ -201,8 +201,8 @@ the pixels.
|
|||||||
**Model ids official (2026-10-08):** the card ids `frank-zealot`,
|
**Model ids official (2026-10-08):** the card ids `frank-zealot`,
|
||||||
`frank-dragoon`, `frank-archon` are now the harness's official model ids
|
`frank-dragoon`, `frank-archon` are now the harness's official model ids
|
||||||
in `models.yml`/`config.yml` (see [[OMP Harness]]); the
|
in `models.yml`/`config.yml` (see [[OMP Harness]]); the
|
||||||
systemd services keep the `frank-primary`/`frank-large`/
|
systemd services keep the `frank-archon`/`frank-dragoon`/
|
||||||
`frank-secondary` names. Homepage, docs page and harness now agree on
|
`frank-zealot` names. Homepage, docs page and harness now agree on
|
||||||
one name per model.
|
one name per model.
|
||||||
|
|
||||||
**Api ids renamed on the wire (2026-10-09):** the promise in
|
**Api ids renamed on the wire (2026-10-09):** the promise in
|
||||||
@@ -316,7 +316,7 @@ flowchart TD
|
|||||||
## Service classification
|
## Service classification
|
||||||
|
|
||||||
**User services** (root page), in section order:
|
**User services** (root page), in section order:
|
||||||
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-large, frank-primary, frank-secondary — status + docs link, no Open link).
|
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot — status + docs link, no Open link).
|
||||||
- **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
|
- **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
|
||||||
- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
|
- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
|
||||||
|
|
||||||
|
|||||||
@@ -38,7 +38,7 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
|
|||||||
| `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
|
| `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
|
||||||
| `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
|
| `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
|
||||||
| `frank-medium` | `192.168.0.235:7770` | the LLM fleet — `vllm:` series; a different machine, so the LAN address works |
|
| `frank-medium` | `192.168.0.235:7770` | the LLM fleet — `vllm:` series; a different machine, so the LAN address works |
|
||||||
| `frank-large`, `frank-secondary` | `host.containers.internal:7770`, `:7771` | the LLM fleet — `llamacpp:` series; same-host net=host binds, so via the gateway |
|
| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet — `llamacpp:` series; same-host net=host binds, so via the gateway |
|
||||||
|
|
||||||
**Three jobs read down, and they are not outages** (audited 2026-09-16):
|
**Three jobs read down, and they are not outages** (audited 2026-09-16):
|
||||||
|
|
||||||
|
|||||||
@@ -115,7 +115,7 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
|
|||||||
- Internal order is fixed (ngram types first, then drafters); CLI order is
|
- Internal order is fixed (ngram types first, then drafters); CLI order is
|
||||||
irrelevant.
|
irrelevant.
|
||||||
- **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
|
- **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
|
||||||
on the qwen35 hybrid (frank-large era) measured −15 % prose / −7 %
|
on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
|
||||||
block-copy (2026-10-03): adding the ngram layer logs `recurrent state
|
block-copy (2026-10-03): adding the ngram layer logs `recurrent state
|
||||||
rollback is not compatible with 'ngram-mod' - disabling rollback support`,
|
rollback is not compatible with 'ngram-mod' - disabling rollback support`,
|
||||||
and MTP verification depends on that rollback, so acceptance drops on ALL
|
and MTP verification depends on that rollback, so acceptance drops on ALL
|
||||||
@@ -215,9 +215,9 @@ The DFlash draft is not a standalone model. It must pair with its target model.
|
|||||||
|
|
||||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||||
|
|
||||||
The Dankserver llama.cpp deployment (`frank-secondary`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||||
|
|
||||||
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-secondary` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
|
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
|
||||||
|
|
||||||
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
|
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
|
||||||
|
|
||||||
@@ -250,7 +250,7 @@ docker run --rm --name vllm-qwen36-dflash \
|
|||||||
|
|
||||||
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
|
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
|
||||||
|
|
||||||
`frank-large` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
|
`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
|
||||||
|
|
||||||
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
|
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
|
||||||
|
|
||||||
@@ -315,7 +315,7 @@ not switch without re-measuring".
|
|||||||
|
|
||||||
The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
|
The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
|
||||||
slot split, quant rung — not just spec flags) is the transient systemd
|
slot split, quant rung — not just spec flags) is the transient systemd
|
||||||
drop-in procedure in [[frank-large-benchmark]] and [[frank-secondary-benchmark]].
|
drop-in procedure in [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]].
|
||||||
|
|
||||||
## Is it worth it? (arithmetic before deployment)
|
## Is it worth it? (arithmetic before deployment)
|
||||||
|
|
||||||
@@ -419,4 +419,4 @@ measurement that would settle it.
|
|||||||
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
|
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
|
||||||
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
|
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
|
||||||
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
|
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
|
||||||
- [[frank-large-benchmark]] and [[frank-secondary-benchmark]] — the single-GPU env drop-in A/B vehicle
|
- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — the single-GPU env drop-in A/B vehicle
|
||||||
@@ -22,7 +22,7 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
|
|||||||
(b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
|
(b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
|
||||||
Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
|
Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
|
||||||
302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
|
302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
|
||||||
hold on the 8060S. See [[frank-large-benchmark]].
|
hold on the 8060S. See [[frank-dragoon-benchmark]].
|
||||||
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
|
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
|
||||||
does not exist on this machine (checked 2026-08-15). GPU DPM is already
|
does not exist on this machine (checked 2026-08-15). GPU DPM is already
|
||||||
`performance`, forced level `auto`. This item is N/A.
|
`performance`, forced level `auto`. This item is N/A.
|
||||||
@@ -48,7 +48,7 @@ lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
|
|||||||
different, driver-reported figure that has always been larger than the kernel
|
different, driver-reported figure that has always been larger than the kernel
|
||||||
cap.
|
cap.
|
||||||
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
|
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
|
||||||
limits documented in [[frank-large-benchmark]].
|
limits documented in [[frank-dragoon-benchmark]].
|
||||||
|
|
||||||
## 1. Self-Hosted Agentic Tools
|
## 1. Self-Hosted Agentic Tools
|
||||||
|
|
||||||
@@ -106,7 +106,7 @@ All require root and a reboot for the modprobe change.
|
|||||||
|
|
||||||
## Sources
|
## Sources
|
||||||
|
|
||||||
- [[frank-large-benchmark]] — measured backend/spec numbers
|
- [[frank-dragoon-benchmark]] — measured backend/spec numbers
|
||||||
- [[deployment]] — deploy and update process
|
- [[deployment]] — deploy and update process
|
||||||
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
|
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
|
||||||
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
|
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
|
||||||
@@ -115,7 +115,7 @@ All require root and a reboot for the modprobe change.
|
|||||||
|
|
||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [[frank-large-benchmark]]
|
- [[frank-dragoon-benchmark]]
|
||||||
- [[DFlash DSpark Speculative Decoding]]
|
- [[DFlash DSpark Speculative Decoding]]
|
||||||
- [[deployment]]
|
- [[deployment]]
|
||||||
- [[index]]
|
- [[index]]
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host run
|
|||||||
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-09-11** with the `amd_iommu=off` decision) |
|
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-09-11** with the `amd_iommu=off` decision) |
|
||||||
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
|
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
|
||||||
|
|
||||||
The NPU is retired (2026-09-11). The `frank-secondary` name now serves Gemma4-12B QAT on the RTX 3090 via llama.cpp (see [[frank-secondary-gemma4-12b]]).
|
The NPU is retired (2026-09-11). The `frank-zealot` name now serves Gemma4-12B QAT on the RTX 3090 via llama.cpp (see [[frank-zealot-gemma4-12b]]).
|
||||||
|
|
||||||
### ROCm
|
### ROCm
|
||||||
|
|
||||||
@@ -101,7 +101,7 @@ cable between claw and server is required for 10G.
|
|||||||
|
|
||||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||||
- The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
|
- The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
|
||||||
- The active service is `frank-large` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-large-gufo-qwen38-27b]].
|
- The active service is `frank-dragoon` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-dragoon-gufo-qwen38-27b]].
|
||||||
- The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path.
|
- The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path.
|
||||||
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
|
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
|
||||||
- 1.1 TB free disk space is ample for model storage.
|
- 1.1 TB free disk space is ample for model storage.
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ timestamp: 2026-09-20
|
|||||||
|
|
||||||
Dankserver is a media/library server and the `frank-medium` LLM host. The
|
Dankserver is a media/library server and the `frank-medium` LLM host. The
|
||||||
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
|
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
|
||||||
FreeToken/CUDA `frank-secondary` era was removed on 2026-09-15 and replaced with
|
FreeToken/CUDA `frank-zealot` era was removed on 2026-09-15 and replaced with
|
||||||
two AMD Radeon AI PRO R9700 cards.
|
two AMD Radeon AI PRO R9700 cards.
|
||||||
|
|
||||||
## Hardware Specifications
|
## Hardware Specifications
|
||||||
@@ -107,8 +107,8 @@ hijack `libhsa-runtime64` were purged on 2026-09-15 — see
|
|||||||
The active inference service is `frank-medium.service`: mainline llama-server on
|
The active inference service is `frank-medium.service`: mainline llama-server on
|
||||||
port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to
|
port 7770, layer-split 32:33 across both R9700s with the DFlash2 drafter pinned to
|
||||||
ROCm1. See [[frank-medium-llamacpp-dflash2]]. The former
|
ROCm1. See [[frank-medium-llamacpp-dflash2]]. The former
|
||||||
`frank-secondary.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap;
|
`frank-zealot.service` (FreeToken/CUDA) was retired with the 2026-09-15 GPU swap;
|
||||||
its unit is archived in `~/.AGENTTRASH/20260915-frank-secondary-retire/host-units/`,
|
its unit is archived in `~/.AGENTTRASH/20260915-frank-zealot-retire/host-units/`,
|
||||||
and the vllm-radiance replica trio in `~/.AGENTTRASH/20260916-vllm-radiance-retire/`.
|
and the vllm-radiance replica trio in `~/.AGENTTRASH/20260916-vllm-radiance-retire/`.
|
||||||
See [[deployment]].
|
See [[deployment]].
|
||||||
|
|
||||||
|
|||||||
@@ -13,12 +13,12 @@ timestamp: 2026-09-03
|
|||||||
|
|
||||||
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
|
> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
|
||||||
> See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
|
> See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
|
||||||
> The `frank-secondary` name and port 52625 then belonged to the vLLM
|
> The `frank-zealot` name and port 52625 then belonged to the vLLM
|
||||||
> service on Dankclaw (nameistoken Quark W8A8); this page is archive.
|
> service on Dankclaw (nameistoken Quark W8A8); this page is archive.
|
||||||
|
|
||||||
# FastFlowLM NPU Inference
|
# FastFlowLM NPU Inference
|
||||||
|
|
||||||
FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-secondary` (port 52625) serves in production. See Promotion.
|
FastFlowLM (FLM) runs LLMs entirely on the AMD Ryzen AI NPU. It targets XDNA2 chips: Strix, Strix Halo, Kraken, and Gorgon Point. The project describes itself as "Ollama, but deeply optimized for NPUs." As of 2026-08-16, FLM is installed on Dankclaw, the NPU validates, and `frank-zealot` (port 52625) serves in production. See Promotion.
|
||||||
|
|
||||||
## Verdict (2026-08-16)
|
## Verdict (2026-08-16)
|
||||||
|
|
||||||
@@ -167,7 +167,7 @@ Proposed role: a small, always-on, low-power endpoint for triage, summarization,
|
|||||||
- `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
|
- `gpt-oss:20b` pulled (4/4 files verified, 13.8 GB `model.q4nx` + tokenizer) into `~/.config/flm/models/GPT-OSS-20B-NPU2/`.
|
||||||
- `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below).
|
- `qwythos:9b` (Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-NPU2) installed 2026-08-17 via `flm-add.py` (downloaded from the HF repo; stdlib-only, no pip needed). Tag registered in the user registry `~/.config/flm/model_list.json` (a copy of the v1.0.1 system registry + the custom entry); kernels symlinked `~/.config/flm/xclbins/Huihui-...-NPU2 -> /opt/fastflowlm/share/flm/xclbins/Qwen3.5-9B-NPU2`. The service env exports `FLM_CONFIG_PATH` and `FLM_XCLBIN_PATH` so the tag resolves (without them flm hits the unknown-tag fallback, see the incident below).
|
||||||
- `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
|
- `flm serve gpt-oss:20b --host 0.0.0.0 -p 52625 --pmode performance` runs as a nohup process. It is not a systemd unit. It does not survive a reboot.
|
||||||
- The `llama-server` (frank-large, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
|
- The `llama-server` (frank-dragoon, the PP head) holds an fd on `/dev/accel/accel0` from its own startup (device enumeration, not compute). FLM opened the same device concurrently without conflict. Verified live 2026-08-16.
|
||||||
|
|
||||||
## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
|
## Measured Results (2026-08-16, gpt-oss-20b on the Dankclaw NPU)
|
||||||
|
|
||||||
@@ -185,7 +185,7 @@ Long-context prefill is not yet measured (runs 1-3 used under 200 prompt tokens)
|
|||||||
|
|
||||||
## Model Bakeoff (measured 2026-08-16, 4 models)
|
## Model Bakeoff (measured 2026-08-16, 4 models)
|
||||||
|
|
||||||
All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-large) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
|
All runs on the Dankclaw NPU (50 TOPS Strix Halo), single stream, `--pmode performance`, PP head (frank-dragoon) serving concurrently on 7770. Decode t/s is the FLM-reported rate. Vision accuracy = ground-truth elements out of 4 (text SPECTRUM, red circle left, blue square right, three green triangles bottom).
|
||||||
|
|
||||||
| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
|
| Model | Tag | Weights | Decode t/s (short) | 8k prefill t/s | 8k decode t/s | Vision decode t/s | Vision accuracy | Max ctx (registry) |
|
||||||
|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
|
|-------|-----|---------|-------------------:|---------------:|--------------:|------------------:|----------------|--------------------|
|
||||||
@@ -203,21 +203,21 @@ Notes:
|
|||||||
|
|
||||||
## Promotion (2026-08-16)
|
## Promotion (2026-08-16)
|
||||||
|
|
||||||
`frank-secondary` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-secondary.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-secondary.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
|
`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
|
||||||
|
|
||||||
### Boot policy (2026-08-17)
|
### Boot policy (2026-08-17)
|
||||||
|
|
||||||
The `frank-secondary` unit is disabled for boot. Only the PP pair (`frank-large` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw:
|
The `frank-zealot` unit is disabled for boot. Only the PP pair (`frank-dragoon` + `frank-pp-worker`) starts at boot. The unit file was removed from `/etc/systemd/system`; the running FLM service keeps serving until stopped or rebooted. To restore the endpoint, run these commands on Dankclaw:
|
||||||
|
|
||||||
```
|
```
|
||||||
sudo ln -sf /home/dank/vllmScripts/scripts/frank-secondary.service /etc/systemd/system/frank-secondary.service
|
sudo ln -sf /home/dank/vllmScripts/scripts/frank-zealot.service /etc/systemd/system/frank-zealot.service
|
||||||
sudo systemctl daemon-reload
|
sudo systemctl daemon-reload
|
||||||
sudo systemctl enable frank-secondary
|
sudo systemctl enable frank-zealot
|
||||||
```
|
```
|
||||||
|
|
||||||
Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
|
Live config (2026-08-16): `FLM_PMODE=turbo`, `FLM_ASR=1` (loads Whisper-V3-Turbo-NPU2; exposes `/v1/audio/transcriptions`), `FLM_CTX_LEN=131072` (verified max — Gemma4 natively supports 128K and the NPU budget fits it; KV occupancy measured at 0.11% for a 123-token exchange, decode 12.0 t/s, 13 GiB headroom).
|
||||||
|
|
||||||
The PP split was renamed `frank-pp-head` → `frank-large` (2026-08-16, commit `48142f5`).
|
The PP split was renamed `frank-pp-head` → `frank-dragoon` (2026-08-16, commit `48142f5`).
|
||||||
|
|
||||||
Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it.
|
Model swap (2026-08-16, commit `50c55ee`): `FLM_MODEL=qwen3.6-moe:35b-a3b`, `FLM_CTX_LEN=262144` (native max), `FLM_PMODE=turbo`, `FLM_ASR=1` kept. Decode measured 12.5-13.1 t/s. The Gemma4 live config above (131072 ctx) applied to `gemma4-it:e4b`. Swap `FLM_MODEL` and `FLM_CTX_LEN` back to restore it.
|
||||||
|
|
||||||
@@ -353,7 +353,7 @@ installed) against `/opt/fastflowlm/share/flm/xclbins/` (available).
|
|||||||
Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
|
Symptom: a client on dankware could not reach `qwen3.6-moe:35b-a3b` on port 52625. The journal shows three separate causes in the 14:19-15:03 window.
|
||||||
|
|
||||||
1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`.
|
1. **EXEC 203 crash loop (14:19-14:20).** The exec bit was lost on `serve-flm-small.sh` on the target. The unit failed `status=203/EXEC: Permission denied` four times, then started after the bit was restored. The git mode is `100755` (commit `3f42210`), and a `git pull` restores the bit. Verify with `git ls-files -s scripts/serve-flm-small.sh`.
|
||||||
2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-secondary` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-secondary' is not supported.` Each such request also triggered the unknown-tag fallback below.
|
2. **Wrong model ID (14:29-14:52).** The client config still used the model ID `frank-zealot` (a service alias). FLM rejects that tag with `[ERROR] Model tag 'frank-zealot' is not supported.` Each such request also triggered the unknown-tag fallback below.
|
||||||
3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load.
|
3. **Swap churn (14:48-15:03).** The swap `gemma4-it:e4b` → `qwen3.6-moe:35b-a3b` restarted the service several times. Each load of the 23 GB model takes about 1.5-2.5 minutes at 262144 ctx. The endpoint does not serve during a load.
|
||||||
|
|
||||||
Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16.
|
Fix: the client model ID must be the FLM registry tag `qwen3.6-moe:35b-a3b`. The working pi agent entry in `~/.pi/agent/models.json` is `baseUrl http://192.168.0.143:52625/v1`, `id qwen3.6-moe:35b-a3b`, `contextWindow 262144`, `reasoning true`, `input [text, image]`. Verified end-to-end from dankware (192.168.0.226) at 15:32 on 2026-08-16.
|
||||||
@@ -390,7 +390,7 @@ pi sessions failed against this endpoint with `Stream ended without finish_reaso
|
|||||||
- **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step.
|
- **The FLM serve process is not under systemd.** A reboot of Dankclaw kills it. The PP head service is enabled and returns. The FLM process must be restarted by hand. A unit file is a natural next step.
|
||||||
- **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this.
|
- **gpt-oss reasoning budget.** The model can spend its full token budget on `reasoning_content` and return empty content. Clients must handle or cap this.
|
||||||
- **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags.
|
- **No llama.cpp GGUF path.** FLM uses its own pre-compiled kernels. Our UD-Q8_K_XL and HauhauCS builds are not importable. NPU work means FLM-specific model tags.
|
||||||
- **NPU production role (2026-08-16).** `frank-secondary` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded.
|
- **NPU production role (2026-08-16).** `frank-zealot` serves `qwen3.6-moe:35b-a3b` on port 52625 (swap churn and the unknown-tag incident are recorded above). The remaining NPU content (benchmarks, runbook steps) is still research until a step lands and is recorded.
|
||||||
|
|
||||||
## Citations
|
## Citations
|
||||||
|
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ read the returned content:
|
|||||||
```python
|
```python
|
||||||
import json, urllib.request
|
import json, urllib.request
|
||||||
req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
|
req = urllib.request.Request("http://<host>:<port>/v1/chat/completions",
|
||||||
json.dumps({"model": "frank-secondary", "messages": [{"role": "user",
|
json.dumps({"model": "frank-zealot", "messages": [{"role": "user",
|
||||||
"content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
|
"content": "Hi"}]}).encode(), {"Content-Type": "application/json"})
|
||||||
print(json.load(urllib.request.urlopen(req, timeout=600)))
|
print(json.load(urllib.request.urlopen(req, timeout=600)))
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -119,5 +119,5 @@ re-download.
|
|||||||
|
|
||||||
- [[Llama Cpp Runtimes]] — which fork ships which custom types
|
- [[Llama Cpp Runtimes]] — which fork ships which custom types
|
||||||
- [[Llama Cpp Code Diff]] — the enum tables and code-level differences
|
- [[Llama Cpp Code Diff]] — the enum tables and code-level differences
|
||||||
- [[frank-large-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production
|
- [[frank-dragoon-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production
|
||||||
- [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads
|
- [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ Fleet law first: **STOP the production service before any GPU experiment**
|
|||||||
GPU and corrupts results; an unisolated GPU experiment has precedent for
|
GPU and corrupts results; an unisolated GPU experiment has precedent for
|
||||||
wedging a host until power cycle. Reference harnesses:
|
wedging a host until power cycle. Reference harnesses:
|
||||||
`scripts/bench-batch-ubatch.sh`, `scripts/autoresearch.sh`,
|
`scripts/bench-batch-ubatch.sh`, `scripts/autoresearch.sh`,
|
||||||
`scripts/autoresearch/frank-primary-arms.sh`.
|
`scripts/autoresearch/frank-archon-arms.sh`.
|
||||||
|
|
||||||
## 1. The window structure (one config per arm, a driver owns the window)
|
## 1. The window structure (one config per arm, a driver owns the window)
|
||||||
|
|
||||||
@@ -27,7 +27,7 @@ wedging a host until power cycle. Reference harnesses:
|
|||||||
with `container state improper`.
|
with `container state improper`.
|
||||||
2. **Boot the arm as the service, not a scratch port** (no scratch ports on
|
2. **Boot the arm as the service, not a scratch port** (no scratch ports on
|
||||||
LLM hosts — one card, one server). The serve script takes its config via
|
LLM hosts — one card, one server). The serve script takes its config via
|
||||||
env-file override (`FRANK_SECONDARY_ENV_FILE`, `FRANK_PRIMARY_AR_ENV_FILE`)
|
env-file override (`FRANK_ZEALOT_ENV_FILE`, `FRANK_ARCHON_AR_ENV_FILE`)
|
||||||
pointing at a staged copy; the checkout is never mutated.
|
pointing at a staged copy; the checkout is never mutated.
|
||||||
3. **Fast-fail the health wait**: poll `podman ps | grep -qx <name>` inside
|
3. **Fast-fail the health wait**: poll `podman ps | grep -qx <name>` inside
|
||||||
the loop — a refused boot dies in one poll instead of 20 min. Capture the
|
the loop — a refused boot dies in one poll instead of 20 min. Capture the
|
||||||
@@ -245,17 +245,17 @@ nearly shipped the wrong setting.
|
|||||||
the comment); verify multi-line command construction with `bash -n` AND a
|
the comment); verify multi-line command construction with `bash -n` AND a
|
||||||
render test.
|
render test.
|
||||||
|
|
||||||
## 8. Results location and recovery (frank-primary arms)
|
## 8. Results location and recovery (frank-archon arms)
|
||||||
|
|
||||||
| copy | path |
|
| copy | path |
|
||||||
|---|---|
|
|---|---|
|
||||||
| per-arm evidence | `$ARMS_DIR` = `~/.jobs/<run-stamp>/` (`.rc`, `.done` beside it) |
|
| per-arm evidence | `$ARMS_DIR` = `~/.jobs/<run-stamp>/` (`.rc`, `.done` beside it) |
|
||||||
| summary | `$ARMS_DIR/summary.tsv` = `arm rc decode_tps decode_tps_t0.7 kv_tokens boot_s` + `<arm>.out/.env/.overlay/.rc/.metrics` |
|
| summary | `$ARMS_DIR/summary.tsv` = `arm rc decode_tps decode_tps_t0.7 kv_tokens boot_s` + `<arm>.out/.env/.overlay/.rc/.metrics` |
|
||||||
| server-side last window | `/home/dank/.autoresearch-frank-primary/run.json` |
|
| server-side last window | `/home/dank/.autoresearch-frank-archon/run.json` |
|
||||||
| durable memo | `.../last/run.json` + `last/hash` (promoted only after bench AND verified restore) |
|
| durable memo | `.../last/run.json` + `last/hash` (promoted only after bench AND verified restore) |
|
||||||
| fallback | `~/.omp/agent/sessions/<ws>/<session>/<N>.{shake,bash}.log` (permanent) |
|
| fallback | `~/.omp/agent/sessions/<ws>/<session>/<N>.{shake,bash}.log` (permanent) |
|
||||||
|
|
||||||
`/tmp/frank-primary-arms-*` and `/tmp/armstub/` are stub self-tests —
|
`/tmp/frank-archon-arms-*` and `/tmp/armstub/` are stub self-tests —
|
||||||
constants 99.9/88.8 identical across arms by construction; **identical
|
constants 99.9/88.8 identical across arms by construction; **identical
|
||||||
numbers across arms == stub or bug, never a measurement.** After compaction,
|
numbers across arms == stub or bug, never a measurement.** After compaction,
|
||||||
re-harvest with `grep -rhoE 'decode_tps[a-z0-9_.]*=[0-9.]+'` over the session
|
re-harvest with `grep -rhoE 'decode_tps[a-z0-9_.]*=[0-9.]+'` over the session
|
||||||
|
|||||||
@@ -161,8 +161,8 @@ of the same RAM, not extra memory.
|
|||||||
## 7. IBS model-access attribution (AMD CPUs)
|
## 7. IBS model-access attribution (AMD CPUs)
|
||||||
|
|
||||||
Real CPU access patterns for mmap'd weights, live, no stop (established on
|
Real CPU access patterns for mmap'd weights, live, no stop (established on
|
||||||
frank-primary, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
|
frank-archon, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
|
||||||
`scripts/frank-primary-expert-activity.py` (self-contained dump + sample +
|
`scripts/frank-archon-expert-activity.py` (self-contained dump + sample +
|
||||||
analyze; `--analyze-only`).
|
analyze; `--analyze-only`).
|
||||||
|
|
||||||
1. GGUF header-only scan (ms for a 50 GiB part) → tensor extents. **Extents
|
1. GGUF header-only scan (ms for a 50 GiB part) → tensor extents. **Extents
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ co-tenants.
|
|||||||
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
|
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
|
||||||
"move layers to the card" converts reclaimable cache into device memory and
|
"move layers to the card" converts reclaimable cache into device memory and
|
||||||
frees nothing for co-tenants. A served model's footprint ≠ weight size
|
frees nothing for co-tenants. A served model's footprint ≠ weight size
|
||||||
(frank-secondary: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
|
(frank-zealot: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
|
||||||
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
|
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
|
||||||
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
|
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
|
||||||
- **Context lengths: derive, never remember.** `llama-bench` computes
|
- **Context lengths: derive, never remember.** `llama-bench` computes
|
||||||
@@ -157,7 +157,7 @@ in the timings. Context cap for full-model residency on that shape stayed
|
|||||||
## Related
|
## Related
|
||||||
|
|
||||||
- [[GGUF Compatibility]] — what a payload is before it is priced
|
- [[GGUF Compatibility]] — what a payload is before it is priced
|
||||||
- [[frank-secondary-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
|
- [[frank-zealot-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
|
||||||
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
|
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
|
||||||
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
|
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
|
||||||
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
|
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
|
||||||
|
|||||||
@@ -8,8 +8,8 @@ timestamp: 2026-10-05T00:00:00-07:00
|
|||||||
|
|
||||||
# Model Swap and Rollback
|
# Model Swap and Rollback
|
||||||
|
|
||||||
Applies to every endpoint swap on the fleet (frank-secondary / frank-large /
|
Applies to every endpoint swap on the fleet (frank-zealot / frank-dragoon /
|
||||||
frank-primary). Repo law: all config flows through git; the live models are
|
frank-archon). Repo law: all config flows through git; the live models are
|
||||||
host state under `/home/dank/models/<name>/`.
|
host state under `/home/dank/models/<name>/`.
|
||||||
|
|
||||||
## 1. Fit preflight — never deploy a swap without confirming it FITS
|
## 1. Fit preflight — never deploy a swap without confirming it FITS
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ conclusions.
|
|||||||
| Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` |
|
| Blocks | 65 (45 on worker, 20 on head) | `--tensor-split 0.68,0.32` |
|
||||||
| FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` |
|
| FA layers | 16 total, interval 4 (11 worker / 5 head) | GGUF `full_attention_interval` |
|
||||||
| KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed |
|
| KV per FA layer per token | 3136 B (K q8_0 1088 + V f16 2048) | env file comment, confirmed |
|
||||||
| ubatch | 256 | `UBATCH_SIZE` in `frank-large.env` |
|
| ubatch | 256 | `UBATCH_SIZE` in `frank-dragoon.env` |
|
||||||
| MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` |
|
| MTP verify step positions | 5 (1 + n_max 4) | `SPEC_DRAFT_N_MAX=4` |
|
||||||
| Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s |
|
| Link (old path) | 1 GbE via USB-C adapter on claw | `enx00e04cc2e1e5`, negotiated 1000 Mb/s |
|
||||||
| Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends |
|
| Link (current path) | 2.5 GbE eno1 to eno1 | negotiated 2500 Mb/s both ends |
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ morning-of runbook. Research background:
|
|||||||
|
|
||||||
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
|
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
|
||||||
|
|
||||||
This split is the production `frank-large` service. Current
|
This split is the production `frank-dragoon` service. Current
|
||||||
state, verified against the live journal and output smoke tests
|
state, verified against the live journal and output smoke tests
|
||||||
(2026-08-17, load 02:06):
|
(2026-08-17, load 02:06):
|
||||||
|
|
||||||
@@ -41,17 +41,17 @@ state, verified against the live journal and output smoke tests
|
|||||||
Vulkan).
|
Vulkan).
|
||||||
- Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
|
- Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
|
||||||
stable; all allocations are upfront).
|
stable; all allocations are upfront).
|
||||||
- Unit `frank-large.service`, launched through
|
- Unit `frank-dragoon.service`, launched through
|
||||||
`serve-pp-head.sh` + `frank-large.env`.
|
`serve-pp-head.sh` + `frank-dragoon.env`.
|
||||||
|
|
||||||
## Service enablement (2026-08-17)
|
## Service enablement (2026-08-17)
|
||||||
|
|
||||||
The PP pair is the only inference service enabled for boot. Both units
|
The PP pair is the only inference service enabled for boot. Both units
|
||||||
(`frank-large` on Dankclaw, `frank-pp-worker` on Dankserver) are
|
(`frank-dragoon` on Dankclaw, `frank-pp-worker` on Dankserver) are
|
||||||
enabled. The NPU unit (`frank-secondary` on Dankclaw) and all retired
|
enabled. The NPU unit (`frank-zealot` on Dankclaw) and all retired
|
||||||
units are disabled. Restart order: worker first, then head. The Dankserver
|
units are disabled. Restart order: worker first, then head. The Dankserver
|
||||||
helper `frank-restart-small.sh` is obsolete (it restarts the removed
|
helper `frank-restart-small.sh` is obsolete (it restarts the removed
|
||||||
`frank-secondary` unit); restart the worker with
|
`frank-zealot` unit); restart the worker with
|
||||||
`sudo systemctl restart frank-pp-worker`. The Dankclaw helper
|
`sudo systemctl restart frank-pp-worker`. The Dankclaw helper
|
||||||
`frank-restart-large.sh` (sync + cache drop + restart) is correct for the
|
`frank-restart-large.sh` (sync + cache drop + restart) is correct for the
|
||||||
head. `deploy.sh dankserver` is broken until retargeted. See
|
head. `deploy.sh dankserver` is broken until retargeted. See
|
||||||
@@ -173,9 +173,9 @@ llama-server b10433 ggml-rpc-server b10433
|
|||||||
|------|---------|---------|
|
|------|---------|---------|
|
||||||
| `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
|
| `scripts/frank-pp-worker.env` | server | worker tunables (bind, port, device, cache) |
|
||||||
| `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
|
| `scripts/serve-pp-worker.sh` | server | runs `ggml-rpc-server` from the b10433 worktree build |
|
||||||
| `scripts/frank-large.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
|
| `scripts/frank-dragoon.env` | claw | head tunables (split, ctx, spec, KV, I/O); was `frank-pp-head.env` before the 2026-08-16 rename |
|
||||||
| `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
|
| `scripts/serve-pp-head.sh` | claw | runs `llama-server --rpc ...` (see "Gotchas") |
|
||||||
| `scripts/frank-large.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
|
| `scripts/frank-dragoon.service` / `frank-pp-worker.service` | both | optional systemd units (manual install); the head unit was renamed from `frank-pp-head.service` on 2026-08-16 and is symlinked from the repo into `/etc/systemd/system` |
|
||||||
| `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
|
| `scripts/pp-test.sh` | any box with ssh to both | test orchestrator (preflight/start/verify/bench/stop-pp/restore) |
|
||||||
| `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
|
| `scripts/pp-bench.py` | claw | API driver; reports server-side `print_timing` |
|
||||||
|
|
||||||
@@ -229,7 +229,7 @@ FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh stop-pp
|
|||||||
FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
|
FRANK_SUDO_PW=<pw> ./scripts/pp-test.sh restore
|
||||||
```
|
```
|
||||||
|
|
||||||
Config variants: edit `scripts/frank-large.env` on the claw (git commit +
|
Config variants: edit `scripts/frank-dragoon.env` on the claw (git commit +
|
||||||
pull if the bench runs from the repo), then re-run `start` + `bench`:
|
pull if the bench runs from the repo), then re-run `start` + `bench`:
|
||||||
|
|
||||||
- No spec baseline: `SPEC_NONE=1`
|
- No spec baseline: `SPEC_NONE=1`
|
||||||
|
|||||||
@@ -65,6 +65,6 @@ graph LR
|
|||||||
## See Also
|
## See Also
|
||||||
|
|
||||||
- [[DFlash DSpark Speculative Decoding]]
|
- [[DFlash DSpark Speculative Decoding]]
|
||||||
- [[frank-secondary-benchmark]]
|
- [[frank-zealot-benchmark]]
|
||||||
- [[Dankserver System Specs]]
|
- [[Dankserver System Specs]]
|
||||||
- [[Dankclaw System Specs]]
|
- [[Dankclaw System Specs]]
|
||||||
|
|||||||
@@ -78,12 +78,12 @@ Laws that steer everything here:
|
|||||||
variants **[verified (tool existence: heretic; rest survey)]**.
|
variants **[verified (tool existence: heretic; rest survey)]**.
|
||||||
- Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
|
- Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
|
||||||
weights, so it composes with any expert quantization.
|
weights, so it composes with any expert quantization.
|
||||||
- Fleet status: frank-large runs an abliterated base **by Frank's choice**
|
- Fleet status: frank-dragoon runs an abliterated base **by Frank's choice**
|
||||||
(AGENTS: censored-base refusals were the accepted trade of the 27B era);
|
(AGENTS: censored-base refusals were the accepted trade of the 27B era);
|
||||||
frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.
|
frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.
|
||||||
|
|
||||||
### HauhauCS QX_K_P "Perfect" quants
|
### HauhauCS QX_K_P "Perfect" quants
|
||||||
- Our frank-secondary publisher. Method, from the publisher's own cards
|
- Our frank-zealot publisher. Method, from the publisher's own cards
|
||||||
**[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
|
**[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
|
||||||
(2) quantize with **imatrix calibration on the ABLITERATED weights**,
|
(2) quantize with **imatrix calibration on the ABLITERATED weights**,
|
||||||
(3) K_P = a **per-model, per-tensor mixed-precision profile** —
|
(3) K_P = a **per-model, per-tensor mixed-precision profile** —
|
||||||
@@ -106,7 +106,7 @@ Laws that steer everything here:
|
|||||||
|
|
||||||
## Per-card playbooks
|
## Per-card playbooks
|
||||||
|
|
||||||
### RTX 3090 — the unlocked card (frank-secondary runs llama.cpp BY CHOICE)
|
### RTX 3090 — the unlocked card (frank-zealot runs llama.cpp BY CHOICE)
|
||||||
The 3090 is the only fleet device with no format lock: radiance needs
|
The 3090 is the only fleet device with no format lock: radiance needs
|
||||||
`.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other
|
`.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other
|
||||||
engine. sm_86 + CUDA is the industry's default target, so the runtime
|
engine. sm_86 + CUDA is the industry's default target, so the runtime
|
||||||
@@ -179,7 +179,7 @@ Q8_K_P 40.6 GiB + mmproj 0.84) — Q4_K_P IS the protected-tensor blend.
|
|||||||
The self-made delta: calibration on OUR corpora, our hot-expert law,
|
The self-made delta: calibration on OUR corpora, our hot-expert law,
|
||||||
and a KLD gate scored at the serving shape.
|
and a KLD gate scored at the serving shape.
|
||||||
|
|
||||||
### R9700 — frank-primary (radiance)
|
### R9700 — frank-archon (radiance)
|
||||||
The full self-quant pipeline already exists end to end: fetch BF16 →
|
The full self-quant pipeline already exists end to end: fetch BF16 →
|
||||||
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,
|
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,
|
||||||
`RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) →
|
`RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) →
|
||||||
@@ -219,7 +219,7 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
|
|||||||
until proven), and does its MTP cast to the fp8/u2 encodings like
|
until proven), and does its MTP cast to the fp8/u2 encodings like
|
||||||
qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.
|
qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.
|
||||||
|
|
||||||
### Strix Halo — frank-large (halogen) and friends
|
### Strix Halo — frank-dragoon (halogen) and friends
|
||||||
- `.hgn`: **closed production door.** No public quantizer exists
|
- `.hgn`: **closed production door.** No public quantizer exists
|
||||||
(verified absence across peonist-ai repos); the format's lock is the
|
(verified absence across peonist-ai repos); the format's lock is the
|
||||||
engine's lock. We consume vendor artifacts; the repo-side tooling is
|
engine's lock. We consume vendor artifacts; the repo-side tooling is
|
||||||
@@ -255,6 +255,6 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
|
|||||||
`scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
|
`scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
|
||||||
`scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
|
`scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
|
||||||
[[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
|
[[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
|
||||||
[[frank-primary-radiance-swift15-w4dn8]],
|
[[frank-archon-radiance-swift15-w4dn8]],
|
||||||
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]],
|
[[frank-zealot-llama-qwen35-9b-q6k-2x200704]],
|
||||||
[[frank-large-flash-server-v2-swift15]], [[GGUF Compatibility]].
|
[[frank-dragoon-flash-server-v2-swift15]], [[GGUF Compatibility]].
|
||||||
|
|||||||
@@ -247,4 +247,4 @@ see [[dankserver-qwen38-fp8-expert-offload-tuning]] for why it was rejected);
|
|||||||
measured campaigns
|
measured campaigns
|
||||||
- [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
|
- [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
|
||||||
- [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
|
- [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
|
||||||
- [[frank-large-benchmark]] and [[frank-secondary-benchmark]] — window discipline on live hosts
|
- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — window discipline on live hosts
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user