Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d4ce76b68b | ||
|
|
65bd5ccea7 | ||
|
|
5976ad0550 | ||
|
|
5ae6b7b928 | ||
|
|
cc0bc06288 | ||
|
|
bdead570e2 | ||
|
|
156ea71879 | ||
|
|
ab80c1366d | ||
|
|
c5a816e6a8 | ||
|
|
cb25796a6b | ||
|
|
f31e7c5ed8 | ||
|
|
028afdeb58 | ||
|
|
acf2a5cee5 | ||
|
|
c52ba781e3 | ||
|
|
0bf26b0741 | ||
|
|
394fd8535b | ||
|
|
55f87fcdce |
No files matched your search
@@ -1,4 +1,4 @@
|
|||||||
# Tunables for frank-large (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
|
# Tunables for frank-dragoon (kingjones777/Qwen3.8-Flash-Next-ROCmFP4-
|
||||||
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
|
# STRIX-GGUF on the kingjones30/ROCmFPX llama.cpp runtime, Dankclaw,
|
||||||
# Strix Halo gfx1151, unified memory).
|
# Strix Halo gfx1151, unified memory).
|
||||||
#
|
#
|
||||||
@@ -56,7 +56,7 @@ CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
|||||||
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
||||||
|
|
||||||
# Alias + network.
|
# Alias + network.
|
||||||
ALIAS=frank-large
|
ALIAS=frank-dragoon
|
||||||
BIND_ADDR=0.0.0.0
|
BIND_ADDR=0.0.0.0
|
||||||
LISTEN_PORT=7770
|
LISTEN_PORT=7770
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-large)
|
Description=ROCmFPX (kingjones30 fork) llama.cpp Qwen3.8-Flash-Next ROCmFP4 STRIX 177B MoE (dankclaw, frank-dragoon)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -7,8 +7,8 @@ Wants=network-online.target
|
|||||||
Type=simple
|
Type=simple
|
||||||
User=dank
|
User=dank
|
||||||
Group=dank
|
Group=dank
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
@@ -6,7 +6,7 @@
|
|||||||
# is reachable on this port regardless of the boot tag. The NPU is a
|
# is reachable on this port regardless of the boot tag. The NPU is a
|
||||||
# single-lane device: generations serialize FIFO, and the first request after a
|
# single-lane device: generations serialize FIFO, and the first request after a
|
||||||
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
# model swap pays the ~6-11 s load cost. The NPU draws from the unified RAM
|
||||||
# pool shared with frank-large (iGPU).
|
# pool shared with frank-dragoon (iGPU).
|
||||||
FLM_MODEL=nuextract3:4b
|
FLM_MODEL=nuextract3:4b
|
||||||
FLM_HOST=0.0.0.0
|
FLM_HOST=0.0.0.0
|
||||||
FLM_PORT=7771
|
FLM_PORT=7771
|
||||||
|
|||||||
+3
-3
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
|
# Serve kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF as
|
||||||
# frank-large via the kingjones30/ROCmFPX llama.cpp runtime on
|
# frank-dragoon via the kingjones30/ROCmFPX llama.cpp runtime on
|
||||||
# Dankclaw (Strix Halo gfx1151, unified memory).
|
# Dankclaw (Strix Halo gfx1151, unified memory).
|
||||||
#
|
#
|
||||||
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
# Runtime: kingjones30/ROCmFPX (fork of charlie12345/ROCmFPX with the
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -44,7 +44,7 @@ fi
|
|||||||
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
|
: "${MMPROJ_PATH:=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf}"
|
||||||
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
||||||
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
: "${CHAT_TEMPLATE_KWARGS:={}}"
|
||||||
: "${ALIAS:=frank-large}"
|
: "${ALIAS:=frank-dragoon}"
|
||||||
: "${BIND_ADDR:=0.0.0.0}"
|
: "${BIND_ADDR:=0.0.0.0}"
|
||||||
: "${LISTEN_PORT:=7770}"
|
: "${LISTEN_PORT:=7770}"
|
||||||
: "${CTX_SIZE:=401408}"
|
: "${CTX_SIZE:=401408}"
|
||||||
@@ -4,7 +4,7 @@
|
|||||||
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
|
# Runs `flm serve <tag>` on the XDNA2 NPU (50 TOPS, Dankclaw). The NPU is a
|
||||||
# single-lane device: generations serialize in FIFO order, so this is a
|
# single-lane device: generations serialize in FIFO order, so this is a
|
||||||
# background/triage endpoint, not an interactive multi-user chat. The NPU
|
# background/triage endpoint, not an interactive multi-user chat. The NPU
|
||||||
# draws from the unified RAM pool shared with frank-large (the iGPU
|
# draws from the unified RAM pool shared with frank-dragoon (the iGPU
|
||||||
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
|
# llama.cpp service); see serverAppConfigs-docs/LLM/frank-npu.md
|
||||||
# for the memory budget and the model-selection tradeoff.
|
# for the memory budget and the model-selection tradeoff.
|
||||||
#
|
#
|
||||||
|
|||||||
@@ -9,11 +9,11 @@ services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
|
|||||||
merge absorbed the former `vllmScripts` repo, history preserved).
|
merge absorbed the former `vllmScripts` repo, history preserved).
|
||||||
|
|
||||||
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
|
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
|
||||||
Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
|
Calibre-Web, Jellyfin, Plex) plus the `frank-archon` LLM service.
|
||||||
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
||||||
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
||||||
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (flash-server
|
Alloy, Pi-hole) plus TWO LLM services: `frank-dragoon` (flash-server
|
||||||
container on the gfx1151 iGPU) and `frank-secondary` (llama.cpp CUDA on the
|
container on the gfx1151 iGPU) and `frank-zealot` (llama.cpp CUDA on the
|
||||||
RTX 3090). Caddy on
|
RTX 3090). Caddy on
|
||||||
Dankclaw reverse-proxies both hosts.
|
Dankclaw reverse-proxies both hosts.
|
||||||
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
||||||
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
|
|
||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
||||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||||
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
the [runbook index](#runbook-index). The full swap history lives in
|
the [runbook index](#runbook-index). The full swap history lives in
|
||||||
@@ -35,11 +35,16 @@ the [runbook index](#runbook-index). The full swap history lives in
|
|||||||
|
|
||||||
## Repository Layout
|
## Repository Layout
|
||||||
|
|
||||||
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
|
- `dankserver/` — quadlet units + Caddyfile for the media stack (the
|
||||||
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
|
compose.yaml inventory retired 2026-10-13; its mounts had diverged from
|
||||||
- `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
|
the quadlets)
|
||||||
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
|
- `dankclaw/` — quadlet units + Caddyfile for the AI stack (compose retired 2026-09-16)
|
||||||
scripts, systemd units, env tunables, bench harnesses
|
- `shared/` — the service inventory (`services.yml`), homepage, Prometheus,
|
||||||
|
Loki, Alloy, SearXNG configs
|
||||||
|
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh` and
|
||||||
|
`service-map.py` (the inventory's only reader), LLM serve scripts + the
|
||||||
|
shared artifact gates (`scripts/lib/llm-gates.sh`), systemd units, env
|
||||||
|
tunables, bench harnesses
|
||||||
- `old/` — retired LLM scripts (reference only)
|
- `old/` — retired LLM scripts (reference only)
|
||||||
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
||||||
The managed skill library is distilled there one page per skill-cluster:
|
The managed skill library is distilled there one page per skill-cluster:
|
||||||
@@ -66,14 +71,19 @@ All edits flow through git. Never edit files on a remote machine directly.
|
|||||||
./scripts/deploy.sh dankclaw
|
./scripts/deploy.sh dankclaw
|
||||||
./scripts/deploy.sh dankserver
|
./scripts/deploy.sh dankserver
|
||||||
./scripts/deploy.sh dankclaw --service searxng
|
./scripts/deploy.sh dankclaw --service searxng
|
||||||
./scripts/deploy.sh dankserver --service frank-primary
|
./scripts/deploy.sh dankserver --service frank-archon
|
||||||
./scripts/deploy.sh dankclaw --dry-run
|
./scripts/deploy.sh dankclaw --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
The planner maps changed paths to services; the remote runner restarts only
|
The inventory (`shared/services.yml`, read only through
|
||||||
the selected service and records its revision outside the checkout.
|
`scripts/service-map.py`) maps changed paths to services; the remote runner
|
||||||
|
restarts only the selected service and records its revision outside the
|
||||||
|
checkout. Both `--service` allow-lists, the planner, and the changed-test
|
||||||
|
seam mapper ARE the inventory — adding, moving, or retiring a service is a
|
||||||
|
data edit, not a script edit
|
||||||
|
(`serverAppConfigs-docs/Architecture/Service Inventory.md`).
|
||||||
Documentation-only changes are a no-op. LLM services are selected only by
|
Documentation-only changes are a no-op. LLM services are selected only by
|
||||||
their own files — every endpoint (`frank-large`/`frank-secondary`/`frank-primary`)
|
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
||||||
by its unit/env/serve trio in `scripts/` — so
|
by its unit/env/serve trio in `scripts/` — so
|
||||||
unrelated
|
unrelated
|
||||||
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
|
container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
|
||||||
@@ -85,7 +95,7 @@ restarts each service whose marker is behind HEAD *and* whose paths changed
|
|||||||
since that marker — so a backlog accumulated by service-scoped deploys fires
|
since that marker — so a backlog accumulated by service-scoped deploys fires
|
||||||
all at once, and one failing health gate aborts the run with every later
|
all at once, and one failing health gate aborts the run with every later
|
||||||
service left pending. Name the service unless a drain is what you want
|
service left pending. Name the service unless a drain is what you want
|
||||||
(`./scripts/deploy.sh dankserver --service frank-primary`).
|
(`./scripts/deploy.sh dankserver --service frank-archon`).
|
||||||
|
|
||||||
### Deployment completeness (MANDATORY)
|
### Deployment completeness (MANDATORY)
|
||||||
|
|
||||||
@@ -164,12 +174,17 @@ Tests are mandatory before commit (`bats` required on the dev machine):
|
|||||||
./tests/run.sh
|
./tests/run.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Seams: compose validity, env consistency (no leaked secrets), Caddyfile
|
Seams: env consistency (no leaked secrets), Caddyfile
|
||||||
validity, quadlet units, docs completeness **including vault link integrity**,
|
validity, quadlet units, docs completeness **including vault link integrity**,
|
||||||
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
||||||
deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
|
LLM artifact gates (`llm_gates`/`llm_deploy_gates` — the shared
|
||||||
and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
|
`scripts/lib/llm-gates.sh` every endpoint's serve and deploy gates run on),
|
||||||
no project, because such a spec silently never runs.
|
deployment flow, service-scoped planning, the **service inventory**
|
||||||
|
(`service_inventory.bats`: inventory soundness, the pinned path->service
|
||||||
|
plan fixture, render drift for the Pi-hole hosts file), homepage
|
||||||
|
config/pages/deploy/checker, and `e2e_scaffold` — which fails if a
|
||||||
|
Playwright spec file's number prefix is in no project, because such a spec
|
||||||
|
silently never runs.
|
||||||
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
||||||
|
|
||||||
## Secret Rules
|
## Secret Rules
|
||||||
@@ -229,17 +244,17 @@ archive.
|
|||||||
**STOP the production service BEFORE benchmarking** — these servers serve
|
**STOP the production service BEFORE benchmarking** — these servers serve
|
||||||
real users, and a bench beside a live service starves the GPU and corrupts
|
real users, and a bench beside a live service starves the GPU and corrupts
|
||||||
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
||||||
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-secondary were measured. Rolling your own bench
|
endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-dragoon and frank-zealot were measured. Rolling your own bench
|
||||||
requires asking first.
|
requires asking first.
|
||||||
Scratch ports are not used on LLM hosts — a
|
Scratch ports are not used on LLM hosts — a
|
||||||
test process cannot share the single GPU anyway; validate against the service
|
test process cannot share the single GPU anyway; validate against the service
|
||||||
itself.
|
itself.
|
||||||
|
|
||||||
(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
|
(A gufo-era law; frank-dragoon ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
|
||||||
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
|
halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
|
||||||
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
|
rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
|
||||||
settings, and will invert the ranking.**
|
settings, and will invert the ranking.**
|
||||||
Measured 2026-09-28 on frank-large at depth 32768: its corpus
|
Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus
|
||||||
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
|
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
|
||||||
draft widths, i.e. the same highly-predictable text every run, and 93 % of
|
draft widths, i.e. the same highly-predictable text every run, and 93 % of
|
||||||
every proposal accepted. It reported 29–37 t/s decode where real traffic
|
every proposal accepted. It reported 29–37 t/s decode where real traffic
|
||||||
@@ -264,12 +279,12 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
|
|||||||
merely wedged kernel usually still answers ARP.
|
merely wedged kernel usually still answers ARP.
|
||||||
|
|
||||||
1. Stop and mask (self-healing would restart it):
|
1. Stop and mask (self-healing would restart it):
|
||||||
- Dankclaw `frank-large` (root unit, halogen podman on the iGPU):
|
- Dankclaw `frank-dragoon` (root unit, halogen podman on the iGPU):
|
||||||
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
|
`sudo systemctl stop frank-dragoon && sudo systemctl mask frank-dragoon`
|
||||||
- Dankclaw `frank-secondary` (root unit, llama.cpp on the 3090):
|
- Dankclaw `frank-zealot` (root unit, llama.cpp on the 3090):
|
||||||
`sudo systemctl stop frank-secondary && sudo systemctl mask frank-secondary`
|
`sudo systemctl stop frank-zealot && sudo systemctl mask frank-zealot`
|
||||||
- Dankserver `frank-primary` (root unit, radiance on both R9700s, TP=2):
|
- Dankserver `frank-archon` (root unit, radiance on both R9700s, TP=2):
|
||||||
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
|
`sudo systemctl stop frank-archon && sudo systemctl mask frank-archon`
|
||||||
2. Bench.
|
2. Bench.
|
||||||
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
|
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
|
||||||
`/etc/systemd/system/` is itself a symlink into the checkout — mask
|
`/etc/systemd/system/` is itself a symlink into the checkout — mask
|
||||||
@@ -287,14 +302,15 @@ The harness reaches the fleet through `.omp/agent/models.yml` in the
|
|||||||
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
||||||
where the harness runs; there is no copy on the servers. Each provider is one
|
where the harness runs; there is no copy on the servers. Each provider is one
|
||||||
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
||||||
the StarCraft unit names (2026-10-08) and since 2026-10-09 they are also the
|
the StarCraft unit names (2026-10-08), on the wire since 2026-10-09, and
|
||||||
ids on the wire:** `frank-archon` → `192.168.0.235:7770`
|
since 2026-10-10 they are also the SERVICE names:** `frank-archon` →
|
||||||
(`FRANK_PRIMARY_SERVED_NAME`), `frank-dragoon` → `192.168.0.143:7770`
|
`192.168.0.235:7770` (`FRANK_ARCHON_SERVED_NAME`), `frank-dragoon` →
|
||||||
(`HALOGEN_MODEL_ID`), `frank-zealot` → `192.168.0.143:7771` (`ALIAS`). The
|
`192.168.0.143:7770` (`HALOGEN_MODEL_ID`), `frank-zealot` →
|
||||||
SERVICES, units, scripts, env files and runbooks keep the old names
|
`192.168.0.143:7771` (`ALIAS`). The 2026-10-10 cutover renamed the units,
|
||||||
(`frank-primary`, `frank-large`, `frank-secondary`); only the served alias,
|
scripts, env files, containers, deploy services and runbooks to match —
|
||||||
the `models.yml`/`config.yml` ids, the homepage cards, the OpenWebUI grants
|
the three former service names are fully retired and exist nowhere in
|
||||||
and the Hermes routing were renamed. Verify a rename on `/v1/models`, never
|
the repo; historical text in runbooks uses the current names. Verify a
|
||||||
|
rename on `/v1/models`, never
|
||||||
on a chat response: radiance echoes the **requested** model name, so a 200
|
on a chat response: radiance echoes the **requested** model name, so a 200
|
||||||
proves nothing about the served id (llama.cpp and halogen ignore it).
|
proves nothing about the served id (llama.cpp and halogen ignore it).
|
||||||
|
|
||||||
@@ -309,16 +325,16 @@ provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
|
|||||||
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
|
`models.yml`, then fix every `modelRoles` entry that named the old endpoint —
|
||||||
a role pointing at a provider the file does not define fails at call time. As
|
a role pointing at a provider the file does not define fails at call time. As
|
||||||
of 2026-10-06 **all three endpoints accept images** (`input: [text,
|
of 2026-10-06 **all three endpoints accept images** (`input: [text,
|
||||||
image]`) — `frank-large` returned to vision with the flash-server v2 swap
|
image]`) — `frank-dragoon` returned to vision with the flash-server v2 swap
|
||||||
(the v2 vision tower loads beside the checkpoint; the text-only halogen
|
(the v2 vision tower loads beside the checkpoint; the text-only halogen
|
||||||
27B window closed 2026-10-06) — while
|
27B window closed 2026-10-06) — while
|
||||||
`frank-primary` takes them natively (radiance runs the checkpoint's
|
`frank-archon` takes them natively (radiance runs the checkpoint's
|
||||||
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
||||||
are the engine core; `--mm-max-patches` available),
|
are the engine core; `--mm-max-patches` available),
|
||||||
and `frank-secondary` through
|
and `frank-zealot` through
|
||||||
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
|
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
|
||||||
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
|
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
|
||||||
Gemma4 era used). `frank-secondary`'s `contextWindow` is
|
Gemma4 era used). `frank-zealot`'s `contextWindow` is
|
||||||
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
|
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
|
||||||
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
|
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
|
||||||
the harness must compact before llama.cpp answers a longer prompt with a 400.
|
the harness must compact before llama.cpp answers a longer prompt with a 400.
|
||||||
@@ -333,20 +349,20 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
|||||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||||
|
|
||||||
### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
|
||||||
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
|
||||||
driven by `scripts/serve-frank-large.sh` under a **root systemd unit**
|
driven by `scripts/serve-frank-dragoon.sh` under a **root systemd unit**
|
||||||
(`scripts/frank-large.service`, `User=dank`, `Restart=always`,
|
(`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
|
||||||
`ExecStop=podman stop -t 60 frank-large-halogen`, the script runs `podman
|
`ExecStop=podman stop -t 60 frank-dragoon-halogen`, the script runs `podman
|
||||||
run` FOREGROUND — the container's exit code is the service's exit so
|
run` FOREGROUND — the container's exit code is the service's exit so
|
||||||
`Restart=always` stays real, and the engine ledger lands on disk) on port
|
`Restart=always` stays real, and the engine ledger lands on disk) on port
|
||||||
7770 from the Strix Halo iGPU. Engine stdout goes to
|
7770 from the Strix Halo iGPU. Engine stdout goes to
|
||||||
`/home/dank/logs/frank_large.log` — **the boot ledger (weights resident, KV
|
`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
|
||||||
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
|
||||||
Cutover 2026-10-06 from the halogen 27B-dense era
|
Cutover 2026-10-06 from the halogen 27B-dense era
|
||||||
([[frank-large-halogen27b-dflash2]], retired the same window).
|
([[frank-dragoon-halogen27b-dflash2]], retired the same window).
|
||||||
|
|
||||||
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
|
||||||
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
|
||||||
@@ -368,10 +384,10 @@ sidecar is deliberately not fetched).
|
|||||||
**Product facts vs the 27B era, both regressions FIXED:** the base is
|
**Product facts vs the 27B era, both regressions FIXED:** the base is
|
||||||
**abliterated** (the 27B was a censored-base finetune; refusals were an
|
**abliterated** (the 27B was a censored-base finetune; refusals were an
|
||||||
accepted trade there) and **vision is ON** (the 27B engine silently
|
accepted trade there) and **vision is ON** (the 27B engine silently
|
||||||
ignored its encoder — frank-secondary owned images for exactly one day at this
|
ignored its encoder — frank-zealot owned images for exactly one day at this
|
||||||
endpoint).
|
endpoint).
|
||||||
|
|
||||||
**Engine laws that shape the config** (all in `scripts/frank-large.env`
|
**Engine laws that shape the config** (all in `scripts/frank-dragoon.env`
|
||||||
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
||||||
policy disables the MTP head for whole requests after low acceptance and
|
policy disables the MTP head for whole requests after low acceptance and
|
||||||
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
||||||
@@ -413,7 +429,7 @@ page cache beside the checkpoint — the card's quickstart arithmetic does
|
|||||||
not bill it as resident, but READ the boot ledger and re-size if the
|
not bill it as resident, but READ the boot ledger and re-size if the
|
||||||
engine prints a clamp.
|
engine prints a clamp.
|
||||||
|
|
||||||
Fetch: `scripts/download-frank-large-flashnext.sh` into
|
Fetch: `scripts/download-frank-dragoon-flashnext.sh` into
|
||||||
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
|
||||||
sha256 + `--verify-only`, no token on the command line; resume goes
|
sha256 + `--verify-only`, no token on the command line; resume goes
|
||||||
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
|
||||||
@@ -422,18 +438,18 @@ n-gram + vision tower, flat tokenizer with all six files as regular
|
|||||||
files) — the 2026-10-01 crash-loop lesson.
|
files) — the 2026-10-01 crash-loop lesson.
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md`.
|
`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md`.
|
||||||
Rollback: revert the swap commit and redeploy — the 27B payload
|
Rollback: revert the swap commit and redeploy — the 27B payload
|
||||||
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
|
||||||
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
||||||
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||||
|
|
||||||
### frank-secondary — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
|
### frank-zealot — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
|
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
|
||||||
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
||||||
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
||||||
(`scripts/frank-secondary.service`, `relink_unit`ed into
|
(`scripts/frank-zealot.service`, `relink_unit`ed into
|
||||||
`/etc/systemd/system/`), serving
|
`/etc/systemd/system/`), serving
|
||||||
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
||||||
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
||||||
@@ -489,7 +505,7 @@ tokens). With `--parallel 2` a compaction request no longer serializes behind
|
|||||||
a live generation (the Gemma4 era's `--parallel 1` law).
|
a live generation (the Gemma4 era's `--parallel 1` law).
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md`.
|
`serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md`.
|
||||||
Rollback: revert the swap commit — the Gemma4 payload
|
Rollback: revert the swap commit — the Gemma4 payload
|
||||||
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
||||||
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
||||||
@@ -497,7 +513,7 @@ Rollback: revert the swap commit — the Gemma4 payload
|
|||||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||||
|
|
||||||
|
|
||||||
### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
||||||
|
|
||||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||||
@@ -524,41 +540,61 @@ the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
|
|||||||
Rollback: revert the swap commit — the fp8-native `.rad`
|
Rollback: revert the swap commit — the fp8-native `.rad`
|
||||||
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
||||||
|
|
||||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 2`,
|
||||||
`--max-model-len 200000` (the harness contextWindow must quote this),
|
`--max-model-len 200000` (the harness contextWindow must quote this),
|
||||||
`--placement expert_tiered --host-pool-mib 40960` + **explicit VRAM
|
`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
|
||||||
partition `--vram-weights-mib 26071 --vram-kv-mib 3543`** (the vram-pair
|
partition `--vram-weights-mib 22208 --vram-kv-mib 6912`** (the vram-pair
|
||||||
replaced `--expert-vs-cache-ratio`; the pair sizes the fp8-KV shape at the
|
replaced `--expert-vs-cache-ratio`; the two are ONE budget —
|
||||||
deployed boot), `--kv-cache-dtype fp8` (inherited from the fp8-native
|
`weights + kv ≤ 29,120 MiB`, from the measured card 31,856 minus 1,710
|
||||||
env), `--num-speculative-tokens 2`,
|
consumed outside both budgets minus 1,024 to keep free), `--kv-cache-dtype`
|
||||||
`--max-num-batched-tokens 2048`, prefix cache 4 GiB pinned host + 64 GiB
|
**`bf16` — unquantized KV since 2026-10-11**, `--num-speculative-tokens 2`,
|
||||||
|
`--max-num-batched-tokens 2048`, prefix cache 8 GiB pinned host + 64 GiB
|
||||||
disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric
|
disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric
|
||||||
v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`,
|
v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`,
|
||||||
`--override-chat-template`). Boot ledger: KV pool 3.46 GiB (~480k fp8
|
`--override-chat-template`). Boot ledger: KV pool 6.75 GiB = **462,752 bf16
|
||||||
tokens), two expert pools 57.84 GiB, cards settled 1.41 GiB free.
|
tokens, "backs 2 sequences all at 200000 tokens at once"** (the fp8 pool's
|
||||||
|
386,904 backed 1.93), weights slab filled to 147 KiB of slack, host pool
|
||||||
|
10,375 slots, cards settled 1.85 GiB free.
|
||||||
|
|
||||||
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
||||||
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
||||||
prefix cache, and the rule that a prompt over `--max-model-len` is
|
prefix cache, and the rule that a prompt over `--max-model-len` is
|
||||||
refused, never truncated. What changed: the fp8-native era's NVMe expert
|
refused, never truncated. What changed: the fp8-native era's NVMe expert
|
||||||
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
|
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
|
||||||
40 GiB pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
|
host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
|
||||||
on the deployed shape (512-token streamed probe). The campaign also
|
on the fp8 shape (512-token streamed probe). The campaign also
|
||||||
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
||||||
accept 0.7246); production shipped the fp8-KV + spec-2 pair at cutover —
|
accept 0.7246).
|
||||||
the KLD gate scored at the fp8-KV shape, so the gate and the boot agree.
|
|
||||||
Disk-tier sizing law: the engine forces ~9.6 GiB of disk-tier weight
|
**The bf16 KV law (2026-10-11)**: `--kv-cache-dtype` takes ONLY `bf16|fp8` —
|
||||||
|
one dtype for the whole cache, so llama.cpp's exact-K/quantized-V pair (what
|
||||||
|
frank-zealot runs) has NO radiance equivalent; the rung under bf16 is fp8 on
|
||||||
|
both halves. Measured per rank from the two boot ledgers: `kv_attn` (13
|
||||||
|
layers, 4-token blocks) pages 26.00 → 52.00 KiB, while `kv_qsa_bkey` (13
|
||||||
|
layers) stays 3.25 KiB at BOTH dtypes (the QSA block key is an index, not
|
||||||
|
attention state) — so a token costs 7.3125 KiB at fp8 and 13.8125 KiB at
|
||||||
|
bf16: **1.89×, not 2×**. Pool sizing must also pay the dtype-independent terms
|
||||||
|
that live in the same pool (~110 MiB a sequence of recurrent state + 448 MiB
|
||||||
|
of prefix checkpoints): `(vram_kv − 110·seqs − 448) MiB / 13.8125 KiB` =
|
||||||
|
462,946 where the engine reported 462,752 (0.04 % page rounding). The cost is
|
||||||
|
paid twice on this engine, and half-refunded: idle KV VRAM is also the expert
|
||||||
|
cache's spillover, and the bigger pool *lends* 6.08 GiB to the slab where the
|
||||||
|
fp8 pool lent 2.69 GiB. `--max-num-seqs 3 → 2` came with the change (two slots
|
||||||
|
× the window is the capacity contract; the "seqs 2 cannot boot" law belongs to
|
||||||
|
the retired tcclaviger fork's torch.compile, not radiance) and it also drops
|
||||||
|
the fused all-reduce to 2 × 3 = 6 rows, under the 25-row bound where wht6 is
|
||||||
|
bit-exact. Disk-tier sizing law: the engine forces ~9.6 GiB of disk-tier weight
|
||||||
buffers + arena onto the weights side, so `weights + kv` is capped near
|
buffers + arena onto the weights side, so `weights + kv` is capped near
|
||||||
29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`.
|
29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`.
|
||||||
|
|
||||||
**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB expert
|
**The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert
|
||||||
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
||||||
the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against
|
the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against
|
||||||
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
||||||
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
||||||
script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to
|
script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=150000`, cap on this host
|
||||||
start until both knobs are raised in the stop window (sysfs write first,
|
204,800) refuses to start until both knobs are raised in the stop window
|
||||||
kernel cmdline + reboot only if read-only).
|
(sysfs write first, kernel cmdline + reboot only if read-only).
|
||||||
|
|
||||||
**Contract facts** (source-verified): the HTTP listener binds AFTER the
|
**Contract facts** (source-verified): the HTTP listener binds AFTER the
|
||||||
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
|
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
|
||||||
@@ -570,11 +606,11 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
|||||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||||
|
|
||||||
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`.
|
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
||||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
|
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
||||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
|
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
||||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
|
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
||||||
|
|
||||||
## Key Lessons Learned
|
## Key Lessons Learned
|
||||||
Durable operational lessons from the fleet's history. Full detail in the
|
Durable operational lessons from the fleet's history. Full detail in the
|
||||||
@@ -736,11 +772,11 @@ linked runbooks.
|
|||||||
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
- Swap resize ordering: create and `swapon` the replacement BEFORE `swapoff`
|
||||||
of the old one, or live pages must land in RAM that `MemAvailable`
|
of the old one, or live pages must land in RAM that `MemAvailable`
|
||||||
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
overstates. Dankclaw swap (64 GiB at `/swap.img`, fstab-driven) protects
|
||||||
the co-tenants, not frank-large — pinned weights don't swap, and paging
|
the co-tenants, not frank-dragoon — pinned weights don't swap, and paging
|
||||||
the KV pool is worse than the OOM it prevents.
|
the KV pool is worse than the OOM it prevents.
|
||||||
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
||||||
`swapon --show` before reasoning about headroom.
|
`swapon --show` before reasoning about headroom.
|
||||||
- **Silent RAM corruption outranks every software theory.** frank-primary's
|
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
||||||
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
||||||
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
||||||
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
|
||||||
@@ -763,7 +799,7 @@ linked runbooks.
|
|||||||
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
`LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
|
||||||
podman to runc without any raise). Upstream halogen's own run lines pass
|
podman to runc without any raise). Upstream halogen's own run lines pass
|
||||||
NO `--ulimit` at all — copy vendor run lines before adding flags
|
NO `--ulimit` at all — copy vendor run lines before adding flags
|
||||||
(measured 2026-10-05 on the frank-large cutover).
|
(measured 2026-10-05 on the frank-dragoon cutover).
|
||||||
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
- **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
|
||||||
`third_party/composable_kernel/include/ck/config.h` generated manually
|
`third_party/composable_kernel/include/ck/config.h` generated manually
|
||||||
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
(untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
|
||||||
@@ -789,7 +825,7 @@ linked runbooks.
|
|||||||
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
|
||||||
Record the child pid and kill it in a `finally`, then prove `free` came back
|
Record the child pid and kill it in a `finally`, then prove `free` came back
|
||||||
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
before believing a number. Kill by pid — never `pkill -f llama-server`,
|
||||||
which also hits frank-secondary.
|
which also hits frank-zealot.
|
||||||
|
|
||||||
**llama.cpp serving**
|
**llama.cpp serving**
|
||||||
|
|
||||||
@@ -813,7 +849,7 @@ linked runbooks.
|
|||||||
builds its context against whatever model the draft context holds. Two
|
builds its context against whatever model the draft context holds. Two
|
||||||
context-backed types cannot share one process, and MTP's head only loads
|
context-backed types cannot share one process, and MTP's head only loads
|
||||||
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
when MTP **is** the draft model. **A LEGAL chain can still lose on a
|
||||||
recurrent model:** `draft-mtp,ngram-mod` on frank-large's qwen35 hybrid
|
recurrent model:** `draft-mtp,ngram-mod` on frank-dragoon's qwen35 hybrid
|
||||||
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
|
||||||
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
logs `recurrent state rollback is not compatible with 'ngram-mod' -
|
||||||
disabling rollback support`, and MTP verification depends on that rollback,
|
disabling rollback support`, and MTP verification depends on that rollback,
|
||||||
@@ -855,7 +891,7 @@ linked runbooks.
|
|||||||
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
|
free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
|
||||||
model — the sweep's or the seam's — can see it.)
|
model — the sweep's or the seam's — can see it.)
|
||||||
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
|
- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
|
||||||
frank-secondary ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
|
frank-zealot ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
|
||||||
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
|
live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
|
||||||
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
|
ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
|
||||||
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
|
2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
|
||||||
@@ -864,9 +900,9 @@ linked runbooks.
|
|||||||
than the ubatch cannot see it. The logical `--batch-size` was again free. And
|
than the ubatch cannot see it. The logical `--batch-size` was again free. And
|
||||||
the predecessor's "ubatch 2048 failed the scratch boot" was a shape
|
the predecessor's "ubatch 2048 failed the scratch boot" was a shape
|
||||||
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
|
observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
|
||||||
it by restarting the SERVICE through a transient `FRANK_SECONDARY_ENV_FILE`
|
it by restarting the SERVICE through a transient `FRANK_ZEALOT_ENV_FILE`
|
||||||
drop-in, never a scratch port — one card holds one server. See
|
drop-in, never a scratch port — one card holds one server. See
|
||||||
[[frank-secondary-llama-mimo-q6k-2x200704]].
|
[[frank-zealot-llama-mimo-q6k-2x200704]].
|
||||||
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
- **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
|
||||||
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
|
size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
|
||||||
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
|
worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
|
||||||
@@ -888,7 +924,7 @@ linked runbooks.
|
|||||||
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
|
CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
|
||||||
(`400 exceed_context_size_error: request (184001 tokens) exceeds the
|
(`400 exceed_context_size_error: request (184001 tokens) exceeds the
|
||||||
available context size (180224 tokens)`) even when the pool has room. So
|
available context size (180224 tokens)`) even when the pool has room. So
|
||||||
frank-secondary's 2 → 3 reslice moved each conversation from the model's native
|
frank-zealot's 2 → 3 reslice moved each conversation from the model's native
|
||||||
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
|
262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
|
||||||
the harness would discover the ceiling as a 400 on a long conversation. Side
|
the harness would discover the ceiling as a 400 on a long conversation. Side
|
||||||
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
|
effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
|
||||||
@@ -899,9 +935,9 @@ linked runbooks.
|
|||||||
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
|
day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
|
||||||
itself is a further measured −10 %, weight-bandwidth, not slot geometry).
|
itself is a further measured −10 %, weight-bandwidth, not slot geometry).
|
||||||
All moves measured; both slot directions cost or buy window, never pool.
|
All moves measured; both slot directions cost or buy window, never pool.
|
||||||
See [[frank-secondary-llama-mimo-q6k-2x200704]].
|
See [[frank-zealot-llama-mimo-q6k-2x200704]].
|
||||||
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
- **Price a VRAM budget from two boots, never from a formula — and an mmproj
|
||||||
costs ~1.5x its file size.** The frank-secondary 3090 fit was solved from two
|
costs ~1.5x its file size.** The frank-zealot 3090 fit was solved from two
|
||||||
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
|
measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
|
||||||
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
|
**non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
|
||||||
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
|
14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
|
||||||
@@ -909,24 +945,24 @@ linked runbooks.
|
|||||||
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
|
ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
|
||||||
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
|
"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
|
||||||
solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
solve `KV ≤ card − non-KV − headroom` and round to a multiple of
|
||||||
`PARALLEL × 4096`. Recorded in `scripts/frank-secondary.env` so the next resize
|
`PARALLEL × 4096`. Recorded in `scripts/frank-zealot.env` so the next resize
|
||||||
starts from measurement.
|
starts from measurement.
|
||||||
- **`images` and `image_url` are NOT interchangeable: only the content part
|
- **`images` and `image_url` are NOT interchangeable: only the content part
|
||||||
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
|
||||||
model still invents an answer if the client used the legacy top-level
|
model still invents an answer if the client used the legacy top-level
|
||||||
`images` field on `/v1/chat/completions` — frank-secondary answered "Brown." for
|
`images` field on `/v1/chat/completions` — frank-zealot answered "Brown." for
|
||||||
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
|
||||||
failure is silent because a hallucinated color is a plausible answer. Probe
|
failure is silent because a hallucinated color is a plausible answer. Probe
|
||||||
vision with a solid color and expect an exact word.
|
vision with a solid color and expect an exact word.
|
||||||
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
||||||
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
||||||
then served frank-large: every tool-using agent on that endpoint was broken
|
then served frank-dragoon: every tool-using agent on that endpoint was broken
|
||||||
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
|
||||||
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
`--jinja` on and the serve script pins it; halogen also accepts `tools` and
|
||||||
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
returns `tool_calls`. Check tool support before routing a role at an endpoint.
|
||||||
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
|
||||||
option and the same ~12x prefill collapse described in the ROCm note above.
|
option and the same ~12x prefill collapse described in the ROCm note above.
|
||||||
f16 KV needs nothing; frank-secondary ran f16 for that reason until
|
f16 KV needs nothing; frank-zealot ran f16 for that reason until
|
||||||
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
2026-09-17, when `build-cuda` was rebuilt with the option ON so the
|
||||||
endpoint could take q8_0 KV.
|
endpoint could take q8_0 KV.
|
||||||
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
- **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
|
||||||
@@ -944,7 +980,7 @@ linked runbooks.
|
|||||||
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
raises on anything else — HTTP 500, not a clamp — so the harness's levels
|
||||||
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
|
||||||
A role whose level is unmapped fails on its FIRST request, and the failure
|
A role whose level is unmapped fails on its FIRST request, and the failure
|
||||||
names Jinja, not the model: every subagent on `frank-secondary` died with
|
names Jinja, not the model: every subagent on `frank-zealot` died with
|
||||||
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
||||||
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
||||||
differ.
|
differ.
|
||||||
@@ -964,7 +1000,7 @@ linked runbooks.
|
|||||||
times (`RuntimeError: No HIP GPUs are available`).
|
times (`RuntimeError: No HIP GPUs are available`).
|
||||||
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
- SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
|
||||||
ports off the gluetun service.
|
ports off the gluetun service.
|
||||||
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-large,
|
- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
|
||||||
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
|
2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind
|
||||||
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
|
needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path
|
||||||
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
|
||||||
@@ -1018,9 +1054,9 @@ Active:
|
|||||||
|
|
||||||
| Service | Runbook |
|
| Service | Runbook |
|
||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-secondary (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, Swift1.5-Qwen3.8-Flash-Next GPTQ w4 + bf16-protected experts, int8 trunk, bf16 KV, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1028,41 +1064,42 @@ Retired (rollback targets and history):
|
|||||||
|
|
||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-secondary`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-secondary-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
|
| radiance StillDeadcode Qwen3.8-Flash-Next native-FP8, 2×R9700 TP=2 (`frank-archon`) | Dankserver | 2026-10-08 → w4dn8 swap (rollback target — `.rad` payload on disk) | `frank-archon-radiance-qwen38-native-fp8.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` |
|
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||||
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-zealot-llama-mimo-q6k-2x200704.md` |
|
||||||
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-secondary`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
|
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-zealot`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||||
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-dragoon-rocmfpx-q8agent-mtp.md` |
|
||||||
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
|
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-dragoon-gufo-qwen38-27b.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-secondary`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-secondary-llama-mimo-q6k-2x200704.md`) | `frank-secondary-llama-mimo-q6k.md` |
|
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-dragoon`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-dragoon-vllm-mimo-v26-bf16.md` |
|
||||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-secondary-llama-mimo-q5km.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-zealot`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-zealot-llama-mimo-q6k-2x200704.md`) | `frank-zealot-llama-mimo-q6k.md` |
|
||||||
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-secondary`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-secondary-sglang-mimo-v26.md` |
|
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-zealot-llama-mimo-q5km.md` |
|
||||||
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
|
| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-zealot`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-zealot-sglang-mimo-v26.md` |
|
||||||
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
|
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-archon`)|Dankserver|2026-09-30 → 10-01|`frank-archon-glm53-flash-w4a16.md`|
|
||||||
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
|
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-21 → 09-28 | `frank-dragoon-unsloth-qwen38-27b-q8xl.md` |
|
||||||
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` |
|
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-archon`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-archon-r9v-radiance-qwen38-uncensored.md` |
|
||||||
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
|
| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-archon`) | Dankserver | 2026-09-26 → 09-27 | `frank-archon-llamacpp-qwen38-flash-next-q8.md` |
|
||||||
|
| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-dragoon`) | Dankclaw | 2026-09-15 → 09-16 | `frank-dragoon-ik-uncensored.md` |
|
||||||
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
|
| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
|
||||||
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-primary`) | Dankserver | 2026-09-26 (same-day swap) | `frank-primary-lucebox-ds4.md` |
|
| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-archon`) | Dankserver | 2026-09-26 (same-day swap) | `frank-archon-lucebox-ds4.md` |
|
||||||
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
|
| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
|
||||||
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
|
| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-dragoon`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
|
||||||
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-large`) | Dankclaw | 2026-09-19 → 09-21 | `frank-large-ciru-orca.md` |
|
| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-dragoon`) | Dankclaw | 2026-09-19 → 09-21 | `frank-dragoon-ciru-orca.md` |
|
||||||
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-unsloth-qwen38-flash-next.md` |
|
| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-unsloth-qwen38-flash-next.md` |
|
||||||
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-kairic-edge-qwen38-27b.md` |
|
| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-kairic-edge-qwen38-27b.md` |
|
||||||
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-secondary`) | Dankclaw | 2026-09-16 → 09-17 | `frank-secondary-qwen38-gsq-rco.md` |
|
| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-zealot`) | Dankclaw | 2026-09-16 → 09-17 | `frank-zealot-qwen38-gsq-rco.md` |
|
||||||
| vLLM Gemma4-12B w4a16 (`frank-secondary`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-secondary-vllm-gemma4.md` |
|
| vLLM Gemma4-12B w4a16 (`frank-zealot`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-zealot-vllm-gemma4.md` |
|
||||||
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-secondary`) | Dankclaw | 2026-09-17 → 10-01 | `frank-secondary-gemma4-12b.md` |
|
| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-zealot`) | Dankclaw | 2026-09-17 → 10-01 | `frank-zealot-gemma4-12b.md` |
|
||||||
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
|
| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
|
||||||
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
|
| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
|
||||||
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
|
| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
|
||||||
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
|
| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
|
||||||
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
|
| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
|
||||||
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
|
| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
|
||||||
| FreeToken (official + Huihui FP8, `frank-secondary`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
|
| FreeToken (official + Huihui FP8, `frank-zealot`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
|
||||||
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
|
| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
|
||||||
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
|
| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
|
||||||
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
|
| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ one deploy script.
|
|||||||
| Host | User | IP | Podman stack | LLM service | LLM port |
|
| Host | User | IP | Podman stack | LLM service | LLM port |
|
||||||
|------|------|----|--------------|-------------|----------|
|
|------|------|----|--------------|-------------|----------|
|
||||||
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
|
||||||
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
|
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
|
||||||
|
|
||||||
## Active Services
|
## Active Services
|
||||||
|
|
||||||
@@ -30,7 +30,7 @@ one deploy script.
|
|||||||
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
|
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
|
||||||
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
|
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
|
||||||
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
|
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
|
||||||
| frank-large | dankclaw | — (bare endpoint) | 7770 |
|
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
|
||||||
|
|
||||||
## Structure
|
## Structure
|
||||||
|
|
||||||
|
|||||||
+9
-9
@@ -1,26 +1,26 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# autoresearch.sh — canonical frank-primary benchmark entry point (dev box).
|
# autoresearch.sh — canonical frank-archon benchmark entry point (dev box).
|
||||||
#
|
#
|
||||||
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
|
# The service runs on dankserver (192.168.0.235:7770), the two R9700s live
|
||||||
# there, and the benchmark must own those cards, so the whole stop -> boot
|
# there, and the benchmark must own those cards, so the whole stop -> boot
|
||||||
# scratch -> bench -> restore sequence runs ON dankserver (see
|
# scratch -> bench -> restore sequence runs ON dankserver (see
|
||||||
# scripts/autoresearch-frank-primary.sh). This wrapper:
|
# scripts/autoresearch-frank-archon.sh). This wrapper:
|
||||||
# 1. copies the candidate config (the repo's scripts/frank-primary.env, the
|
# 1. copies the candidate config (the repo's scripts/frank-archon.env, the
|
||||||
# file the autoresearch loop edits) and the bench script to dankserver,
|
# file the autoresearch loop edits) and the bench script to dankserver,
|
||||||
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
|
# 2. runs the bench over ssh, relaying its stdout (the METRIC lines) and
|
||||||
# exit code back here verbatim.
|
# exit code back here verbatim.
|
||||||
# It deliberately does NOT push to origin: the scp dirties dankserver's
|
# It deliberately does NOT push to origin: the scp dirties dankserver's
|
||||||
# working tree, and the loop's uncommitted candidate config must not be
|
# working tree, and the loop's uncommitted candidate config must not be
|
||||||
# pushed. Only this file and scripts/autoresearch-frank-primary.sh are
|
# pushed. Only this file and scripts/autoresearch-frank-archon.sh are
|
||||||
# committed; the candidate env rides over ssh each run.
|
# committed; the candidate env rides over ssh each run.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REMOTE=dankserver
|
REMOTE=dankserver
|
||||||
REMOTE_REPO=/home/dank/serverAppConfigs
|
REMOTE_REPO=/home/dank/serverAppConfigs
|
||||||
|
|
||||||
scp -q "$DIR/scripts/frank-primary.env" \
|
scp -q "$DIR/scripts/frank-archon.env" \
|
||||||
"$REMOTE:$REMOTE_REPO/scripts/frank-primary.env" || { echo "scp env failed" >&2; exit 1; }
|
"$REMOTE:$REMOTE_REPO/scripts/frank-archon.env" || { echo "scp env failed" >&2; exit 1; }
|
||||||
scp -q "$DIR/scripts/autoresearch-frank-primary.sh" \
|
scp -q "$DIR/scripts/autoresearch-frank-archon.sh" \
|
||||||
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-primary.sh" || { echo "scp script failed" >&2; exit 1; }
|
"$REMOTE:$REMOTE_REPO/scripts/autoresearch-frank-archon.sh" || { echo "scp script failed" >&2; exit 1; }
|
||||||
|
|
||||||
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-primary.sh $REMOTE_REPO/scripts/frank-primary.env"
|
ssh "$REMOTE" "bash $REMOTE_REPO/scripts/autoresearch-frank-archon.sh $REMOTE_REPO/scripts/frank-archon.env"
|
||||||
+24
-27
@@ -56,6 +56,21 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════════════════
|
||||||
|
# Named snippet: the pre-auth guard chain. Anubis (:8923) challenges,
|
||||||
|
# the embedded Authentik outpost authenticates, and authenticated
|
||||||
|
# traffic returns to the http://:8017 listener's post-auth handlers.
|
||||||
|
# Every guarded vhost is `import guarded` — the chain is EDITED HERE,
|
||||||
|
# never inlined per site (rank 6: it used to be nine verbatim copies).
|
||||||
|
# Exemptions live in their own vhosts (auth. is the IdP itself; git
|
||||||
|
# smart-HTTP and self-authenticating services document their reason).
|
||||||
|
# ═══════════════════════════════════════════════════════════════
|
||||||
|
(guarded) {
|
||||||
|
reverse_proxy anubis:8923 {
|
||||||
|
header_up X-Real-Ip {remote_host}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
# --- Bare-IP access (health checks, convenience) ---
|
# --- Bare-IP access (health checks, convenience) ---
|
||||||
:80 {
|
:80 {
|
||||||
redir https://www.franksdank.cloud/ 308
|
redir https://www.franksdank.cloud/ 308
|
||||||
@@ -72,9 +87,7 @@ franksdank.cloud {
|
|||||||
}
|
}
|
||||||
|
|
||||||
www.franksdank.cloud {
|
www.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Legacy name: the old homepage name redirects straight onto www ---
|
# --- Legacy name: the old homepage name redirects straight onto www ---
|
||||||
@@ -100,9 +113,7 @@ auth.franksdank.cloud {
|
|||||||
|
|
||||||
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
|
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
|
||||||
search.franksdank.cloud {
|
search.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
|
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
|
||||||
@@ -126,30 +137,22 @@ gitea.franksdank.cloud {
|
|||||||
# never reach Grafana — RP mode needs none and works regardless of topology.
|
# never reach Grafana — RP mode needs none and works regardless of topology.
|
||||||
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
|
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
|
||||||
grafana.franksdank.cloud {
|
grafana.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Prometheus ---
|
# --- Prometheus ---
|
||||||
prometheus.franksdank.cloud {
|
prometheus.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
|
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
|
||||||
loki.franksdank.cloud {
|
loki.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
|
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
|
||||||
seerr.franksdank.cloud {
|
seerr.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
|
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
|
||||||
@@ -157,9 +160,7 @@ seerr.franksdank.cloud {
|
|||||||
# group gate). Egress rides the searxng gluetun tunnel; reached via the
|
# group gate). Egress rides the searxng gluetun tunnel; reached via the
|
||||||
# host-published port 8082 (gluetun netns), not by container name.
|
# host-published port 8082 (gluetun netns), not by container name.
|
||||||
marreta.franksdank.cloud {
|
marreta.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
||||||
@@ -233,16 +234,12 @@ books.franksdank.cloud {
|
|||||||
|
|
||||||
# --- RomM ---
|
# --- RomM ---
|
||||||
romm.franksdank.cloud {
|
romm.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Copyparty ---
|
# --- Copyparty ---
|
||||||
files.franksdank.cloud {
|
files.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Jellyfin ---
|
# --- Jellyfin ---
|
||||||
|
|||||||
@@ -1,30 +1,32 @@
|
|||||||
# Quadlet unit: node-exporter (rootless, boot-persistent).
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankclaw
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
#
|
#
|
||||||
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
|
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
|
||||||
# Unit name: node-exporter.service (container name: node-exporter)
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
#
|
#
|
||||||
# Host metrics source for Prometheus. This replaces the compose service of the
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
# same name in dankclaw/compose.yaml ("Observability" block).
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
#
|
#
|
||||||
# HOST NAMESPACE. It must run in the host network namespace to report the
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
# host's own interfaces/connections, and --pid=host for the processes and
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
# cgroups collectors. "localhost" inside the container is therefore the host's
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
# loopback, which is what the homepage status timer probes.
|
# loopback, which is what the homepage status timer probes.
|
||||||
#
|
#
|
||||||
# LISTEN ADDRESSES. Until 2026-09-11 this bound :9100 on every interface. It
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
# now binds exactly two: loopback for the homepage status probe, and the LAN
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
# address because Prometheus — which runs in a container on dankclaw_default —
|
# dankclaw_default — scrapes http://192.168.0.143:9100/metrics
|
||||||
# scrapes http://192.168.0.143:9100 (shared/prometheus/prometheus.yml,
|
# (shared/prometheus/prometheus.yml, job "node-dankclaw"). node_exporter has no
|
||||||
# job "node-dankclaw"). node_exporter itself has no authentication, so keeping
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
# the wildcard bind off the table matters: this is the second half of the
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
# pairing that removes the unauthenticated observability surface, and both
|
# host, and both consumers are accounted for.
|
||||||
# consumers are accounted for. Dankserver runs its own copy of this container
|
|
||||||
# (scraped by the "node-dankserver" job) — since 2026-09-16 as a Quadlet unit in
|
|
||||||
# the same shape (dankserver/observability/quadlet/), because the compose-only
|
|
||||||
# version it replaced had been dead for a day and a half before anyone noticed.
|
|
||||||
#
|
#
|
||||||
# MOUNTS. The container reads the host's /proc, /sys and / as /host/* read-only
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
# and is pointed at them with --path.*; it writes nothing anywhere.
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Prometheus node exporter (Dankclaw host metrics)
|
Description=Prometheus node exporter (Dankclaw host metrics)
|
||||||
|
|||||||
+30
-34
@@ -1,47 +1,43 @@
|
|||||||
# A records for the flat franksdank.name scheme (dnsmasq addn-hosts format:
|
# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts
|
||||||
# <IP> <name> [<name> ...]). Served by the Pi-hole quadlet (FTL is a
|
# format: <IP> <name>). RENDERED — do not edit by hand:
|
||||||
# dnsmasq fork): the quadlet mounts this file into /etc/pihole/hosts/ read-only.
|
|
||||||
#
|
#
|
||||||
# Caddy is consolidated on dankclaw (192.168.0.143): it fronts every service
|
# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts
|
||||||
# on both hosts, so every name resolves to that one IP. Keep in sync with the
|
|
||||||
# Caddyfile (tests/pihole.bats cross-checks this).
|
|
||||||
#
|
#
|
||||||
# 192.168.0.143 = dankclaw (runs the Caddy container + the AI/dev stack)
|
# The source of truth is the `dns:` field of shared/services.yml; the
|
||||||
# 192.168.0.235 = dankserver (media stack; reached by Caddy via published
|
# drift gate in tests/service_inventory.bats fails when this file and
|
||||||
# LAN ports — no DNS records point here)
|
# the inventory disagree. Caddy is consolidated on Dankclaw, so every
|
||||||
|
# name — including the services running on dankserver and
|
||||||
|
# frankbook-pro — resolves at the Dankclaw edge.
|
||||||
|
#
|
||||||
|
# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing
|
||||||
|
# comment on a data line becomes a HOSTNAME token. No inline notes
|
||||||
|
# here; the service attribution lives in the inventory.
|
||||||
|
|
||||||
# --- Homepage (naked domain + www) + legacy names ---
|
192.168.0.143 search.franksdank.cloud
|
||||||
|
192.168.0.143 marreta.franksdank.cloud
|
||||||
|
192.168.0.143 pihole.franksdank.cloud
|
||||||
|
192.168.0.143 ai.franksdank.cloud
|
||||||
|
192.168.0.143 gitea.franksdank.cloud
|
||||||
|
192.168.0.143 comfyui.franksdank.cloud
|
||||||
|
192.168.0.143 seerr.franksdank.cloud
|
||||||
|
192.168.0.143 hermes.franksdank.cloud
|
||||||
|
192.168.0.143 hermesadmin.franksdank.cloud
|
||||||
|
192.168.0.143 prometheus.franksdank.cloud
|
||||||
|
192.168.0.143 grafana.franksdank.cloud
|
||||||
|
192.168.0.143 loki.franksdank.cloud
|
||||||
|
192.168.0.143 auth.franksdank.cloud
|
||||||
192.168.0.143 franksdank.cloud
|
192.168.0.143 franksdank.cloud
|
||||||
192.168.0.143 www.franksdank.cloud
|
192.168.0.143 www.franksdank.cloud
|
||||||
192.168.0.143 dankclaw.franksdank.cloud
|
192.168.0.143 dankclaw.franksdank.cloud
|
||||||
192.168.0.143 dankserver.franksdank.cloud
|
192.168.0.143 dankserver.franksdank.cloud
|
||||||
|
|
||||||
# --- Authentik (SSO / OIDC provider) ---
|
|
||||||
192.168.0.143 auth.franksdank.cloud
|
|
||||||
|
|
||||||
# --- Media services (dankserver, proxied) ---
|
|
||||||
192.168.0.143 plex.franksdank.cloud
|
|
||||||
192.168.0.143 plex.franksdank.casa
|
|
||||||
192.168.0.143 books.franksdank.cloud
|
|
||||||
192.168.0.143 romm.franksdank.cloud
|
192.168.0.143 romm.franksdank.cloud
|
||||||
192.168.0.143 files.franksdank.cloud
|
192.168.0.143 files.franksdank.cloud
|
||||||
192.168.0.143 jelly.franksdank.cloud
|
192.168.0.143 jelly.franksdank.cloud
|
||||||
|
192.168.0.143 books.franksdank.cloud
|
||||||
|
192.168.0.143 plex.franksdank.cloud
|
||||||
|
192.168.0.143 plex.franksdank.casa
|
||||||
|
192.168.0.143 prowlarr.franksdank.cloud
|
||||||
|
192.168.0.143 qbittorrent.franksdank.cloud
|
||||||
192.168.0.143 radarr.franksdank.cloud
|
192.168.0.143 radarr.franksdank.cloud
|
||||||
192.168.0.143 lidarr.franksdank.cloud
|
192.168.0.143 lidarr.franksdank.cloud
|
||||||
192.168.0.143 readarr.franksdank.cloud
|
192.168.0.143 readarr.franksdank.cloud
|
||||||
192.168.0.143 prowlarr.franksdank.cloud
|
|
||||||
192.168.0.143 qbittorrent.franksdank.cloud
|
|
||||||
|
|
||||||
# --- AI / search / dev services (dankclaw, local) ---
|
|
||||||
192.168.0.143 search.franksdank.cloud
|
|
||||||
192.168.0.143 comfyui.franksdank.cloud
|
|
||||||
192.168.0.143 ai.franksdank.cloud
|
|
||||||
192.168.0.143 hermes.franksdank.cloud
|
|
||||||
192.168.0.143 hermesadmin.franksdank.cloud
|
|
||||||
192.168.0.143 gitea.franksdank.cloud
|
|
||||||
192.168.0.143 grafana.franksdank.cloud
|
|
||||||
192.168.0.143 prometheus.franksdank.cloud
|
|
||||||
192.168.0.143 loki.franksdank.cloud
|
|
||||||
192.168.0.143 pihole.franksdank.cloud
|
|
||||||
192.168.0.143 seerr.franksdank.cloud
|
|
||||||
192.168.0.143 marreta.franksdank.cloud
|
|
||||||
File renamed without changes.
File renamed without changes.
@@ -1,34 +1,32 @@
|
|||||||
# Quadlet unit: node-exporter (rootless, boot-persistent) — DANKSERVER.
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankserver
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
#
|
#
|
||||||
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
|
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
|
||||||
# Unit name: node-exporter.service (container name: node-exporter)
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
#
|
#
|
||||||
# WHY THIS FILE EXISTS. Until 2026-09-16 the Dankserver node-exporter ran only
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
# from `dankserver/compose.yaml` (the "Observability" block, `restart:
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
# unless-stopped`). Rootless Podman does not restart bare containers after the
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
# explicit stop that shutdown performs, so it died at a reboot and stayed dead:
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
# `podman ps -a` showed it `Exited (143)` and Prometheus' `node-dankserver` job
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
# had been scraping nothing since. The compose entry also bound the default
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
# wildcard address, which is the unauthenticated-metrics-API-on-every-interface
|
|
||||||
# posture that was already removed on Dankclaw. This unit replaces both: one
|
|
||||||
# owner per container name (the compose service is deleted), explicit listen
|
|
||||||
# addresses, and a generator-provided boot path.
|
|
||||||
#
|
#
|
||||||
# HOST NAMESPACE. Networking and PID in the host's namespaces: the collectors
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
# report the host's own interfaces, connections, processes and cgroups. The
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
# host's /proc, /sys and / are mounted read-only under /host and /rootfs, and
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
# node_exporter writes nothing.
|
# loopback, which is what the homepage status timer probes.
|
||||||
#
|
#
|
||||||
# LISTEN ADDRESSES. Exactly two:
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
# - 127.0.0.1:9100 for host-local probes
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
# - 192.168.0.235:9100 for Prometheus, which scrapes this host from Dankclaw
|
# dankclaw_default — scrapes http://192.168.0.235:9100/metrics
|
||||||
# (shared/prometheus/prometheus.yml, job "node-dankserver"). It cannot use
|
# (shared/prometheus/prometheus.yml, job "node-dankserver"). node_exporter has no
|
||||||
# loopback across hosts, and node_exporter has no authentication, so the
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
# wildcard bind stays off the table. This is the same deliberate LAN
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
# listener posture the observability runbook documents.
|
# host, and both consumers are accounted for.
|
||||||
#
|
#
|
||||||
# MIRROR. Keep this file in step with dankclaw/observability/quadlet/
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
# node-exporter.container; only the Description and the LAN address differ.
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Prometheus node exporter (Dankserver host metrics)
|
Description=Prometheus node exporter (Dankserver host metrics)
|
||||||
|
|||||||
+1
-1
@@ -1 +1 @@
|
|||||||
{"model": "frank-large", "prompt": "Below is a telemetry API schema, labeled SCHEMA A. Later in this message the same schema is repeated word for word, labeled SCHEMA B.\nTask: output SCHEMA B verbatim, with no commentary and no reformatting, starting immediately with the opening brace.\n\nSCHEMA A:\n{\"version\":\"2.3.1\",\"endpoints\":[{\"id\":\"ep-000\",\"path\":\"/v2/telemetry/gw5978x49\",\"method\":\"GET\",\"latency_ms\":219,\"owner\":\"team-0\",\"limit\":{\"burst\":221,\"sustained\":149}},{\"id\":\"ep-001\",\"path\":\"/v2/telemetry/gw6279x77\",\"method\":\"POST\",\"latency_ms\":456,\"owner\":\"team-1\",\"limit\":{\"burst\":227,\"sustained\":173}},{\"id\":\"ep-002\",\"path\":\"/v2/telemetry/gw4501x82\",\"method\":\"PUT\",\"latency_ms\":488,\"owner\":\"team-2\",\"limit\":{\"burst\":407,\"sustained\":189}},{\"id\":\"ep-003\",\"path\":\"/v2/telemetry/gw3780xA8\",\"method\":\"DELETE\",\"latency_ms\":90,\"owner\":\"team-3\",\"limit\":{\"burst\":146,\"sustained\":132}},{\"id\":\"ep-004\",\"path\":\"/v2/telemetry/gw6491x17\",\"method\":\"GET\",\"latency_ms\":842,\"owner\":\"team-4\",\"limit\":{\"burst\":488,\"sustained\":193}},{\"id\":\"ep-005\",\"path\":\"/v2/telemetry/gw4848xAC\",\"method\":\"POST\",\"latency_ms\":321,\"owner\":\"team-5\",\"limit\":{\"burst\":116,\"sustained\":51}},{\"id\":\"ep-006\",\"path\":\"/v2/telemetry/gw3414x06\",\"method\":\"PUT\",\"latency_ms\":51,\"owner\":\"team-6\",\"limit\":{\"burst\":126,\"sustained\":122}},{\"id\":\"ep-007\",\"path\":\"/v2/telemetry/gw2193x74\",\"method\":\"DELETE\",\"latency_ms\":428,\"owner\":\"team-0\",\"limit\":{\"burst\":454,\"sustained\":162}},{\"id\":\"ep-008\",\"path\":\"/v2/telemetry/gw4185xB7\",\"method\":\"GET\",\"latency_ms\":717,\"owner\":\"team-1\",\"limit\":{\"burst\":197,\"sustained\":127}},{\"id\":\"ep-009\",\"path\":\"/v2/telemetry/gw7547x3E\",\"method\":\"POST\",\"latency_ms\":155,\"owner\":\"team-2\",\"limit\":{\"burst\":336,\"sustained\":177}},{\"id\":\"ep-010\",\"path\":\"/v2/telemetry/gw1090xE4\",\"method\":\"PUT\",\"latency_ms\":772,\"owner\":\"team-3\",\"limit\":{\"burst\":441,\"sustained\":198}},{\"id\":\"ep-011\",\"path\":\"/v2/telemetry/gw2746xC7\",\"method\":\"DELETE\",\"latency_ms\":439,\"owner\":\"team-4\",\"limit\":{\"burst\":113,\"sustained\":46}},{\"id\":\"ep-012\",\"path\":\"/v2/telemetry/gw9486x76\",\"method\":\"GET\",\"latency_ms\":55,\"owner\":\"team-5\",\"limit\":{\"burst\":286,\"sustained\":64}},{\"id\":\"ep-013\",\"path\":\"/v2/telemetry/gw2988x74\",\"method\":\"POST\",\"latency_ms\":140,\"owner\":\"team-6\",\"limit\":{\"burst\":411,\"sustained\":119}},{\"id\":\"ep-014\",\"path\":\"/v2/telemetry/gw9702xFC\",\"method\":\"PUT\",\"latency_ms\":576,\"owner\":\"team-0\",\"limit\":{\"burst\":305,\"sustained\":82}},{\"id\":\"ep-015\",\"path\":\"/v2/telemetry/gw8251x9C\",\"method\":\"DELETE\",\"latency_ms\":838,\"owner\":\"team-1\",\"limit\":{\"burst\":369,\"sustained\":130}},{\"id\":\"ep-016\",\"path\":\"/v2/telemetry/gw7991xD4\",\"method\":\"GET\",\"latency_ms\":565,\"owner\":\"team-2\",\"limit\":{\"burst\":229,\"sustained\":41}},{\"id\":\"ep-017\",\"path\":\"/v2/telemetry/gw8777x73\",\"method\":\"POST\",\"latency_ms\":269,\"owner\":\"team-3\",\"limit\":{\"burst\":385,\"sustained\":64}},{\"id\":\"ep-018\",\"path\":\"/v2/telemetry/gw5543xC4\",\"method\":\"PUT\",\"latency_ms\":800,\"owner\":\"team-4\",\"limit\":{\"burst\":267,\"sustained\":125}},{\"id\":\"ep-019\",\"path\":\"/v2/telemetry/gw4919x46\",\"method\":\"DELETE\",\"latency_ms\":454,\"owner\":\"team-5\",\"limit\":{\"burst\":40,\"sustained\":183}},{\"id\":\"ep-020\",\"path\":\"/v2/telemetry/gw5681x3C\",\"method\":\"GET\",\"latency_ms\":282,\"owner\":\"team-6\",\"limit\":{\"burst\":172,\"sustained\":82}},{\"id\":\"ep-021\",\"path\":\"/v2/telemetry/gw9849x14\",\"method\":\"POST\",\"latency_ms\":145,\"owner\":\"team-0\",\"limit\":{\"burst\":78,\"sustained\":60}},{\"id\":\"ep-022\",\"path\":\"/v2/telemetry/gw7275xB1\",\"method\":\"PUT\",\"latency_ms\":160,\"owner\":\"team-1\",\"limit\":{\"burst\":362,\"sustained\":55}},{\"id\":\"ep-023\",\"path\":\"/v2/telemetry/gw2052x6A\",\"method\":\"DELETE\",\"latency_ms\":421,\"owner\":\"team-2\",\"limit\":{\"burst\":170,\"sustained\":139}},{\"id\":\"ep-024\",\"path\":\"/v2/telemetry/gw8633x6A\",\"method\":\"GET\",\"latency_ms\":67,\"owner\":\"team-3\",\"limit\":{\"burst\":106,\"sustained\":108}},{\"id\":\"ep-025\",\"path\":\"/v2/telemetry/gw7381xE7\",\"method\":\"POST\",\"latency_ms\":792,\"owner\":\"team-4\",\"limit\":{\"burst\":300,\"sustained\":179}},{\"id\":\"ep-026\",\"path\":\"/v2/telemetry/gw1320xDB\",\"method\":\"PUT\",\"latency_ms\":787,\"owner\":\"team-5\",\"limit\":{\"burst\":295,\"sustained\":98}},{\"id\":\"ep-027\",\"path\":\"/v2/telemetry/gw8814x01\",\"method\":\"DELETE\",\"latency_ms\":364,\"owner\":\"team-6\",\"limit\":{\"burst\":153,\"sustained\":193}},{\"id\":\"ep-028\",\"path\":\"/v2/telemetry/gw7389xDA\",\"method\":\"GET\",\"latency_ms\":859,\"owner\":\"team-0\",\"limit\":{\"burst\":215,\"sustained\":138}},{\"id\"Line truncated
|
{"model": "frank-dragoon", "prompt": "Below is a telemetry API schema, labeled SCHEMA A. Later in this message the same schema is repeated word for word, labeled SCHEMA B.\nTask: output SCHEMA B verbatim, with no commentary and no reformatting, starting immediately with the opening brace.\n\nSCHEMA A:\n{\"version\":\"2.3.1\",\"endpoints\":[{\"id\":\"ep-000\",\"path\":\"/v2/telemetry/gw5978x49\",\"method\":\"GET\",\"latency_ms\":219,\"owner\":\"team-0\",\"limit\":{\"burst\":221,\"sustained\":149}},{\"id\":\"ep-001\",\"path\":\"/v2/telemetry/gw6279x77\",\"method\":\"POST\",\"latency_ms\":456,\"owner\":\"team-1\",\"limit\":{\"burst\":227,\"sustained\":173}},{\"id\":\"ep-002\",\"path\":\"/v2/telemetry/gw4501x82\",\"method\":\"PUT\",\"latency_ms\":488,\"owner\":\"team-2\",\"limit\":{\"burst\":407,\"sustained\":189}},{\"id\":\"ep-003\",\"path\":\"/v2/telemetry/gw3780xA8\",\"method\":\"DELETE\",\"latency_ms\":90,\"owner\":\"team-3\",\"limit\":{\"burst\":146,\"sustained\":132}},{\"id\":\"ep-004\",\"path\":\"/v2/telemetry/gw6491x17\",\"method\":\"GET\",\"latency_ms\":842,\"owner\":\"team-4\",\"limit\":{\"burst\":488,\"sustained\":193}},{\"id\":\"ep-005\",\"path\":\"/v2/telemetry/gw4848xAC\",\"method\":\"POST\",\"latency_ms\":321,\"owner\":\"team-5\",\"limit\":{\"burst\":116,\"sustained\":51}},{\"id\":\"ep-006\",\"path\":\"/v2/telemetry/gw3414x06\",\"method\":\"PUT\",\"latency_ms\":51,\"owner\":\"team-6\",\"limit\":{\"burst\":126,\"sustained\":122}},{\"id\":\"ep-007\",\"path\":\"/v2/telemetry/gw2193x74\",\"method\":\"DELETE\",\"latency_ms\":428,\"owner\":\"team-0\",\"limit\":{\"burst\":454,\"sustained\":162}},{\"id\":\"ep-008\",\"path\":\"/v2/telemetry/gw4185xB7\",\"method\":\"GET\",\"latency_ms\":717,\"owner\":\"team-1\",\"limit\":{\"burst\":197,\"sustained\":127}},{\"id\":\"ep-009\",\"path\":\"/v2/telemetry/gw7547x3E\",\"method\":\"POST\",\"latency_ms\":155,\"owner\":\"team-2\",\"limit\":{\"burst\":336,\"sustained\":177}},{\"id\":\"ep-010\",\"path\":\"/v2/telemetry/gw1090xE4\",\"method\":\"PUT\",\"latency_ms\":772,\"owner\":\"team-3\",\"limit\":{\"burst\":441,\"sustained\":198}},{\"id\":\"ep-011\",\"path\":\"/v2/telemetry/gw2746xC7\",\"method\":\"DELETE\",\"latency_ms\":439,\"owner\":\"team-4\",\"limit\":{\"burst\":113,\"sustained\":46}},{\"id\":\"ep-012\",\"path\":\"/v2/telemetry/gw9486x76\",\"method\":\"GET\",\"latency_ms\":55,\"owner\":\"team-5\",\"limit\":{\"burst\":286,\"sustained\":64}},{\"id\":\"ep-013\",\"path\":\"/v2/telemetry/gw2988x74\",\"method\":\"POST\",\"latency_ms\":140,\"owner\":\"team-6\",\"limit\":{\"burst\":411,\"sustained\":119}},{\"id\":\"ep-014\",\"path\":\"/v2/telemetry/gw9702xFC\",\"method\":\"PUT\",\"latency_ms\":576,\"owner\":\"team-0\",\"limit\":{\"burst\":305,\"sustained\":82}},{\"id\":\"ep-015\",\"path\":\"/v2/telemetry/gw8251x9C\",\"method\":\"DELETE\",\"latency_ms\":838,\"owner\":\"team-1\",\"limit\":{\"burst\":369,\"sustained\":130}},{\"id\":\"ep-016\",\"path\":\"/v2/telemetry/gw7991xD4\",\"method\":\"GET\",\"latency_ms\":565,\"owner\":\"team-2\",\"limit\":{\"burst\":229,\"sustained\":41}},{\"id\":\"ep-017\",\"path\":\"/v2/telemetry/gw8777x73\",\"method\":\"POST\",\"latency_ms\":269,\"owner\":\"team-3\",\"limit\":{\"burst\":385,\"sustained\":64}},{\"id\":\"ep-018\",\"path\":\"/v2/telemetry/gw5543xC4\",\"method\":\"PUT\",\"latency_ms\":800,\"owner\":\"team-4\",\"limit\":{\"burst\":267,\"sustained\":125}},{\"id\":\"ep-019\",\"path\":\"/v2/telemetry/gw4919x46\",\"method\":\"DELETE\",\"latency_ms\":454,\"owner\":\"team-5\",\"limit\":{\"burst\":40,\"sustained\":183}},{\"id\":\"ep-020\",\"path\":\"/v2/telemetry/gw5681x3C\",\"method\":\"GET\",\"latency_ms\":282,\"owner\":\"team-6\",\"limit\":{\"burst\":172,\"sustained\":82}},{\"id\":\"ep-021\",\"path\":\"/v2/telemetry/gw9849x14\",\"method\":\"POST\",\"latency_ms\":145,\"owner\":\"team-0\",\"limit\":{\"burst\":78,\"sustained\":60}},{\"id\":\"ep-022\",\"path\":\"/v2/telemetry/gw7275xB1\",\"method\":\"PUT\",\"latency_ms\":160,\"owner\":\"team-1\",\"limit\":{\"burst\":362,\"sustained\":55}},{\"id\":\"ep-023\",\"path\":\"/v2/telemetry/gw2052x6A\",\"method\":\"DELETE\",\"latency_ms\":421,\"owner\":\"team-2\",\"limit\":{\"burst\":170,\"sustained\":139}},{\"id\":\"ep-024\",\"path\":\"/v2/telemetry/gw8633x6A\",\"method\":\"GET\",\"latency_ms\":67,\"owner\":\"team-3\",\"limit\":{\"burst\":106,\"sustained\":108}},{\"id\":\"ep-025\",\"path\":\"/v2/telemetry/gw7381xE7\",\"method\":\"POST\",\"latency_ms\":792,\"owner\":\"team-4\",\"limit\":{\"burst\":300,\"sustained\":179}},{\"id\":\"ep-026\",\"path\":\"/v2/telemetry/gw1320xDB\",\"method\":\"PUT\",\"latency_ms\":787,\"owner\":\"team-5\",\"limit\":{\"burst\":295,\"sustained\":98}},{\"id\":\"ep-027\",\"path\":\"/v2/telemetry/gw8814x01\",\"method\":\"DELETE\",\"latency_ms\":364,\"owner\":\"team-6\",\"limit\":{\"burst\":153,\"sustained\":193}},{\"id\":\"ep-028\",\"path\":\"/v2/telemetry/gw7389xDA\",\"method\":\"GET\",\"latency_ms\":859,\"owner\":\"team-0\",\"limit\":{\"burst\":215,\"sustained\":138}},{\"idLine truncated
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
|
# download-frank-archon-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
|
||||||
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
|
# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
|
||||||
# vLLM frank-primary deployment on dankserver.
|
# vLLM frank-archon deployment on dankserver.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary-fp8.sh [dest_dir]
|
# bash scripts/download-frank-archon-fp8.sh [dest_dir]
|
||||||
#
|
#
|
||||||
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
|
# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
|
||||||
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
|
# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash
|
# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-dragoon's DFlash
|
||||||
# speculative-decode arm (vLLM --speculative-config method=dflash). Public
|
# speculative-decode arm (vLLM --speculative-config method=dflash). Public
|
||||||
# repo — no auth header on the command line.
|
# repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# the draft for that target.
|
# the draft for that target.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-dflash.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="z-lab/Qwen3.5-9B-DFlash"
|
REPO="z-lab/Qwen3.5-9B-DFlash"
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large
|
# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-dragoon
|
||||||
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
|
# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
|
||||||
# command line (a Bearer token in argv is world-readable via /proc).
|
# command line (a Bearer token in argv is world-readable via /proc).
|
||||||
#
|
#
|
||||||
# Per-file FRESH downloads, no resume: the house rule (see
|
# Per-file FRESH downloads, no resume: the house rule (see
|
||||||
# download-frank-secondary.sh); -f/--fail-with-body plus size+sha256 against the
|
# download-frank-zealot.sh); -f/--fail-with-body plus size+sha256 against the
|
||||||
# Hub LFS oid is what makes an access error fail instead of "succeeding".
|
# Hub LFS oid is what makes an access error fail instead of "succeeding".
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-mimo.sh [DEST_DIR] [--verify-only]
|
||||||
#
|
#
|
||||||
# The verified set below is exactly what the 2026-10-01 bench window served:
|
# The verified set below is exactly what the 2026-10-01 bench window served:
|
||||||
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
|
# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
|
# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
|
||||||
# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the
|
# family's MTP head (15 mtp.* tensors) exists. frank-dragoon runs it as the
|
||||||
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
|
# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
|
||||||
# mtp tensors; that is why SGLang's NEXTN on frank-secondary draws 0.00
|
# mtp tensors; that is why SGLang's NEXTN on frank-zealot draws 0.00
|
||||||
# acceptance). Public repo — no auth header on the command line.
|
# acceptance). Public repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
|
# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-qwen35.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="Qwen/Qwen3.5-9B"
|
REPO="Qwen/Qwen3.5-9B"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-secondary
|
# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-zealot
|
||||||
# (SGLang, RTX 3090). Public repo — no auth header on the command line.
|
# (SGLang, RTX 3090). Public repo — no auth header on the command line.
|
||||||
#
|
#
|
||||||
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
|
# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
# the house rule.
|
# the house rule.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
|
||||||
#
|
#
|
||||||
# LFS shards are verified against the Hub LFS sha256; non-LFS text files
|
# LFS shards are verified against the Hub LFS sha256; non-LFS text files
|
||||||
# against their declared byte size. Idempotent: complete files are skipped.
|
# against their declared byte size. Idempotent: complete files are skipped.
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# bench-frank-large.sh — 64k-context benchmark for the frank-default-large stack.
|
# bench-frank-dragoon.sh — 64k-context benchmark for the frank-default-large stack.
|
||||||
#
|
#
|
||||||
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
|
# Measures generation throughput of Qwen3.6-27B Q8_K_P + DFlash + ngram-mod on
|
||||||
# a single backend at 64k context. It starts llama-server on a scratch port,
|
# a single backend at 64k context. It starts llama-server on a scratch port,
|
||||||
@@ -7,15 +7,15 @@
|
|||||||
# prompt, and prints tokens/sec.
|
# prompt, and prints tokens/sec.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# ./scripts/bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]
|
# ./scripts/bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]
|
||||||
#
|
#
|
||||||
# Example:
|
# Example:
|
||||||
# ./scripts/bench-frank-large.sh ROCm0
|
# ./scripts/bench-frank-dragoon.sh ROCm0
|
||||||
# ./scripts/bench-frank-large.sh Vulkan0 --generations 3
|
# ./scripts/bench-frank-dragoon.sh Vulkan0 --generations 3
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
DEVICE="${1:?usage: bench-frank-large.sh <ROCm0|Vulkan0> [--generations N]}"
|
DEVICE="${1:?usage: bench-frank-dragoon.sh <ROCm0|Vulkan0> [--generations N]}"
|
||||||
GENERATIONS=1
|
GENERATIONS=1
|
||||||
if [ "${2:-}" = "--generations" ]; then
|
if [ "${2:-}" = "--generations" ]; then
|
||||||
GENERATIONS="${3:?--generations needs a count}"
|
GENERATIONS="${3:?--generations needs a count}"
|
||||||
@@ -41,7 +41,7 @@ SERVICE_HEALTH[dankclaw]="http://127.0.0.1:7770/health"
|
|||||||
# Drop page caches before restart: with the model's pages fully cached, the
|
# Drop page caches before restart: with the model's pages fully cached, the
|
||||||
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
|
# KV/graph allocation stalls at load (VRAM stuck at 28 GiB). See AGENTS.md.
|
||||||
DROP_CACHES[dankclaw]=1
|
DROP_CACHES[dankclaw]=1
|
||||||
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-large).
|
# Passwordless restart helper (root-owned, installed via /etc/sudoers.d/frank-dragoon).
|
||||||
# When present, deploy.sh uses it instead of inline sudo commands.
|
# When present, deploy.sh uses it instead of inline sudo commands.
|
||||||
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
|
HELPER[dankclaw]="/usr/local/sbin/frank-restart-large.sh"
|
||||||
# Health wait in iterations of 2s. Cold loads after the cache drop measured
|
# Health wait in iterations of 2s. Cold loads after the cache drop measured
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-large-ling).
|
# Tunables for the Ling-3.0-flash TRAIL on dankclaw (frank-dragoon-ling).
|
||||||
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
|
# This REPLACES frank-default-large (Qwen3.6-27B) temporarily. To switch
|
||||||
# back: systemctl stop frank-large-ling && systemctl start frank-default-large.
|
# back: systemctl stop frank-dragoon-ling && systemctl start frank-default-large.
|
||||||
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
|
# Runtime: llama.cpp FORK bailingmoe3-support; Vulkan + AMD GTT.
|
||||||
#
|
#
|
||||||
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
|
# Memory model (amdgpu Strix Halo gfx1151, 8x16GB LPDDR5 = 128 GiB physical):
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-large-ling TRIAL)
|
Description=llama.cpp Ling-3.0-flash Q6_K Vulkan+MTP (frank-dragoon-ling TRIAL)
|
||||||
After=network.target
|
After=network.target
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
@@ -8,7 +8,7 @@
|
|||||||
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
|
# Vulkan exposes as an ~111 GiB heap). MTP drafter is bundled in the same
|
||||||
# GGUF (block 42) - no separate --spec-draft-model needed.
|
# GGUF (block 42) - no separate --spec-draft-model needed.
|
||||||
#
|
#
|
||||||
# Tie-back: to restore the Qwen3.6-27B service, stop frank-large-ling and
|
# Tie-back: to restore the Qwen3.6-27B service, stop frank-dragoon-ling and
|
||||||
# start frank-default-large (its env/unit are committed and unchanged).
|
# start frank-default-large (its env/unit are committed and unchanged).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-default-large.ling.env in this directory unless
|
# Tunables come from frank-default-large.ling.env in this directory unless
|
||||||
|
|||||||
@@ -4,11 +4,11 @@
|
|||||||
# Runtime: llama.cpp llama-server (>= b10411).
|
# Runtime: llama.cpp llama-server (>= b10411).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-default-large.env in this directory unless the
|
# Tunables come from frank-default-large.env in this directory unless the
|
||||||
# FRANK_LARGE_ENV_FILE environment variable overrides that path.
|
# FRANK_DRAGOON_ENV_FILE environment variable overrides that path.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
+1
-1
@@ -1 +1 @@
|
|||||||
{"model": "frank-large", "prompt": "# Subsurface Aquifer Monitoring Network - System Design Document (Revision 7)\n\nThis document describes the deployment of a subsurface aquifer monitoring network for the Cascadia sedimentary basin. It is written for the operations crew and the regional water authority. Every sensor node, relay, aggregation service, calibration procedure, and failure mode below carries a stable alphanumeric identifier that appears in every table and in the field notes.\n\n## Sensor nodes\nNode SN-000-08B9 sits in a discharge lens at 359.0 m below datum and reports a salinity of 2.38 ppt with the fractured interval reading 5.40 degC at dawn and the wellhead porous 1.72 m3/d while the electrode pair returns 3157 uS/cm.\nNode SN-001-0A65 sits in a percolate lens at 25.9 m below datum and steps down to a salinity of 2.13 ppt with the diapir interval reading 8.55 degC at dawn and the wellhead silty 0.67 m3/d while the electrode pair returns 979 uS/cm.\nNode SN-002-0571 sits in a diapir lens at 287.8 m below datum and peaks at a salinity of 7.63 ppt with the karst interval reading 4.26 degC at dawn and the wellhead spooned 11.50 m3/d while the electrode pair returns 2987 uS/cm.\nNode SN-003-0CE9 sits in a confining lens at 169.5 m below datum and logs a salinity of 0.96 ppt with the alluvial interval reading 5.03 degC at dawn and the wellhead salinized 12.08 m3/d while the electrode pair returns 2607 uS/cm.\nNode SN-004-0E2A sits in a freshened lens at 70.4 m below datum and settles near a salinity of 8.68 ppt with the spooned interval reading 4.88 degC at dawn and the wellhead siphon 4.38 m3/d while the electrode pair returns 2707 uS/cm.\nNode SN-005-0DDD sits in a percolate lens at 44.6 m below datum and reports a salinity of 6.25 ppt with the alluvial interval reading 10.85 degC at dawn and the wellhead brackish 13.99 m3/d while the electrode pair returns 3618 uS/cm.\nNode SN-006-0EFC sits in a till lens at 309.5 m below datum and reports a salinity of 3.49 ppt with the lacustrine interval reading 7.38 degC at dawn and the wellhead spooned 3.21 m3/d while the electrode pair returns 1253 uS/cm.\nNode SN-007-0B02 sits in a recharge lens at 262.0 m below datum and steps down to a salinity of 2.38 ppt with the discharge interval reading 8.18 degC at dawn and the wellhead perched 4.05 m3/d while the electrode pair returns 3900 uS/cm.\nNode SN-008-0448 sits in a karst lens at 406.5 m below datum and reports a salinity of 7.32 ppt with the upconing interval reading 6.17 degC at dawn and the wellhead fluvial 0.75 m3/d while the electrode pair returns 1448 uS/cm.\nNode SN-009-043C sits in a alluvial lens at 159.3 m below datum and flags a salinity of 6.20 ppt with the percolate interval reading 7.60 degC at dawn and the wellhead brackish 13.01 m3/d while the electrode pair returns 2025 uS/cm.\nNode SN-010-0FF5 sits in a karst lens at 264.2 m below datum and flags a salinity of 7.05 ppt with the freshened interval reading 7.88 degC at dawn and the wellhead siphon 8.41 m3/d while the electrode pair returns 1635 uS/cm.\nNode SN-011-098A sits in a drain lens at 365.7 m below datum and reports a salinity of 8.11 ppt with the wedge interval reading 5.51 degC at dawn and the wellhead glacial 2.52 m3/d while the electrode pair returns 2916 uS/cm.\nNode SN-012-0AE8 sits in a till lens at 259.6 m below datum and steps down to a salinity of 9.13 ppt with the recharge interval reading 11.60 degC at dawn and the wellhead spooned 0.46 m3/d while the electrode pair returns 3077 uS/cm.\nNode SN-013-0B83 sits in a salinized lens at 171.2 m below datum and peaks at a salinity of 2.83 ppt with the lacustrine interval reading 5.56 degC at dawn and the wellhead discharge 0.35 m3/d while the electrode pair returns 3083 uS/cm.\nNode SN-014-032E sits in a drain lens at 249.6 m below datum and flags a salinity of 1.09 ppt with the salinized interval reading 9.58 degC at dawn and the wellhead perched 12.00 m3/d while the electrode pair returns 2221 uS/cm.\nNode SN-015-004F sits in a aeolian lens at 264.4 m below datum and reports a salinity of 7.34 ppt with the salinized interval reading 8.76 degC at dawn and the wellhead perched 0.31 m3/d while the electrode pair returns 1483 uS/cm.\nNode SN-016-07C6 sits in a aeolian lens at 155.9 m below datum and reports a salinity of 0.64 ppt with the mineralized interval reading 11.75 degC at dawn and the wellhead diluted 13.46 m3/d while the electrode pair returns 524 uS/cm.\nNode SN-017-06C5 sits in a salinized lens at 72.1 m below datum and averages a salinity of 4.52 ppt with the siphon interval reading 8.93 degC at dawn and the wellhead confining 3.98 m3/d while the electrode pair returns 3687 uS/cm.\nNode SN-018-0E64 sits in a siphon lens at 334.9 m below datum and steps down to a salinity of 6.73 ppt with the salinized interval reading 7.63 degC at dawn and the wellhead freshened 9.64 m3/d while the electrode pair returns 1681 uS/cm.\nNode SN-019-0122 sits in a spooned lens at 1Line truncated
|
{"model": "frank-dragoon", "prompt": "# Subsurface Aquifer Monitoring Network - System Design Document (Revision 7)\n\nThis document describes the deployment of a subsurface aquifer monitoring network for the Cascadia sedimentary basin. It is written for the operations crew and the regional water authority. Every sensor node, relay, aggregation service, calibration procedure, and failure mode below carries a stable alphanumeric identifier that appears in every table and in the field notes.\n\n## Sensor nodes\nNode SN-000-08B9 sits in a discharge lens at 359.0 m below datum and reports a salinity of 2.38 ppt with the fractured interval reading 5.40 degC at dawn and the wellhead porous 1.72 m3/d while the electrode pair returns 3157 uS/cm.\nNode SN-001-0A65 sits in a percolate lens at 25.9 m below datum and steps down to a salinity of 2.13 ppt with the diapir interval reading 8.55 degC at dawn and the wellhead silty 0.67 m3/d while the electrode pair returns 979 uS/cm.\nNode SN-002-0571 sits in a diapir lens at 287.8 m below datum and peaks at a salinity of 7.63 ppt with the karst interval reading 4.26 degC at dawn and the wellhead spooned 11.50 m3/d while the electrode pair returns 2987 uS/cm.\nNode SN-003-0CE9 sits in a confining lens at 169.5 m below datum and logs a salinity of 0.96 ppt with the alluvial interval reading 5.03 degC at dawn and the wellhead salinized 12.08 m3/d while the electrode pair returns 2607 uS/cm.\nNode SN-004-0E2A sits in a freshened lens at 70.4 m below datum and settles near a salinity of 8.68 ppt with the spooned interval reading 4.88 degC at dawn and the wellhead siphon 4.38 m3/d while the electrode pair returns 2707 uS/cm.\nNode SN-005-0DDD sits in a percolate lens at 44.6 m below datum and reports a salinity of 6.25 ppt with the alluvial interval reading 10.85 degC at dawn and the wellhead brackish 13.99 m3/d while the electrode pair returns 3618 uS/cm.\nNode SN-006-0EFC sits in a till lens at 309.5 m below datum and reports a salinity of 3.49 ppt with the lacustrine interval reading 7.38 degC at dawn and the wellhead spooned 3.21 m3/d while the electrode pair returns 1253 uS/cm.\nNode SN-007-0B02 sits in a recharge lens at 262.0 m below datum and steps down to a salinity of 2.38 ppt with the discharge interval reading 8.18 degC at dawn and the wellhead perched 4.05 m3/d while the electrode pair returns 3900 uS/cm.\nNode SN-008-0448 sits in a karst lens at 406.5 m below datum and reports a salinity of 7.32 ppt with the upconing interval reading 6.17 degC at dawn and the wellhead fluvial 0.75 m3/d while the electrode pair returns 1448 uS/cm.\nNode SN-009-043C sits in a alluvial lens at 159.3 m below datum and flags a salinity of 6.20 ppt with the percolate interval reading 7.60 degC at dawn and the wellhead brackish 13.01 m3/d while the electrode pair returns 2025 uS/cm.\nNode SN-010-0FF5 sits in a karst lens at 264.2 m below datum and flags a salinity of 7.05 ppt with the freshened interval reading 7.88 degC at dawn and the wellhead siphon 8.41 m3/d while the electrode pair returns 1635 uS/cm.\nNode SN-011-098A sits in a drain lens at 365.7 m below datum and reports a salinity of 8.11 ppt with the wedge interval reading 5.51 degC at dawn and the wellhead glacial 2.52 m3/d while the electrode pair returns 2916 uS/cm.\nNode SN-012-0AE8 sits in a till lens at 259.6 m below datum and steps down to a salinity of 9.13 ppt with the recharge interval reading 11.60 degC at dawn and the wellhead spooned 0.46 m3/d while the electrode pair returns 3077 uS/cm.\nNode SN-013-0B83 sits in a salinized lens at 171.2 m below datum and peaks at a salinity of 2.83 ppt with the lacustrine interval reading 5.56 degC at dawn and the wellhead discharge 0.35 m3/d while the electrode pair returns 3083 uS/cm.\nNode SN-014-032E sits in a drain lens at 249.6 m below datum and flags a salinity of 1.09 ppt with the salinized interval reading 9.58 degC at dawn and the wellhead perched 12.00 m3/d while the electrode pair returns 2221 uS/cm.\nNode SN-015-004F sits in a aeolian lens at 264.4 m below datum and reports a salinity of 7.34 ppt with the salinized interval reading 8.76 degC at dawn and the wellhead perched 0.31 m3/d while the electrode pair returns 1483 uS/cm.\nNode SN-016-07C6 sits in a aeolian lens at 155.9 m below datum and reports a salinity of 0.64 ppt with the mineralized interval reading 11.75 degC at dawn and the wellhead diluted 13.46 m3/d while the electrode pair returns 524 uS/cm.\nNode SN-017-06C5 sits in a salinized lens at 72.1 m below datum and averages a salinity of 4.52 ppt with the siphon interval reading 8.93 degC at dawn and the wellhead confining 3.98 m3/d while the electrode pair returns 3687 uS/cm.\nNode SN-018-0E64 sits in a siphon lens at 334.9 m below datum and steps down to a salinity of 6.73 ppt with the salinized interval reading 7.63 degC at dawn and the wellhead freshened 9.64 m3/d while the electrode pair returns 1681 uS/cm.\nNode SN-019-0122 sits in a spooned lens atLine truncated
|
||||||
File renamed without changes.
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
Decode throughput benchmark for frank-large.
|
Decode throughput benchmark for frank-dragoon.
|
||||||
Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
|
Starts the server with a given --threads-batch value, sends an 8K+ token prompt,
|
||||||
generates 256 tokens, measures generation tok/s.
|
generates 256 tokens, measures generation tok/s.
|
||||||
|
|
||||||
@@ -13,12 +13,12 @@ SERVE = "/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server"
|
|||||||
PORT = 7770
|
PORT = 7770
|
||||||
N_GEN = 256
|
N_GEN = 256
|
||||||
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
|
PROMPT_FILE = "/tmp/bench-prompts/prompt_code.txt"
|
||||||
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-large.env")
|
ENV_FILE = os.path.join(os.path.dirname(__file__), "frank-dragoon.env")
|
||||||
SERVER_DIR = os.path.dirname(SERVE)
|
SERVER_DIR = os.path.dirname(SERVE)
|
||||||
ROCM_PATH = "/opt/rocm"
|
ROCM_PATH = "/opt/rocm"
|
||||||
|
|
||||||
def make_env():
|
def make_env():
|
||||||
"""Build the same env as serve-frank-large.sh."""
|
"""Build the same env as serve-frank-dragoon.sh."""
|
||||||
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
|
env = {**os.environ, "HSA_OVERRIDE_GFX_VERSION": "11.5.1", "HIP_VISIBLE_DEVICES": "0"}
|
||||||
|
|
||||||
# Source env file for tunables
|
# Source env file for tunables
|
||||||
@@ -59,7 +59,7 @@ def wait_ready():
|
|||||||
def send_request(prompt, max_tokens=N_GEN):
|
def send_request(prompt, max_tokens=N_GEN):
|
||||||
"""Send chat completion, measure generation throughput."""
|
"""Send chat completion, measure generation throughput."""
|
||||||
body = json.dumps({
|
body = json.dumps({
|
||||||
"model": "frank-large",
|
"model": "frank-dragoon",
|
||||||
"messages": [{"role": "user", "content": prompt}],
|
"messages": [{"role": "user", "content": prompt}],
|
||||||
"max_tokens": max_tokens,
|
"max_tokens": max_tokens,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
Benchmark n-gram speculative decoding vs no-spec for frank-large.
|
Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
|
||||||
|
|
||||||
Tests prompt-processing throughput with 16K token prompts (manageable request size).
|
Tests prompt-processing throughput with 16K token prompts (manageable request size).
|
||||||
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
|
Code-heavy prompt (high n-gram redundancy) vs prose prompt (low redundancy).
|
||||||
@@ -103,7 +103,7 @@ def start_server(port, spec_type):
|
|||||||
def run_test(prompt_text, spec_type, port):
|
def run_test(prompt_text, spec_type, port):
|
||||||
"""Send a request and extract prompt processing metrics."""
|
"""Send a request and extract prompt processing metrics."""
|
||||||
data = json.dumps({
|
data = json.dumps({
|
||||||
"model": "frank-large",
|
"model": "frank-dragoon",
|
||||||
"messages": [{"role": "user", "content": prompt_text}],
|
"messages": [{"role": "user", "content": prompt_text}],
|
||||||
"max_tokens": TEST_TOKENS,
|
"max_tokens": TEST_TOKENS,
|
||||||
"stream": False,
|
"stream": False,
|
||||||
@@ -189,7 +189,7 @@ def main():
|
|||||||
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
|
print(f" Prose: {prose_prompt_file.stat().st_size:,} bytes")
|
||||||
|
|
||||||
print("=" * 60)
|
print("=" * 60)
|
||||||
print(" N-gram Mod Benchmark — frank-large")
|
print(" N-gram Mod Benchmark — frank-dragoon")
|
||||||
print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
|
print(f" STRIX ROCmFP4 · 64K context · 16K prompts")
|
||||||
print("=" * 60)
|
print("=" * 60)
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,19 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Benchmark n-gram speculative decoding vs no-spec for frank-large.
|
# Benchmark n-gram speculative decoding vs no-spec for frank-dragoon.
|
||||||
#
|
#
|
||||||
# This tests prompt-processing throughput at 64k context with two distinct
|
# This tests prompt-processing throughput at 64k context with two distinct
|
||||||
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
|
# prompts: one code-heavy and one prose-heavy. Uses llama-server on scratch
|
||||||
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
|
# ports with the OpenAI-compatible API to measure prompt_eval_duration_ms.
|
||||||
#
|
#
|
||||||
# Run on Dankclaw after stopping the production service:
|
# Run on Dankclaw after stopping the production service:
|
||||||
# sudo systemctl stop frank-large
|
# sudo systemctl stop frank-dragoon
|
||||||
# ./scripts/bench-ngram-mod.sh
|
# ./scripts/bench-ngram-mod.sh
|
||||||
# sudo systemctl start frank-large
|
# sudo systemctl start frank-dragoon
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a; source "$ENV_FILE"; set +a
|
set -a; source "$ENV_FILE"; set +a
|
||||||
fi
|
fi
|
||||||
@@ -184,7 +184,7 @@ run_single_test() {
|
|||||||
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
|
response=$(curl -s -X POST "http://127.0.0.1:$port/v1/completions" \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d "{
|
-d "{
|
||||||
\"model\": \"frank-large\",
|
\"model\": \"frank-dragoon\",
|
||||||
\"prompt\": \"$prompt_text\",
|
\"prompt\": \"$prompt_text\",
|
||||||
\"max_tokens\": 128,
|
\"max_tokens\": 128,
|
||||||
\"temperature\": 0,
|
\"temperature\": 0,
|
||||||
@@ -220,7 +220,7 @@ run_single_test() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
echo "╔══════════════════════════════════════════════════════════╗"
|
echo "╔══════════════════════════════════════════════════════════╗"
|
||||||
echo "║ N-gram Mod Benchmark — frank-large ║"
|
echo "║ N-gram Mod Benchmark — frank-dragoon ║"
|
||||||
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
|
echo "║ STRIX ROCmFP4 · 64k context · two prompts ║"
|
||||||
echo "╚══════════════════════════════════════════════════════════╝"
|
echo "╚══════════════════════════════════════════════════════════╝"
|
||||||
echo ""
|
echo ""
|
||||||
|
|||||||
@@ -15,9 +15,9 @@ import time
|
|||||||
import urllib.request
|
import urllib.request
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
SERVICE = "frank-large"
|
SERVICE = "frank-dragoon"
|
||||||
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-large.env"
|
ENV_FILE = "/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ALIAS = "frank-large"
|
ALIAS = "frank-dragoon"
|
||||||
DEFAULT_HOST = "localhost"
|
DEFAULT_HOST = "localhost"
|
||||||
DEFAULT_PORT = 7770
|
DEFAULT_PORT = 7770
|
||||||
DEFAULT_MAX_TOKENS = 16384
|
DEFAULT_MAX_TOKENS = 16384
|
||||||
|
|||||||
+2
-2
@@ -1,4 +1,4 @@
|
|||||||
# Tunables for frank-large (mradermacher i1-Q4_K_M of
|
# Tunables for frank-dragoon (mradermacher i1-Q4_K_M of
|
||||||
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
|
# orcarouter/Qwen3.8-Flash-Next-Uncensored on ik_llama.cpp, Dankclaw,
|
||||||
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
|
# RTX 3090 CUDA + Strix Halo CPU path, since 2026-09-15).
|
||||||
#
|
#
|
||||||
@@ -65,7 +65,7 @@ MMPROJ_PATH=/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K
|
|||||||
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
||||||
|
|
||||||
# Alias + network.
|
# Alias + network.
|
||||||
ALIAS=frank-large
|
ALIAS=frank-dragoon
|
||||||
BIND_ADDR=0.0.0.0
|
BIND_ADDR=0.0.0.0
|
||||||
LISTEN_PORT=7770
|
LISTEN_PORT=7770
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-large)
|
Description=ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M 177B MoE (dankclaw, frank-dragoon)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -12,8 +12,8 @@ Group=dank
|
|||||||
# budget still needs room. Parity with the halogen quadlet's
|
# budget still needs room. Parity with the halogen quadlet's
|
||||||
# Ulimit=memlock=-1:-1.
|
# Ulimit=memlock=-1:-1.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
+3
-3
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-large
|
# Serve mradermacher/Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M as frank-dragoon
|
||||||
# via ik_llama.cpp on Dankclaw.
|
# via ik_llama.cpp on Dankclaw.
|
||||||
#
|
#
|
||||||
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
|
# Topology (ikawrakow's hybrid recipe; upstream issue #2402 for the PLE):
|
||||||
@@ -26,7 +26,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -66,7 +66,7 @@ fi
|
|||||||
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
|
# CPU-overridden weights mmapped (src/llama-load-tensors.cpp); the env file
|
||||||
# sets it. Refuse to run without it.
|
# sets it. Refuse to run without it.
|
||||||
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
|
if [[ "${GGML_CUDA_NO_PINNED_WEIGHTS:-}" != "1" ]]; then
|
||||||
echo "frank-large: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
|
echo "frank-dragoon: refusing to start — GGML_CUDA_NO_PINNED_WEIGHTS=1 is mandatory on this host" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -112,7 +112,7 @@ fi
|
|||||||
: "${SPEC_DRAFT_POLL_BATCH:=1}"
|
: "${SPEC_DRAFT_POLL_BATCH:=1}"
|
||||||
|
|
||||||
# Text only: the server must not try to autoload the vision tower. Vision
|
# Text only: the server must not try to autoload the vision tower. Vision
|
||||||
# stays on frank-large (Dankclaw), which is where the fleet routes images.
|
# stays on frank-dragoon (Dankclaw), which is where the fleet routes images.
|
||||||
NO_MMPROJ="${NO_MMPROJ:-1}"
|
NO_MMPROJ="${NO_MMPROJ:-1}"
|
||||||
|
|
||||||
export HIP_VISIBLE_DEVICES
|
export HIP_VISIBLE_DEVICES
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""autoresearch-bench.py — deterministic greedy battery for frank-large.
|
"""autoresearch-bench.py — deterministic greedy battery for frank-dragoon.
|
||||||
|
|
||||||
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
|
Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
|
||||||
Fixed embedded corpus, temperature 0, streaming so decode speed is measured
|
Fixed embedded corpus, temperature 0, streaming so decode speed is measured
|
||||||
|
|||||||
@@ -1,12 +1,12 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# autoresearch-frank-primary.sh — RUNS ON DANKSERVER, driven over ssh by the
|
# autoresearch-frank-archon.sh — RUNS ON DANKSERVER, driven over ssh by the
|
||||||
# dev-box autoresearch.sh. It is the canonical frank-primary benchmark entry
|
# dev-box autoresearch.sh. It is the canonical frank-archon benchmark entry
|
||||||
# point for the autoresearch loop.
|
# point for the autoresearch loop.
|
||||||
#
|
#
|
||||||
# Method (mandated by the Benchmarking Policy): the production service serves
|
# Method (mandated by the Benchmarking Policy): the production service serves
|
||||||
# real users and shares the two R9700s, so a bench beside it starves the GPUs
|
# real users and shares the two R9700s, so a bench beside it starves the GPUs
|
||||||
# and corrupts results. This script therefore:
|
# and corrupts results. This script therefore:
|
||||||
# 1. stops the production frank-primary unit (root, relink symlink: stop,
|
# 1. stops the production frank-archon unit (root, relink symlink: stop,
|
||||||
# NOT mask — mask refuses to replace a symlinked unit file),
|
# NOT mask — mask refuses to replace a symlinked unit file),
|
||||||
# 2. boots a scratch radiance container named `arbench` on the SAME two
|
# 2. boots a scratch radiance container named `arbench` on the SAME two
|
||||||
# cards with the candidate engine argv (the env file the loop edits),
|
# cards with the candidate engine argv (the env file the loop edits),
|
||||||
@@ -18,7 +18,7 @@
|
|||||||
# prefill and decode rates for that single request.
|
# prefill and decode rates for that single request.
|
||||||
# 5. prints METRIC lines and tears the scratch down,
|
# 5. prints METRIC lines and tears the scratch down,
|
||||||
# 6. restores production (systemctl start), via a trap so a failed bench
|
# 6. restores production (systemctl start), via a trap so a failed bench
|
||||||
# never leaves frank-primary down (the stale-probe-tail lesson).
|
# never leaves frank-archon down (the stale-probe-tail lesson).
|
||||||
#
|
#
|
||||||
# Determinism: the prompt is built from a fixed, sorted corpus of repo files
|
# Determinism: the prompt is built from a fixed, sorted corpus of repo files
|
||||||
# with a fixed random seed, so prompt_n is stable across runs. Sampling is
|
# with a fixed random seed, so prompt_n is stable across runs. Sampling is
|
||||||
@@ -32,10 +32,10 @@
|
|||||||
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
|
# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
|
||||||
# differences: the container name (arbench, so prod's identity is untouched)
|
# differences: the container name (arbench, so prod's identity is untouched)
|
||||||
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
|
# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
|
||||||
# prefill is cold — the prod tier at kvcache/frank-primary is left intact.
|
# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-primary.env}"
|
ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
|
||||||
SCRATCH=arbench
|
SCRATCH=arbench
|
||||||
PORT=7770
|
PORT=7770
|
||||||
HEALTH="http://127.0.0.1:$PORT/health"
|
HEALTH="http://127.0.0.1:$PORT/health"
|
||||||
@@ -45,7 +45,7 @@ LOG=/home/dank/logs/arbench.log
|
|||||||
|
|
||||||
restore() {
|
restore() {
|
||||||
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
||||||
sudo systemctl start frank-primary >/dev/null 2>&1 || true
|
sudo systemctl start frank-archon >/dev/null 2>&1 || true
|
||||||
}
|
}
|
||||||
trap restore EXIT
|
trap restore EXIT
|
||||||
|
|
||||||
@@ -57,13 +57,13 @@ fi
|
|||||||
set -a; . "$ENV_FILE"; set +a
|
set -a; . "$ENV_FILE"; set +a
|
||||||
|
|
||||||
# --- stop production; the scratch needs both cards to itself ----------------
|
# --- stop production; the scratch needs both cards to itself ----------------
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
|
# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
|
||||||
FRESH_KV="$FRANK_PRIMARY_STATE_DIR/kvcache/$SCRATCH"
|
FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
|
||||||
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
|
rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
|
|
||||||
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
# --- boot the scratch: the production run line, scratch name + fresh KV ------
|
||||||
podman run -d --name "$SCRATCH" \
|
podman run -d --name "$SCRATCH" \
|
||||||
@@ -72,31 +72,31 @@ podman run -d --name "$SCRATCH" \
|
|||||||
--security-opt seccomp=unconfined \
|
--security-opt seccomp=unconfined \
|
||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device /dev/kfd --device /dev/dri \
|
--device /dev/kfd --device /dev/dri \
|
||||||
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR":/data \
|
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
"$FRANK_PRIMARY_IMAGE" \
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
|
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
--host :: --port "$PORT" \
|
--host :: --port "$PORT" \
|
||||||
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
|
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
|
||||||
--override-chat-template /templates/template.jinja \
|
--override-chat-template /templates/template.jinja \
|
||||||
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
|
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
|
||||||
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
|
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
|
||||||
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
|
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
|
||||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
||||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
||||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
||||||
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
||||||
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
|
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
||||||
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
|
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
|
if [ $? -ne 0 ]; then echo "scratch boot failed; see $LOG" >&2; exit 1; fi
|
||||||
|
|
||||||
+32
-32
@@ -1,19 +1,19 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# frank-primary-arms.sh — run a SEQUENCE of frank-primary bench arms and keep
|
# frank-archon-arms.sh — run a SEQUENCE of frank-archon bench arms and keep
|
||||||
# every arm's evidence. Runs on the operator box; each arm is one full window
|
# every arm's evidence. Runs on the operator box; each arm is one full window
|
||||||
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
|
# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
|
||||||
# restore production and verify), so the fleet is only ever down for one arm at
|
# restore production and verify), so the fleet is only ever down for one arm at
|
||||||
# a time and the production unit is back up between arms.
|
# a time and the production unit is back up between arms.
|
||||||
#
|
#
|
||||||
# An arm is the repo's own frank-primary.env plus an OVERLAY of engine knobs
|
# An arm is the repo's own frank-archon.env plus an OVERLAY of engine knobs
|
||||||
# appended after it (later assignment wins when the serve script sources it),
|
# appended after it (later assignment wins when the serve script sources it),
|
||||||
# staged through FRANK_PRIMARY_AR_ENV_FILE so the checkout is never mutated and
|
# staged through FRANK_ARCHON_AR_ENV_FILE so the checkout is never mutated and
|
||||||
# each arm gets its own fingerprint (the memo can never answer one arm with
|
# each arm gets its own fingerprint (the memo can never answer one arm with
|
||||||
# another arm's numbers).
|
# another arm's numbers).
|
||||||
#
|
#
|
||||||
# Usage: scripts/autoresearch/frank-primary-arms.sh [arm ...]
|
# Usage: scripts/autoresearch/frank-archon-arms.sh [arm ...]
|
||||||
# scripts/autoresearch/frank-primary-arms.sh --list
|
# scripts/autoresearch/frank-archon-arms.sh --list
|
||||||
# Output, under /tmp/frank-primary-arms-<UTC stamp>/ (override with ARMS_DIR):
|
# Output, under /tmp/frank-archon-arms-<UTC stamp>/ (override with ARMS_DIR):
|
||||||
#
|
#
|
||||||
# <arm>.out full autoresearch output (metrics + engine evidence)
|
# <arm>.out full autoresearch output (metrics + engine evidence)
|
||||||
# <arm>.env the exact staged env
|
# <arm>.env the exact staged env
|
||||||
@@ -26,8 +26,8 @@ set -uo pipefail
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
|
REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
|
||||||
BASE_ENV="$REPO/scripts/frank-primary.env"
|
BASE_ENV="$REPO/scripts/frank-archon.env"
|
||||||
ARMS_DIR="${ARMS_DIR:-/tmp/frank-primary-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
|
ARMS_DIR="${ARMS_DIR:-/tmp/frank-archon-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
|
||||||
|
|
||||||
# --- arms ------------------------------------------------------------------
|
# --- arms ------------------------------------------------------------------
|
||||||
# baseline: the shipped shape E, measured with THIS harness so the comparison is
|
# baseline: the shipped shape E, measured with THIS harness so the comparison is
|
||||||
@@ -45,18 +45,18 @@ LADDER3='{"cudagraph_capture_sizes":[4,8,12],"max_cudagraph_capture_size":12,"in
|
|||||||
# the engine as {temperature:0.7}.
|
# the engine as {temperature:0.7}.
|
||||||
GEN07='{"temperature":0.7}'
|
GEN07='{"temperature":0.7}'
|
||||||
OVER_B=(
|
OVER_B=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
)
|
)
|
||||||
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
|
# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
|
||||||
# OFF. It is the fallback if B dies at graph capture, and it prices the arena
|
# OFF. It is the fallback if B dies at graph capture, and it prices the arena
|
||||||
# that precise memory buys (7,522 slots -> ~6,496).
|
# that precise memory buys (7,522 slots -> ~6,496).
|
||||||
OVER_B2=(
|
OVER_B2=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
'FRANK_PRIMARY_PRECISE_MEMORY='
|
'FRANK_ARCHON_PRECISE_MEMORY='
|
||||||
)
|
)
|
||||||
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
|
# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
|
||||||
# 0.95. This is also the direct test of the theory that pinning the capture
|
# 0.95. This is also the direct test of the theory that pinning the capture
|
||||||
@@ -64,12 +64,12 @@ OVER_B2=(
|
|||||||
# the constant 3, which is NOT in the default ladder; 4 is).
|
# the constant 3, which is NOT in the default ladder; 4 is).
|
||||||
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
|
LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
|
||||||
OVER_C=(
|
OVER_C=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER16'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER16'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
'FRANK_PRIMARY_SEQS=16'
|
'FRANK_ARCHON_SEQS=16'
|
||||||
'FRANK_PRIMARY_MNBT=4096'
|
'FRANK_ARCHON_MNBT=4096'
|
||||||
'FRANK_PRIMARY_UTIL=0.95'
|
'FRANK_ARCHON_UTIL=0.95'
|
||||||
)
|
)
|
||||||
|
|
||||||
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
|
# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
|
||||||
@@ -83,11 +83,11 @@ OVER_C=(
|
|||||||
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
|
# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
|
||||||
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
|
# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
|
||||||
# batch shape.
|
# batch shape.
|
||||||
KV401K='FRANK_PRIMARY_KV_BYTES=4050000000'
|
KV401K='FRANK_ARCHON_KV_BYTES=4050000000'
|
||||||
OVER_D=(
|
OVER_D=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
"$KV401K"
|
"$KV401K"
|
||||||
)
|
)
|
||||||
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
|
# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
|
||||||
@@ -97,12 +97,12 @@ OVER_D=(
|
|||||||
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
|
# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
|
||||||
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
|
# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
|
||||||
OVER_E=(
|
OVER_E=(
|
||||||
"FRANK_PRIMARY_SPEC='$SPEC3'"
|
"FRANK_ARCHON_SPEC='$SPEC3'"
|
||||||
"FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
|
"FRANK_ARCHON_COMPCONFIG='$LADDER3'"
|
||||||
'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
|
'FRANK_ARCHON_GEN_CONFIG='"'$GEN07'"
|
||||||
"$KV401K"
|
"$KV401K"
|
||||||
'FRANK_PRIMARY_MNBT=4096'
|
'FRANK_ARCHON_MNBT=4096'
|
||||||
'FRANK_PRIMARY_UTIL=0.99'
|
'FRANK_ARCHON_UTIL=0.99'
|
||||||
)
|
)
|
||||||
|
|
||||||
arm_overlay() {
|
arm_overlay() {
|
||||||
@@ -142,7 +142,7 @@ for arm in "${ARMS[@]}"; do
|
|||||||
# holds JSON must still be one valid JSON object after the file is SOURCED
|
# holds JSON must still be one valid JSON object after the file is SOURCED
|
||||||
# (that is the step that silently eats unquoted braces).
|
# (that is the step that silently eats unquoted braces).
|
||||||
if ! bash -c 'set -a; . "$1" >/dev/null; set +a
|
if ! bash -c 'set -a; . "$1" >/dev/null; set +a
|
||||||
for k in FRANK_PRIMARY_SPEC FRANK_PRIMARY_COMPCONFIG FRANK_PRIMARY_GEN_CONFIG; do
|
for k in FRANK_ARCHON_SPEC FRANK_ARCHON_COMPCONFIG FRANK_ARCHON_GEN_CONFIG; do
|
||||||
v="${!k:-}"
|
v="${!k:-}"
|
||||||
[ -n "$v" ] || continue
|
[ -n "$v" ] || continue
|
||||||
case "$v" in
|
case "$v" in
|
||||||
@@ -158,7 +158,7 @@ for arm in "${ARMS[@]}"; do
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
|
echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
|
||||||
FRANK_PRIMARY_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
|
FRANK_ARCHON_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
|
||||||
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
|
"$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
|
||||||
rc=$?
|
rc=$?
|
||||||
echo "$rc" >"$ARMS_DIR/$arm.rc"
|
echo "$rc" >"$ARMS_DIR/$arm.rc"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""frank-primary decode measurement — runs INSIDE the engine container.
|
"""frank-archon decode measurement — runs INSIDE the engine container.
|
||||||
|
|
||||||
The method is the fleet's established one (the 2026-09-29 / 2026-10-01
|
The method is the fleet's established one (the 2026-09-29 / 2026-10-01
|
||||||
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
|
tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
|
||||||
@@ -28,7 +28,7 @@ Workload (fixed, deterministic, offline):
|
|||||||
|
|
||||||
Env:
|
Env:
|
||||||
AR_BASE endpoint (default http://127.0.0.1:7770)
|
AR_BASE endpoint (default http://127.0.0.1:7770)
|
||||||
AR_MODEL served name (default frank-primary)
|
AR_MODEL served name (default frank-archon)
|
||||||
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
|
AR_PROMPT_CEILING hard prompt ceiling in tokens (default 100000)
|
||||||
AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
|
AR_TEMPS comma-separated sampling temperatures (default "0,0.7")
|
||||||
|
|
||||||
+14
-14
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# frank-primary-window.sh — one full bench window for frank-primary, run ON
|
# frank-archon-window.sh — one full bench window for frank-archon, run ON
|
||||||
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
|
# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
|
||||||
#
|
#
|
||||||
# Discipline (AGENTS.md Benchmarking Policy): the production unit is
|
# Discipline (AGENTS.md Benchmarking Policy): the production unit is
|
||||||
@@ -12,9 +12,9 @@
|
|||||||
#
|
#
|
||||||
# Staging layout (created by the orchestrator, ephemeral — never a config
|
# Staging layout (created by the orchestrator, ephemeral — never a config
|
||||||
# source; the repo checkout is):
|
# source; the repo checkout is):
|
||||||
# /home/dank/.autoresearch-frank-primary/
|
# /home/dank/.autoresearch-frank-archon/
|
||||||
# serve.sh staged candidate serve script
|
# serve.sh staged candidate serve script
|
||||||
# frank-primary.env staged candidate env (the candidate knobs)
|
# frank-archon.env staged candidate env (the candidate knobs)
|
||||||
# measure.py the in-container measure client
|
# measure.py the in-container measure client
|
||||||
# hash candidate fingerprint
|
# hash candidate fingerprint
|
||||||
# run.json result (metrics + boot + kv + fingerprint)
|
# run.json result (metrics + boot + kv + fingerprint)
|
||||||
@@ -27,16 +27,16 @@
|
|||||||
# outcome; a failed restore forces exit 1).
|
# outcome; a failed restore forces exit 1).
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}"
|
STAGE="${FRANK_ARCHON_AR_STAGE:-/home/dank/.autoresearch-frank-archon}"
|
||||||
UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}"
|
UNIT="${FRANK_ARCHON_AR_UNIT:-frank-archon}"
|
||||||
CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}"
|
CTR="${FRANK_ARCHON_AR_CTR:-frank-archon-qwen}"
|
||||||
HEALTH_URL="http://127.0.0.1:7770/health"
|
HEALTH_URL="http://127.0.0.1:7770/health"
|
||||||
MODELS_URL="http://127.0.0.1:7770/v1/models"
|
MODELS_URL="http://127.0.0.1:7770/v1/models"
|
||||||
BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}"
|
BOOT_TIMEOUT="${FRANK_ARCHON_AR_BOOT_TIMEOUT:-1200}"
|
||||||
RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}"
|
RESTORE_TIMEOUT="${FRANK_ARCHON_AR_RESTORE_TIMEOUT:-1200}"
|
||||||
MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}"
|
MEASURE_TIMEOUT="${FRANK_ARCHON_AR_MEASURE_TIMEOUT:-1800}"
|
||||||
SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)"
|
SERVED="$(sed -n 's/^FRANK_ARCHON_SERVED_NAME=//p' "$STAGE/frank-archon.env" 2>/dev/null | head -1)"
|
||||||
SERVED="${SERVED:-frank-primary}"
|
SERVED="${SERVED:-frank-archon}"
|
||||||
|
|
||||||
RESTORED=0
|
RESTORED=0
|
||||||
|
|
||||||
@@ -72,7 +72,7 @@ capture_bench_failure() {
|
|||||||
echo "=== staged serve.out (tail 60) ==="
|
echo "=== staged serve.out (tail 60) ==="
|
||||||
tail -60 "$STAGE/serve.out" 2>/dev/null || true
|
tail -60 "$STAGE/serve.out" 2>/dev/null || true
|
||||||
echo "=== engine log (tail 60) ==="
|
echo "=== engine log (tail 60) ==="
|
||||||
tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true
|
tail -60 /home/dank/logs/frank_archon.log 2>/dev/null || true
|
||||||
} >"$STAGE/fail.log" 2>&1 || true
|
} >"$STAGE/fail.log" 2>&1 || true
|
||||||
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
|
grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
|
||||||
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
|
"$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
|
||||||
@@ -114,7 +114,7 @@ trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
|
|||||||
|
|
||||||
main() {
|
main() {
|
||||||
local f
|
local f
|
||||||
for f in serve.sh frank-primary.env measure.py hash; do
|
for f in serve.sh frank-archon.env measure.py hash; do
|
||||||
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
|
[ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
|
||||||
done
|
done
|
||||||
mkdir -p "$STAGE/last"
|
mkdir -p "$STAGE/last"
|
||||||
@@ -133,7 +133,7 @@ main() {
|
|||||||
|
|
||||||
t0="$(date +%s)"
|
t0="$(date +%s)"
|
||||||
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
|
log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
|
||||||
FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \
|
FRANK_ARCHON_ENV_FILE="$STAGE/frank-archon.env" \
|
||||||
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
|
setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
|
||||||
echo "$!" >"$STAGE/serve.pid"
|
echo "$!" >"$STAGE/serve.pid"
|
||||||
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Decode throughput benchmark for frank-large.
|
# Decode throughput benchmark for frank-dragoon.
|
||||||
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
|
# Measures generation tok/s with 8K+ token prompts and 256 generated tokens.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -75,7 +75,7 @@ print(json.dumps(' '.join(words)))
|
|||||||
t0=$(date +%s%N)
|
t0=$(date +%s%N)
|
||||||
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
|
resp=$(curl -s -m 600 http://127.0.0.1:$PORT/v1/chat/completions \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d "{\"model\":\"frank-large\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
|
-d "{\"model\":\"frank-dragoon\",\"messages\":[{\"role\":\"user\",\"content\":$prompt}],\"max_tokens\":$N_GEN,\"stream\":false}" 2>&1)
|
||||||
t1=$(date +%s%N)
|
t1=$(date +%s%N)
|
||||||
|
|
||||||
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
|
if ! echo "$resp" | python3 -c "import sys,json; json.load(sys.stdin)" 2>/dev/null; then
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""
|
"""
|
||||||
HTTP-based prefill-throughput sweep for frank-large.
|
HTTP-based prefill-throughput sweep for frank-dragoon.
|
||||||
Starts the server with a given --threads-batch value, sends a 64K-prompt via
|
Starts the server with a given --threads-batch value, sends a 64K-prompt via
|
||||||
the chat API (max_tokens=0), measures prompt_tokens/wall_time.
|
the chat API (max_tokens=0), measures prompt_tokens/wall_time.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Threads-batch sweep for frank-large on dankclaw
|
# Threads-batch sweep for frank-dragoon on dankclaw
|
||||||
# Benches prefill throughput at 65536 prompt tokens across threads-batch values.
|
# Benches prefill throughput at 65536 prompt tokens across threads-batch values.
|
||||||
set -uo pipefail
|
set -uo pipefail
|
||||||
|
|
||||||
@@ -10,7 +10,7 @@ TB_LIST=(16 32 48 64 80 96 128)
|
|||||||
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
|
ENV_FILE="/home/dank/GitPackages/ROCmFPX/build-qwen38/bin"
|
||||||
|
|
||||||
echo "========================================"
|
echo "========================================"
|
||||||
echo "THREADS-BATCH SWEEP — frank-large"
|
echo "THREADS-BATCH SWEEP — frank-dragoon"
|
||||||
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
|
echo "Model: $(basename "$MODEL") ($(du -h "$MODEL" | cut -f1))"
|
||||||
echo "Threads-batch values: ${TB_LIST[*]}"
|
echo "Threads-batch values: ${TB_LIST[*]}"
|
||||||
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
|
echo "Prompt: 65536 tokens, n-gen: 0 (prefill-only)"
|
||||||
|
|||||||
@@ -1,14 +1,14 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Benchmark ubatch with llama-bench for frank-large.
|
# Benchmark ubatch with llama-bench for frank-dragoon.
|
||||||
#
|
#
|
||||||
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
# This is intentionally a ubatch-only benchmark. It uses one llama-bench
|
||||||
# process, an 8K prompt workload, and equal batch/ubatch values. No server
|
# process, an 8K prompt workload, and equal batch/ubatch values. No server
|
||||||
# process, HTTP requests, or log scraping are used.
|
# process, HTTP requests, or log scraping are used.
|
||||||
#
|
#
|
||||||
# Run on Dankclaw after stopping frank-large:
|
# Run on Dankclaw after stopping frank-dragoon:
|
||||||
# sudo systemctl stop frank-large
|
# sudo systemctl stop frank-dragoon
|
||||||
# ./scripts/bench-ubatch.sh
|
# ./scripts/bench-ubatch.sh
|
||||||
# sudo systemctl start frank-large
|
# sudo systemctl start frank-dragoon
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
: "${LLAMA_BENCH:=/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-bench}"
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# build-frank-primary-rad.sh — build frank-primary's .rad container from the
|
# build-frank-archon-rad.sh — build frank-archon's .rad container from the
|
||||||
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
|
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint, at NATIVE precision.
|
||||||
#
|
#
|
||||||
# bash scripts/build-frank-primary-rad.sh # convert
|
# bash scripts/build-frank-archon-rad.sh # convert
|
||||||
# bash scripts/build-frank-primary-rad.sh --plan-only # declare, no I/O
|
# bash scripts/build-frank-archon-rad.sh --plan-only # declare, no I/O
|
||||||
# bash scripts/build-frank-primary-rad.sh --restage # redo staging
|
# bash scripts/build-frank-archon-rad.sh --restage # redo staging
|
||||||
# bash scripts/build-frank-primary-rad.sh --force # rebuild existing
|
# bash scripts/build-frank-archon-rad.sh --force # rebuild existing
|
||||||
#
|
#
|
||||||
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
|
# WHY A STAGING DIR (the one non-obvious step): rad-convert selects its
|
||||||
# architecture plugin by the checkpoint's quantisation DESCRIPTOR —
|
# architecture plugin by the checkpoint's quantisation DESCRIPTOR —
|
||||||
@@ -55,8 +55,8 @@ OUT="$OUT_DIR/qwen4exp-fp8-native.rad"
|
|||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$SCRIPT_DIR/frank-primary.env"
|
source "$SCRIPT_DIR/frank-archon.env"
|
||||||
IMAGE="$FRANK_PRIMARY_IMAGE"
|
IMAGE="$FRANK_ARCHON_IMAGE"
|
||||||
|
|
||||||
plan_only=0 restage=0 force=0
|
plan_only=0 restage=0 force=0
|
||||||
for a in "$@"; do
|
for a in "$@"; do
|
||||||
@@ -69,11 +69,11 @@ for a in "$@"; do
|
|||||||
done
|
done
|
||||||
|
|
||||||
if [ ! -f "$SRC/config.json" ]; then
|
if [ ! -f "$SRC/config.json" ]; then
|
||||||
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-primary.sh --fetch)" >&2
|
echo "ERROR: raw checkpoint missing: $SRC (run scripts/download-frank-archon.sh --fetch)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "verifying raw checkpoint first..."
|
echo "verifying raw checkpoint first..."
|
||||||
bash "$SCRIPT_DIR/download-frank-primary.sh"
|
bash "$SCRIPT_DIR/download-frank-archon.sh"
|
||||||
|
|
||||||
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
|
if [ -f "$OUT" ] && [ "$plan_only" -eq 0 ] && [ "$force" -eq 0 ]; then
|
||||||
echo "container already exists: $OUT"
|
echo "container already exists: $OUT"
|
||||||
@@ -83,7 +83,7 @@ fi
|
|||||||
|
|
||||||
# ---- staging ----------------------------------------------------------------
|
# ---- staging ----------------------------------------------------------------
|
||||||
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
|
if [ "$restage" -eq 1 ] && [ -e "$STAGE" ]; then
|
||||||
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-stage"
|
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-stage"
|
||||||
mkdir -p "$TRASH"
|
mkdir -p "$TRASH"
|
||||||
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
|
mv "$STAGE" "$TRASH/stage-$(date +%H%M%S)"
|
||||||
echo "old staging dir moved to $TRASH"
|
echo "old staging dir moved to $TRASH"
|
||||||
@@ -138,7 +138,7 @@ CONVERT_ARGS=(/models/stage-qwen4exp-fp8 -o /models/rad/qwen4exp-fp8-native.rad
|
|||||||
# exactly and the plugin's one-element scale plane is produced.
|
# exactly and the plugin's one-element scale plane is produced.
|
||||||
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
|
CONVERT_ARGS+=(--quant "blk.*.ple_ngram.weight=rtn:codes=fp8_e4m3,block=*x*,scale=bf16,rule=fixed,scale_value=$PLE_SCALE")
|
||||||
[ "$force" -eq 1 ] && [ -f "$OUT" ] && {
|
[ "$force" -eq 1 ] && [ -f "$OUT" ] && {
|
||||||
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-primary-rad"
|
TRASH="$HOME/.AGENTTRASH/$(date +%Y%m%d)-frank-archon-rad"
|
||||||
mkdir -p "$TRASH"
|
mkdir -p "$TRASH"
|
||||||
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
|
mv "$OUT" "$TRASH/qwen4exp-fp8-native.rad.$(date +%H%M%S)"
|
||||||
echo "previous container moved to $TRASH"
|
echo "previous container moved to $TRASH"
|
||||||
@@ -158,8 +158,8 @@ echo "rad-info summary:"
|
|||||||
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
|
podman run --rm --entrypoint rad-info -v /home/dank/models:/models:ro \
|
||||||
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
|
"$IMAGE" "/models/rad/qwen4exp-fp8-native.rad" | head -20
|
||||||
MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
|
MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
|
||||||
echo "container: $OUT (${MIB} MiB; env floor $FRANK_PRIMARY_RAD_MIN_MIB MiB)"
|
echo "container: $OUT (${MIB} MiB; env floor $FRANK_ARCHON_RAD_MIN_MIB MiB)"
|
||||||
if [ "$MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
|
if [ "$MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
|
||||||
echo "ERROR: container below the env floor — the convert was truncated" >&2
|
echo "ERROR: container below the env floor — the convert was truncated" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Build the gufo runtime that serves frank-large on Dankclaw.
|
# Build the gufo runtime that serves frank-dragoon on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/build-gufo.sh # build + install the pinned tag
|
# bash scripts/build-gufo.sh # build + install the pinned tag
|
||||||
# bash scripts/build-gufo.sh --verify # only check the installed tree
|
# bash scripts/build-gufo.sh --verify # only check the installed tree
|
||||||
@@ -93,4 +93,4 @@ cmake --install build/release
|
|||||||
|
|
||||||
echo "== verify =="
|
echo "== verify =="
|
||||||
"$PREFIX/bin/gufo" diagnose
|
"$PREFIX/bin/gufo" diagnose
|
||||||
echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
|
echo "frank-dragoon runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Build the ROCmFPX runtime that serves frank-large on Dankclaw.
|
# Build the ROCmFPX runtime that serves frank-dragoon on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
|
# bash scripts/build-rocmfpx.sh # clone/fetch + build the pin
|
||||||
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree
|
# bash scripts/build-rocmfpx.sh --verify # only check the installed tree
|
||||||
#
|
#
|
||||||
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
|
# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
|
||||||
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's
|
# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-dragoon's
|
||||||
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
|
# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
|
||||||
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
|
# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
|
||||||
# stock llama.cpp, gufo and every other loader on the host reject the file.
|
# stock llama.cpp, gufo and every other loader on the host reject the file.
|
||||||
@@ -30,7 +30,7 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
|
ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
|
||||||
# The commit frank-large was qualified with (measured 2026-10-02).
|
# The commit frank-dragoon was qualified with (measured 2026-10-02).
|
||||||
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
|
ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
|
||||||
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
|
SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
|
||||||
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
|
BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
|
||||||
@@ -86,4 +86,4 @@ cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
|
|||||||
|
|
||||||
echo "== verify =="
|
echo "== verify =="
|
||||||
"$BUILD_DIR/bin/llama-server" --version | head -1
|
"$BUILD_DIR/bin/llama-server" --version | head -1
|
||||||
echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
|
echo "frank-dragoon runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
|
||||||
@@ -26,8 +26,8 @@ RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn
|
|||||||
CALIB=/home/dank/corpora/calib
|
CALIB=/home/dank/corpora/calib
|
||||||
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||||
LOG=/home/dank/logs/swift15_convert.log
|
LOG=/home/dank/logs/swift15_convert.log
|
||||||
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
|
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert
|
||||||
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
|
# table; one encoding per expert table rules out a block-fp8 down tier -- see the recipe header).
|
||||||
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
||||||
|
|
||||||
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
||||||
|
|||||||
@@ -57,9 +57,9 @@ else
|
|||||||
echo " none"
|
echo " none"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
section "frank-primary unit"
|
section "frank-archon unit"
|
||||||
printf ' active: %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)"
|
printf ' active: %s\n' "$(systemctl is-active frank-archon 2>/dev/null | head -1)"
|
||||||
printf ' enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)"
|
printf ' enabled: %s\n' "$(systemctl is-enabled frank-archon 2>/dev/null | head -1)"
|
||||||
|
|
||||||
section "glm serve tail"
|
section "glm serve tail"
|
||||||
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
|
if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
|
||||||
|
|||||||
+54
-461
@@ -1,475 +1,68 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Plan runtime services affected by repository changes.
|
"""deploy-plan.py — map changed paths to deployable services.
|
||||||
|
|
||||||
The planner is intentionally side-effect free. It accepts either an explicit
|
Thin wrapper over scripts/service-map.py, which owns the path->service
|
||||||
path list or two Git revisions. The latter lets compose-file changes resolve
|
semantics in the shared inventory (shared/services.yml): per-service path
|
||||||
to only the service definitions that actually changed instead of restarting an
|
rules, the quadlet netns/network coupling graph, host scoping (a host's
|
||||||
entire compose project.
|
directory can only select that host's services), and canonical restart
|
||||||
|
order. This file keeps the deployer's CLI contract only.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
deploy-plan.py --target NAME --repo DIR --base REV --head REV
|
||||||
|
deploy-plan.py --target NAME [paths...] (paths also on stdin)
|
||||||
|
|
||||||
|
Prints one service name per line (canonical restart order); exit 0 with no
|
||||||
|
output when nothing relevant changed.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
import os
|
||||||
|
|
||||||
import argparse
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
from pathlib import PurePosixPath
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import yaml
|
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
SERVICE_MAP = os.path.join(SCRIPT_DIR, "service-map.py")
|
||||||
# Dependency-sensitive services come before their consumers. Independent
|
|
||||||
# compose services follow. LLM services are deliberately last and are never
|
|
||||||
# implied by a generic host, stack, documentation, or config change.
|
|
||||||
SERVICE_ORDER = (
|
|
||||||
"gluetun",
|
|
||||||
"searxng-valkey",
|
|
||||||
"searxng",
|
|
||||||
"marreta",
|
|
||||||
"pihole",
|
|
||||||
"open-webui",
|
|
||||||
"gitea-postgres",
|
|
||||||
"gitea-server",
|
|
||||||
"comfyui",
|
|
||||||
"seerr",
|
|
||||||
"hermes",
|
|
||||||
"hermes-dashboard",
|
|
||||||
"hermes-webui",
|
|
||||||
"prometheus",
|
|
||||||
"grafana",
|
|
||||||
"loki",
|
|
||||||
"alloy",
|
|
||||||
"node-exporter",
|
|
||||||
"alertmanager",
|
|
||||||
"authentik-postgres",
|
|
||||||
"authentik-server",
|
|
||||||
"authentik-worker",
|
|
||||||
"authentik-outpost",
|
|
||||||
"authentik-outpost-ldap",
|
|
||||||
"caddy",
|
|
||||||
"homepage",
|
|
||||||
"romm-mariadb",
|
|
||||||
"romm-redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
"calibre-server",
|
|
||||||
"plex",
|
|
||||||
"tautulli",
|
|
||||||
"prowlarr",
|
|
||||||
"qbittorrent",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
"frank-large",
|
|
||||||
"frank-secondary",
|
|
||||||
"frank-primary",
|
|
||||||
)
|
|
||||||
|
|
||||||
DANKCLAW_SERVICES = {
|
|
||||||
"hermes-webui",
|
|
||||||
"gluetun",
|
|
||||||
"searxng-valkey",
|
|
||||||
"searxng",
|
|
||||||
"marreta",
|
|
||||||
"pihole",
|
|
||||||
"open-webui",
|
|
||||||
"gitea-postgres",
|
|
||||||
"gitea-server",
|
|
||||||
"comfyui",
|
|
||||||
"seerr",
|
|
||||||
"hermes",
|
|
||||||
"hermes-dashboard",
|
|
||||||
"prometheus",
|
|
||||||
"grafana",
|
|
||||||
"loki",
|
|
||||||
"alloy",
|
|
||||||
"node-exporter",
|
|
||||||
"alertmanager",
|
|
||||||
"authentik-postgres",
|
|
||||||
"authentik-server",
|
|
||||||
"authentik-worker",
|
|
||||||
"authentik-outpost",
|
|
||||||
"authentik-outpost-ldap",
|
|
||||||
"caddy",
|
|
||||||
"homepage",
|
|
||||||
"frank-large",
|
|
||||||
"frank-secondary",
|
|
||||||
}
|
|
||||||
|
|
||||||
DANKSERVER_SERVICES = {
|
|
||||||
"romm-mariadb",
|
|
||||||
"romm-redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
"calibre-server",
|
|
||||||
"plex",
|
|
||||||
"tautulli",
|
|
||||||
"prowlarr",
|
|
||||||
"qbittorrent",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
"frank-primary",
|
|
||||||
"node-exporter",
|
|
||||||
}
|
|
||||||
|
|
||||||
# FrankBook-Pro runs the *arr stack + qBittorrent inside a podman machine on
|
|
||||||
# macOS. Service names overlap the Linux hosts (gluetun, qbittorrent, ...);
|
|
||||||
# the per-target sets and the frankbook-pro intercept in deploy-remote.sh
|
|
||||||
# keep dispatch unambiguous.
|
|
||||||
FRANKBOOK_SERVICES = {
|
|
||||||
"gluetun",
|
|
||||||
"qbittorrent",
|
|
||||||
"prowlarr",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
}
|
|
||||||
|
|
||||||
DANKCLAW_COMPOSE_SERVICES = set()
|
|
||||||
# Dankserver's compose file is still the source of truth for its media stack;
|
|
||||||
# Dankclaw's is gone (deleted 2026-09-16). Every service on Dankclaw is a
|
|
||||||
# Quadlet unit now — the last compose services were the hermes trio, whose units
|
|
||||||
# have owned their containers since 2026-09-12 and whose compose blocks were
|
|
||||||
# dead weight that could only create a duplicate container.
|
|
||||||
|
|
||||||
DANKSERVER_COMPOSE_SERVICES = {
|
|
||||||
"mariadb",
|
|
||||||
"redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
}
|
|
||||||
COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
|
|
||||||
|
|
||||||
|
|
||||||
|
def git_paths(repo: str, base: str, head: str) -> list[str]:
|
||||||
def _add(services: set[str], *names: str) -> None:
|
proc = subprocess.run(
|
||||||
services.update(names)
|
["git", "-C", repo, "diff", "--name-only", base, head],
|
||||||
|
text=True, capture_output=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
def target_services(target: str) -> set[str]:
|
sys.stderr.write(f"deploy-plan: git diff failed: {proc.stderr}")
|
||||||
if target == "dankclaw":
|
raise SystemExit(1)
|
||||||
return DANKCLAW_SERVICES
|
return [line for line in proc.stdout.splitlines() if line]
|
||||||
if target == "dankserver":
|
|
||||||
return DANKSERVER_SERVICES
|
|
||||||
if target == "frankbook-pro":
|
|
||||||
return FRANKBOOK_SERVICES
|
|
||||||
raise ValueError(f"unknown target: {target}")
|
|
||||||
|
|
||||||
|
|
||||||
def _compose_services(target: str) -> set[str]:
|
|
||||||
return DANKCLAW_COMPOSE_SERVICES if target == "dankclaw" else DANKSERVER_COMPOSE_SERVICES
|
|
||||||
|
|
||||||
|
|
||||||
def _git_show(repo: str, revision: str, path: str) -> str | None:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", repo, "show", f"{revision}:{path}"],
|
|
||||||
text=True,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
return result.stdout if result.returncode == 0 else None
|
|
||||||
|
|
||||||
|
|
||||||
def _yaml_document(contents: str | None) -> dict[str, Any]:
|
|
||||||
if contents is None:
|
|
||||||
return {}
|
|
||||||
try:
|
|
||||||
data = yaml.safe_load(contents) or {}
|
|
||||||
except yaml.YAMLError:
|
|
||||||
return {}
|
|
||||||
return data if isinstance(data, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def _yaml_services(contents: str | None) -> dict[str, Any]:
|
|
||||||
services = _yaml_document(contents).get("services", {})
|
|
||||||
return services if isinstance(services, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def compose_changes(repo: str, target: str, base: str, head: str) -> set[str]:
|
|
||||||
"""Return only compose services whose YAML definitions or dependencies changed.
|
|
||||||
|
|
||||||
A target with no compose file (Dankclaw since 2026-09-16) plans nothing:
|
|
||||||
its services are Quadlet units, selected by their own paths.
|
|
||||||
"""
|
|
||||||
path = f"{target}/compose.yaml"
|
|
||||||
if not _git_show(repo, head, path) and not _git_show(repo, base, path):
|
|
||||||
return set()
|
|
||||||
old_doc = _yaml_document(_git_show(repo, base, path))
|
|
||||||
new_doc = _yaml_document(_git_show(repo, head, path))
|
|
||||||
old = old_doc.get("services", {}) if isinstance(old_doc.get("services", {}), dict) else {}
|
|
||||||
new = new_doc.get("services", {}) if isinstance(new_doc.get("services", {}), dict) else {}
|
|
||||||
names = set(old) | set(new)
|
|
||||||
allowed = _compose_services(target)
|
|
||||||
|
|
||||||
# An unavailable or malformed revision is safer as a broad compose plan
|
|
||||||
# than as a silent no-op. It still cannot include LLM units.
|
|
||||||
if (old_doc and not new_doc) or (new_doc and not old_doc and names):
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
|
||||||
|
|
||||||
# Top-level networks/configuration can affect every compose service even
|
|
||||||
# when no individual service mapping changed. Ignore the services key while
|
|
||||||
# comparing the rest of the Compose document.
|
|
||||||
old_globals = {key: value for key, value in old_doc.items() if key != "services"}
|
|
||||||
new_globals = {key: value for key, value in new_doc.items() if key != "services"}
|
|
||||||
if old_globals != new_globals:
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
|
||||||
|
|
||||||
changed = {name for name in names if name in allowed and old.get(name) != new.get(name)}
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in changed}
|
|
||||||
|
|
||||||
|
|
||||||
def services_for_path(path: str, target: str) -> set[str]:
|
|
||||||
"""Return services affected by one repository-relative path."""
|
|
||||||
path = path.replace("\\", "/")
|
|
||||||
p = PurePosixPath(path)
|
|
||||||
parts = p.parts
|
|
||||||
services: set[str] = set()
|
|
||||||
if not parts:
|
|
||||||
return services
|
|
||||||
|
|
||||||
# Non-runtime material is deliberately a no-op.
|
|
||||||
if parts[0] in {"docs", "tests", "old", ".githooks"}:
|
|
||||||
return services
|
|
||||||
if path in {"README.md", "AGENTS.md"} or parts[0] == ".obsidian":
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "shared":
|
|
||||||
if len(parts) < 2:
|
|
||||||
return services
|
|
||||||
group = parts[1]
|
|
||||||
if group == "searxng":
|
|
||||||
_add(services, "searxng")
|
|
||||||
elif group == "prometheus":
|
|
||||||
_add(services, "prometheus")
|
|
||||||
if len(parts) >= 3 and parts[2] == "alertmanager.yml":
|
|
||||||
# The Alertmanager config ships from this directory: a change
|
|
||||||
# to it must redeploy the service that installs it.
|
|
||||||
_add(services, "alertmanager")
|
|
||||||
elif group == "loki":
|
|
||||||
_add(services, "loki")
|
|
||||||
elif group == "alloy":
|
|
||||||
_add(services, "alloy")
|
|
||||||
elif group == "grafana":
|
|
||||||
# Provisioned datasources and dashboards: git is the source of
|
|
||||||
# truth, so a change here must restart the service that reads them.
|
|
||||||
_add(services, "grafana")
|
|
||||||
elif group == "homepage":
|
|
||||||
# Homepage is a Dankclaw-owned deployment. Shared homepage files
|
|
||||||
# are intentionally not copied during a Dankserver-only run.
|
|
||||||
if target == "dankclaw":
|
|
||||||
_add(services, "homepage")
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "dankclaw":
|
|
||||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
|
||||||
# Dankclaw has no compose file since 2026-09-16: every service is a
|
|
||||||
# Quadlet unit, each selected by its own path. A change (or a
|
|
||||||
# deletion) of this path therefore restarts nothing.
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "Caddyfile":
|
|
||||||
_add(services, "caddy")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "Containerfile.comfyui":
|
|
||||||
_add(services, "comfyui")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] in {"caddy", "comfyui", "loki", "alloy", "hermes", "anubis", "seerr"}:
|
|
||||||
_add(services, parts[1])
|
|
||||||
return services
|
|
||||||
if len(parts) >= 4 and parts[1] == "authentik" and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name in {"authentik-postgres", "authentik-server", "authentik-worker", "authentik-outpost", "authentik-outpost-ldap"}:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "searxng":
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3]
|
|
||||||
if name == "gluetun.container":
|
|
||||||
# SearXNG and Marreta (+ its browser companion) share
|
|
||||||
# Gluetun's network namespace. A Gluetun recreation
|
|
||||||
# tears the netns down, so every sharer must restart.
|
|
||||||
_add(services, "gluetun", "searxng", "marreta")
|
|
||||||
elif name == "searxng.container":
|
|
||||||
_add(services, "searxng")
|
|
||||||
elif name == "searxng-valkey.container":
|
|
||||||
_add(services, "searxng-valkey", "searxng")
|
|
||||||
elif name == "searxng.network":
|
|
||||||
_add(services, "gluetun", "searxng-valkey", "searxng", "marreta")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "marreta":
|
|
||||||
# One deployable owns both the app and its browser companion;
|
|
||||||
# both live in gluetun's netns. A change to either unit selects
|
|
||||||
# the marreta service, which restarts browser-then-app.
|
|
||||||
_add(services, "marreta")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "pihole":
|
|
||||||
_add(services, "pihole")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "open-webui":
|
|
||||||
_add(services, "open-webui")
|
|
||||||
return services
|
|
||||||
# frank-large left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
|
||||||
# build, root unit): its files live in scripts/ and map there.
|
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
|
||||||
# The five observability containers became Quadlet units on
|
|
||||||
# 2026-09-11. A unit change selects exactly its own service;
|
|
||||||
# SERVICE_ORDER already puts each dependency ahead of its
|
|
||||||
# consumer (prometheus before grafana, loki before alloy).
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter", "alertmanager"}:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "gitea":
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3]
|
|
||||||
if name == "gitea-postgres.container":
|
|
||||||
_add(services, "gitea-postgres")
|
|
||||||
elif name == "gitea-server.container":
|
|
||||||
_add(services, "gitea-server")
|
|
||||||
elif name == "gitea-caddy.container":
|
|
||||||
# Removing the old split proxy requires the canonical
|
|
||||||
# Caddy handler to retire its installed unit/container
|
|
||||||
# before it claims ports 80/443.
|
|
||||||
_add(services, "caddy")
|
|
||||||
elif name == "gitea.network":
|
|
||||||
_add(services, "gitea-postgres", "gitea-server")
|
|
||||||
return services
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "dankserver":
|
|
||||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
|
||||||
# Compose service keys map onto the deployed unit names.
|
|
||||||
_add(services, *(COMPOSE_NAME_MAP.get(n, n) for n in DANKSERVER_COMPOSE_SERVICES))
|
|
||||||
return services
|
|
||||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
|
||||||
quadlet_services = {
|
|
||||||
"romm": {"romm", "romm-mariadb", "romm-redis"},
|
|
||||||
"jellyfin": {"jellyfin"},
|
|
||||||
"calibre-web": {"calibre-web"},
|
|
||||||
"calibre-server": {"calibre-server"},
|
|
||||||
"copyparty": {"copyparty"},
|
|
||||||
"plex": {"plex"},
|
|
||||||
"tautulli": {"tautulli"},
|
|
||||||
}
|
|
||||||
_add(services, *quadlet_services.get(parts[1], set()))
|
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
|
||||||
# Dankserver's node-exporter became a Quadlet unit on 2026-09-16
|
|
||||||
# (serverAppConfigs-docs/Infrastructure/Observability.md): its
|
|
||||||
# compose service died at a reboot and stayed dead, so Prometheus'
|
|
||||||
# node-dankserver job had nothing to scrape.
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name == "node-exporter":
|
|
||||||
_add(services, "node-exporter")
|
|
||||||
return services
|
|
||||||
# frank-primary's quadlet-era directory left the tree on 2026-09-26
|
|
||||||
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
|
|
||||||
# the trio in scripts/ now maps it). Nothing lives under
|
|
||||||
# dankserver/frank-primary/ except dated trash, and trash plans
|
|
||||||
# nothing (the retire rule above the service blocks).
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "frankbook-pro":
|
|
||||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name == "gluetun":
|
|
||||||
# Every *arr + qBittorrent container shares gluetun's
|
|
||||||
# network namespace; a gluetun recreation kills all of
|
|
||||||
# them, so the plan restarts the whole netns family.
|
|
||||||
_add(services, "gluetun", "qbittorrent",
|
|
||||||
"prowlarr", "sonarr", "radarr", "lidarr", "readarr")
|
|
||||||
elif name in FRANKBOOK_SERVICES:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "scripts":
|
|
||||||
name = p.name
|
|
||||||
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
|
|
||||||
_add(services, "homepage")
|
|
||||||
if name in {"signal-relay.py", "signal-relay.service"}:
|
|
||||||
# The Signal relay is installed by the alertmanager deploy, so a
|
|
||||||
# change to it redeploying alertmanager is the only path that
|
|
||||||
# reaches it on the host.
|
|
||||||
_add(services, "alertmanager")
|
|
||||||
if name in {"frank-secondary.env", "frank-secondary.service", "serve-frank-secondary.sh"}:
|
|
||||||
# llama.cpp frank-secondary (2026-09-16) keeps its unit, env and serve
|
|
||||||
# files here; each one selects frank-secondary only. The ik_llama.cpp
|
|
||||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
|
||||||
# which is archive material and plans nothing.
|
|
||||||
_add(services, "frank-secondary")
|
|
||||||
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
|
|
||||||
# CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and
|
|
||||||
# serve files here the same way: a root unit on a host-built
|
|
||||||
# runtime. The download script fetches the payload, which is
|
|
||||||
# host-side state, so it selects nothing.
|
|
||||||
_add(services, "frank-large")
|
|
||||||
if name == "qwen-sharp-v22.4.0.jinja":
|
|
||||||
# The Qwen-Sharp chat template frank-large loads via
|
|
||||||
# --chat-template-file. It lived under dankserver/frank-medium/ until
|
|
||||||
# the 2026-09-26 cutover retired that directory; editing it changes
|
|
||||||
# what the live frank-large endpoint renders, so it plans
|
|
||||||
# frank-large.
|
|
||||||
_add(services, "frank-large")
|
|
||||||
if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}:
|
|
||||||
# llama.cpp frank-primary (2026-09-26, replacing the Lucebox
|
|
||||||
# DS4.1 quadlet) keeps its unit, env and serve files here the
|
|
||||||
# same way: a root unit on the unsloth b11030-mix prebuilt
|
|
||||||
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
|
||||||
# the payload, which is host-side state, so it selects nothing.
|
|
||||||
_add(services, "frank-primary")
|
|
||||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
|
||||||
# restart a runtime service.
|
|
||||||
return services
|
|
||||||
|
|
||||||
# Examples and env templates are not deployed runtime state.
|
|
||||||
if p.name in {".env.example", "env.example"}:
|
|
||||||
return services
|
|
||||||
return services
|
|
||||||
|
|
||||||
def changed_paths(repo: str, base: str, head: str) -> list[str]:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", repo, "diff", "--name-only", "--diff-filter=ACDMRTUXB", base, head],
|
|
||||||
text=True,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
return [line for line in result.stdout.splitlines() if line]
|
|
||||||
|
|
||||||
|
|
||||||
def plan(paths: list[str], target: str, repo: str | None = None,
|
|
||||||
base: str | None = None, head: str | None = None) -> list[str]:
|
|
||||||
services: set[str] = set()
|
|
||||||
for path in paths:
|
|
||||||
if path == f"{target}/compose.yaml" and repo and base and head:
|
|
||||||
services.update(compose_changes(repo, target, base, head))
|
|
||||||
else:
|
|
||||||
services.update(services_for_path(path, target))
|
|
||||||
allowed = target_services(target)
|
|
||||||
return [name for name in SERVICE_ORDER if name in services and name in allowed]
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
args = sys.argv[1:]
|
||||||
parser.add_argument("--target", choices=("dankclaw", "dankserver", "frankbook-pro"), required=True)
|
if not args or args[0] != "--target":
|
||||||
parser.add_argument("--paths", nargs="+")
|
print(__doc__, file=sys.stderr)
|
||||||
parser.add_argument("--repo", help="repository path for revision-aware planning")
|
return 2
|
||||||
parser.add_argument("--base", help="base Git revision")
|
if len(args) < 2:
|
||||||
parser.add_argument("--head", help="head Git revision")
|
print("missing target name after --target", file=sys.stderr)
|
||||||
args = parser.parse_args()
|
return 2
|
||||||
|
target = args[1]
|
||||||
if not args.paths and not (args.repo and args.base and args.head):
|
# `--paths` is the historic seam keyword: a flag, not an option — the
|
||||||
parser.error("provide --paths or all of --repo, --base, and --head")
|
# paths are the positional arguments that follow it.
|
||||||
paths = args.paths or changed_paths(args.repo, args.base, args.head)
|
rest = [a for a in args[2:] if a != "--paths"]
|
||||||
print("\n".join(plan(paths, args.target, args.repo, args.base, args.head)))
|
if "--repo" in rest:
|
||||||
|
try:
|
||||||
|
repo = rest[rest.index("--repo") + 1]
|
||||||
|
base = rest[rest.index("--base") + 1]
|
||||||
|
head = rest[rest.index("--head") + 1]
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
print("--repo, --base and --head must appear together", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
paths = git_paths(repo, base, head)
|
||||||
|
else:
|
||||||
|
paths = rest or [line.strip() for line in sys.stdin if line.strip()]
|
||||||
|
if not paths:
|
||||||
|
return 0
|
||||||
|
proc = subprocess.run([sys.executable, SERVICE_MAP, "plan", "--host", target] + paths,
|
||||||
|
text=True, capture_output=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
sys.stderr.write(proc.stderr)
|
||||||
|
return proc.returncode
|
||||||
|
sys.stdout.write(proc.stdout)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+211
-281
@@ -9,6 +9,13 @@
|
|||||||
# pending until its service is explicitly deployed.
|
# pending until its service is explicitly deployed.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
# The LLM handlers' artifact gates are the serve scripts' own library, so
|
||||||
|
# the deployer and the unit refuse the same missing payload with the same
|
||||||
|
# words — no deploy can pass a gate the unit would fail. GATE_PREFIX is set
|
||||||
|
# per handler.
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/llm-gates.sh"
|
||||||
|
|
||||||
TARGET="${1:-}"
|
TARGET="${1:-}"
|
||||||
REVISION="${2:-}"
|
REVISION="${2:-}"
|
||||||
shift 2 || true
|
shift 2 || true
|
||||||
@@ -80,31 +87,10 @@ cd "$REMOTE_DIR"
|
|||||||
mkdir -p "$STATE_DIR"
|
mkdir -p "$STATE_DIR"
|
||||||
|
|
||||||
service_is_allowed() {
|
service_is_allowed() {
|
||||||
local service="$1"
|
# The inventory (shared/services.yml via scripts/service-map.py) is the
|
||||||
case "$TARGET:$service" in
|
# only allow-list: a service is deployable on a target iff the inventory
|
||||||
dankclaw:gluetun|dankclaw:searxng-valkey|dankclaw:searxng|dankclaw:pihole|\
|
# places it there. Retiring a service from a host is a data edit.
|
||||||
dankclaw:open-webui|dankclaw:gitea-postgres|dankclaw:gitea-server|\
|
python3 scripts/service-map.py allows --host "$TARGET" --service "$1"
|
||||||
dankclaw:comfyui|dankclaw:hermes|\
|
|
||||||
dankclaw:hermes-dashboard|dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|\
|
|
||||||
dankclaw:loki|dankclaw:alloy|dankclaw:node-exporter|dankclaw:alertmanager|\
|
|
||||||
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
|
||||||
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
|
||||||
dankclaw:frank-large|dankclaw:frank-secondary|\
|
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
|
||||||
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
|
|
||||||
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
|
|
||||||
dankserver:frank-primary|dankserver:node-exporter|\
|
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
|
||||||
frankbook-pro:readarr)
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
mark_status() {
|
mark_status() {
|
||||||
@@ -192,7 +178,7 @@ add_pending_services() {
|
|||||||
continue
|
continue
|
||||||
fi
|
fi
|
||||||
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
|
[[ -n "$recorded" && "$recorded" != "$REVISION" ]] || continue
|
||||||
if [[ "$TARGET" == dankserver && "$service" == frank-large ]]; then continue; fi
|
if [[ "$TARGET" == dankserver && "$service" == frank-dragoon ]]; then continue; fi
|
||||||
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
|
if ! changed="$(python3 scripts/deploy-plan.py --target "$TARGET" \
|
||||||
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
|
--repo "$REMOTE_DIR" --base "$recorded" --head "$REVISION")"; then
|
||||||
echo "WARN: unable to compare state for $service; retaining it as pending" >&2
|
echo "WARN: unable to compare state for $service; retaining it as pending" >&2
|
||||||
@@ -274,13 +260,11 @@ install_signal_relay() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
copy_loki_config() {
|
copy_loki_config() {
|
||||||
run_privileged mkdir -p /opt/loki/config
|
copy_config /opt/loki/config shared/loki/loki-config.yaml
|
||||||
run_privileged cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
|
|
||||||
}
|
}
|
||||||
|
|
||||||
copy_alloy_config() {
|
copy_alloy_config() {
|
||||||
run_privileged mkdir -p /opt/alloy/config
|
copy_config /opt/alloy/config shared/alloy/config.alloy
|
||||||
run_privileged cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
|
|
||||||
}
|
}
|
||||||
|
|
||||||
copy_grafana_provisioning() {
|
copy_grafana_provisioning() {
|
||||||
@@ -289,14 +273,11 @@ copy_grafana_provisioning() {
|
|||||||
# datasource would be invisible to review, lost with the data volume, and
|
# datasource would be invisible to review, lost with the data volume, and
|
||||||
# unknown to anyone reading the repository. The unit already mounts
|
# unknown to anyone reading the repository. The unit already mounts
|
||||||
# /opt/grafana/provisioning read-only, so this only has to populate it.
|
# /opt/grafana/provisioning read-only, so this only has to populate it.
|
||||||
run_privileged mkdir -p /opt/grafana/provisioning/datasources \
|
copy_config /opt/grafana/provisioning/datasources \
|
||||||
/opt/grafana/provisioning/dashboards
|
shared/grafana/provisioning/datasources/prometheus.yml
|
||||||
run_privileged cp shared/grafana/provisioning/datasources/prometheus.yml \
|
copy_config /opt/grafana/provisioning/dashboards \
|
||||||
/opt/grafana/provisioning/datasources/prometheus.yml
|
shared/grafana/provisioning/dashboards/provider.yml \
|
||||||
run_privileged cp shared/grafana/provisioning/dashboards/provider.yml \
|
shared/grafana/provisioning/dashboards/llm-fleet.json
|
||||||
/opt/grafana/provisioning/dashboards/provider.yml
|
|
||||||
run_privileged cp shared/grafana/provisioning/dashboards/llm-fleet.json \
|
|
||||||
/opt/grafana/provisioning/dashboards/llm-fleet.json
|
|
||||||
}
|
}
|
||||||
|
|
||||||
render_caddyfile() {
|
render_caddyfile() {
|
||||||
@@ -441,31 +422,6 @@ deploy_caddy() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
deploy_compose_service() {
|
|
||||||
local service="$1"
|
|
||||||
if [[ "$TARGET" == dankserver ]]; then
|
|
||||||
# Dankserver's compose file is the source-of-truth inventory, but its
|
|
||||||
# production containers are individually systemd-managed/bare. Do not
|
|
||||||
# use compose up here: it can create a second container with a wrong
|
|
||||||
# name and can interfere with a running workload.
|
|
||||||
echo "WARN: $service is recorded in dankserver/compose.yaml; production container management is host-specific and no restart was attempted."
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
if [[ "$TARGET" == dankclaw ]]; then
|
|
||||||
# Dankclaw has no compose file since 2026-09-16: every service is a
|
|
||||||
# Quadlet unit with its own handler above. Reaching this means the
|
|
||||||
# dispatcher is missing a case, and the old fallback (`podman-compose
|
|
||||||
# up -d`) would now be a silent no-op or a duplicate container.
|
|
||||||
echo "no compose services on dankclaw; $service has no quadlet handler" >&2
|
|
||||||
return 2
|
|
||||||
fi
|
|
||||||
case "$service" in
|
|
||||||
*)
|
|
||||||
echo "unknown compose service: $service" >&2
|
|
||||||
return 2
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
# grafana is the one observability service that holds a credential: the
|
# grafana is the one observability service that holds a credential: the
|
||||||
# Authentik OAuth2 client secret. It is seeded into a per-service env file (the
|
# Authentik OAuth2 client secret. It is seeded into a per-service env file (the
|
||||||
@@ -568,6 +524,35 @@ quadlet_unit_dir="$HOME/.config/containers/systemd"
|
|||||||
# dankserver's user manager had no systemd container generator dir before the
|
# dankserver's user manager had no systemd container generator dir before the
|
||||||
# quadlet migration; every handler that installs a unit must find it present.
|
# quadlet migration; every handler that installs a unit must find it present.
|
||||||
mkdir -p "$quadlet_unit_dir"
|
mkdir -p "$quadlet_unit_dir"
|
||||||
|
# copy_config DEST SRC... — the privileged config copy every service uses:
|
||||||
|
# ensure DEST, copy each tracked SRC into it. The deployer owns /opt state;
|
||||||
|
# the files in git are the source of truth, so a service config that only
|
||||||
|
# exists on the host is a gap by policy.
|
||||||
|
copy_config() {
|
||||||
|
local dest="$1"; shift
|
||||||
|
run_privileged mkdir -p "$dest"
|
||||||
|
local src
|
||||||
|
for src in "$@"; do
|
||||||
|
run_privileged cp "$src" "$dest/"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# retire_unit_file PATH TRASH [UNIT] — the repo Deletion & Trash policy, in
|
||||||
|
# code: stop and disable the named unit, move the unit FILE into the home
|
||||||
|
# trash (never rm), report. The dragoon quadlet retirement and the
|
||||||
|
# compose-era comfyui.service retirement were the same six lines twice.
|
||||||
|
retire_unit_file() {
|
||||||
|
local path="$1" trash="$2" unit="${3:-}"
|
||||||
|
[[ -e "$path" ]] || return 0
|
||||||
|
if [[ -n "$unit" ]]; then
|
||||||
|
systemctl --user stop "$unit" 2>/dev/null || true
|
||||||
|
systemctl --user disable "$unit" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
mkdir -p "$trash"
|
||||||
|
mv -- "$path" "$trash/$(basename "$path")"
|
||||||
|
echo "retired $(basename "$path") -> $trash"
|
||||||
|
}
|
||||||
|
|
||||||
install_quadlet() {
|
install_quadlet() {
|
||||||
local service="$1"
|
local service="$1"
|
||||||
local source_dir="$2"
|
local source_dir="$2"
|
||||||
@@ -655,15 +640,8 @@ deploy_comfyui() {
|
|||||||
# generator produces, and it is what kept a stale, device-less container
|
# generator produces, and it is what kept a stale, device-less container
|
||||||
# alive through `restart: unless-stopped` (9,146 failures on
|
# alive through `restart: unless-stopped` (9,146 failures on
|
||||||
# "No HIP GPUs are available"). Never `rm`: it moves into the home trash.
|
# "No HIP GPUs are available"). Never `rm`: it moves into the home trash.
|
||||||
local legacy="$HOME/.config/systemd/user/comfyui.service"
|
retire_unit_file "$HOME/.config/systemd/user/comfyui.service" \
|
||||||
if [[ -e "$legacy" ]]; then
|
"$HOME/.AGENTTRASH/20260916-comfyui-compose-unit" comfyui.service
|
||||||
local trash="$HOME/.AGENTTRASH/20260916-comfyui-compose-unit"
|
|
||||||
mkdir -p "$trash"
|
|
||||||
systemctl --user stop comfyui.service 2>/dev/null || true
|
|
||||||
systemctl --user disable comfyui.service 2>/dev/null || true
|
|
||||||
mv -- "$legacy" "$trash/comfyui.service"
|
|
||||||
echo "retired the compose-era comfyui.service -> $trash"
|
|
||||||
fi
|
|
||||||
# The stale container carries the old config (no /dev/kfd, no /dev/dri) and
|
# The stale container carries the old config (no /dev/kfd, no /dev/dri) and
|
||||||
# would keep the quadlet's container from taking the name. It is replaced
|
# would keep the quadlet's container from taking the name. It is replaced
|
||||||
# by the unit in the next step, so nothing is lost.
|
# by the unit in the next step, so nothing is lost.
|
||||||
@@ -721,9 +699,15 @@ deploy_gitea_unit() {
|
|||||||
systemctl --user restart "$service.service"
|
systemctl --user restart "$service.service"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The root units are symlinks from the systemd system directory into the
|
||||||
|
# checkout. SYSTEMD_SYSTEM_DIR is production (/etc/systemd/system); the knob
|
||||||
|
# exists so the deploy seam can run a whole handler — relink included — in a
|
||||||
|
# sandbox and read the unit links as evidence, unprivileged.
|
||||||
|
SYSTEMD_SYSTEM_DIR="${SYSTEMD_SYSTEM_DIR:-/etc/systemd/system}"
|
||||||
|
|
||||||
relink_unit() {
|
relink_unit() {
|
||||||
local service="$1"
|
local service="$1"
|
||||||
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "/etc/systemd/system/$service.service"
|
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "$SYSTEMD_SYSTEM_DIR/$service.service"
|
||||||
}
|
}
|
||||||
|
|
||||||
# Authentik runs as three boot-persistent Quadlet units (postgres -> server ->
|
# Authentik runs as three boot-persistent Quadlet units (postgres -> server ->
|
||||||
@@ -942,17 +926,17 @@ deploy_plex() {
|
|||||||
systemctl --user restart plex.service
|
systemctl --user restart plex.service
|
||||||
# Readiness gate: /identity answers MediaContainer XML once PMS is up.
|
# Readiness gate: /identity answers MediaContainer XML once PMS is up.
|
||||||
#
|
#
|
||||||
# HTTPS, NOT HTTP. PMS drops plain HTTP off-loopback: the probe's
|
# HTTPS, NOT HTTP. PMS serves plain HTTP on loopback only: a probe
|
||||||
# connection arrives on the container's eth0, so an http:// request is
|
# arriving on the container's interface gets its connection accepted and
|
||||||
# accepted and then answered with nothing (curl 52, Empty reply) against a
|
# then an empty reply (curl 52) from a perfectly healthy server, so an
|
||||||
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full
|
# http:// probe would burn the whole timeout and fail a service that
|
||||||
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered
|
# answers https://127.0.0.1:32400/identity in milliseconds. The Caddyfile
|
||||||
# 200 in 44 ms the whole time. The same law is why the Caddyfile dials the
|
# dials the same upstream over TLS for the same reason
|
||||||
# plex upstream over TLS (tests/caddyfile.bats pins it).
|
# (tests/caddyfile.bats pins it).
|
||||||
# -k: PMS presents its own self-signed cert on 32400.
|
# -k: PMS presents its own self-signed cert on 32400.
|
||||||
#
|
#
|
||||||
# Reality check from the 2026-09 cutover: the FIRST container boot of a
|
# The budget accounts for a first boot of a large pre-existing library,
|
||||||
# years-old library ran ~10 min, so the gate still allows 600s.
|
# which spends minutes indexing before it answers anything.
|
||||||
# Fail the deploy (pending state preserved) rather than mark it deployed blind.
|
# Fail the deploy (pending state preserved) rather than mark it deployed blind.
|
||||||
for ((i = 1; i <= 120; i++)); do
|
for ((i = 1; i <= 120; i++)); do
|
||||||
curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
|
curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
|
||||||
@@ -1092,21 +1076,21 @@ deploy_anubis() {
|
|||||||
wait_container_healthy anubis 60 || return 1
|
wait_container_healthy anubis 60 || return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
# frank-large is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
|
# frank-dragoon is the halogen-flash-server CONTAINER (0.16.2, digest-pinned
|
||||||
# through scripts/frank-large.env) serving the Quat3rnion ABLITERATED
|
# through scripts/frank-dragoon.env) serving the Quat3rnion ABLITERATED
|
||||||
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
|
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint —
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
|
# qwen38-flash-next-v2-swift15-abliterated.hgn (66,687,678,464 B, ~62 GiB
|
||||||
# resident) + the REQUIRED n-gram lookup table + the vision tower —
|
# resident) + the REQUIRED n-gram lookup table + the vision tower —
|
||||||
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
|
# MTP spec decode (HALOGEN_SPEC_ADAPT=0), 2 slots x 200192, image input —
|
||||||
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script
|
# on the Strix Halo iGPU. It stays a ROOT unit because the serve script
|
||||||
# drives `podman run` directly (the frank-primary pattern): relink_unit
|
# drives `podman run` directly (the frank-archon pattern): relink_unit
|
||||||
# points /etc/systemd/system/frank-large.service at the repo file, boot
|
# points /etc/systemd/system/frank-dragoon.service at the repo file, boot
|
||||||
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
|
# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
|
||||||
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
|
# fleet parity. 2026-10-06 cutover from the halogen 27B-dense shape —
|
||||||
# rollback = revert the swap commit's trio; the 27B payload stays on
|
# rollback = revert the swap commit's trio; the 27B payload stays on
|
||||||
# disk (~/models/halogen-qwen3.8-27b).
|
# disk (~/models/halogen-qwen3.8-27b).
|
||||||
#
|
#
|
||||||
# The gates mirror serve-frank-large.sh's predicates EXACTLY (the
|
# The gates mirror serve-frank-dragoon.sh's predicates EXACTLY (the
|
||||||
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
|
# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
|
||||||
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
|
# each .hgn payload (checkpoint, n-gram table, vision tower) at the exact
|
||||||
# byte count, the tokenizer directory holding every named file as a
|
# byte count, the tokenizer directory holding every named file as a
|
||||||
@@ -1114,24 +1098,19 @@ deploy_anubis() {
|
|||||||
# artifact would leave a restart-looping unit and a dark endpoint. The
|
# artifact would leave a restart-looping unit and a dark endpoint. The
|
||||||
# port gate is the same as every cutover since 2026-09-16: nothing
|
# port gate is the same as every cutover since 2026-09-16: nothing
|
||||||
# starts until 7770 is released.
|
# starts until 7770 is released.
|
||||||
LLM_LARGE_IMAGE="${LLM_LARGE_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
||||||
LLM_LARGE_CKPT="${LLM_LARGE_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
|
LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-v2-swift15-abliterated.hgn}"
|
||||||
LLM_LARGE_CKPT_BYTES="${LLM_LARGE_CKPT_BYTES:-66687678464}"
|
LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678464}"
|
||||||
LLM_LARGE_NGRAM="${LLM_LARGE_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
|
LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-ngram.hgn}"
|
||||||
LLM_LARGE_NGRAM_BYTES="${LLM_LARGE_NGRAM_BYTES:-51200246144}"
|
LLM_DRAGOON_NGRAM_BYTES="${LLM_DRAGOON_NGRAM_BYTES:-51200246144}"
|
||||||
LLM_LARGE_VISION="${LLM_LARGE_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
|
LLM_DRAGOON_VISION="${LLM_DRAGOON_VISION:-/home/dank/models/halogen-flashnext-v2/qwen38-flash-next-vision.hgn}"
|
||||||
LLM_LARGE_VISION_BYTES="${LLM_LARGE_VISION_BYTES:-897916416}"
|
LLM_DRAGOON_VISION_BYTES="${LLM_DRAGOON_VISION_BYTES:-897916416}"
|
||||||
LLM_LARGE_TOK_DIR="${LLM_LARGE_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
|
LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-v2/tokenizer}"
|
||||||
LLM_LARGE_TOK_FILES="${LLM_LARGE_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
||||||
|
|
||||||
retire_halogen_large_quadlet() {
|
retire_halogen_dragoon_quadlet() {
|
||||||
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
|
retire_unit_file "$quadlet_unit_dir/frank-dragoon.container" \
|
||||||
systemctl --user stop frank-large.service 2>/dev/null || true
|
"$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet" frank-dragoon.service
|
||||||
if [[ -e "$quadlet_unit_dir/frank-large.container" ]]; then
|
|
||||||
mkdir -p "$trash"
|
|
||||||
mv -- "$quadlet_unit_dir/frank-large.container" "$trash/frank-large.container"
|
|
||||||
echo "retired quadlet frank-large.container -> $trash"
|
|
||||||
fi
|
|
||||||
systemctl --user daemon-reload
|
systemctl --user daemon-reload
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1153,187 +1132,159 @@ wait_port_free() {
|
|||||||
ss -tan "sport = :$port" >&2 || true
|
ss -tan "sport = :$port" >&2 || true
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
# assert_unit_owns_port SERVICE PORT — the false-green gate every LLM
|
||||||
|
# endpoint shares. A health answer proves the PORT replied, never WHO
|
||||||
|
# replied: on 2026-09-26 a leftover probe held :7770, satisfied the curl,
|
||||||
|
# and the deploy reported green over a stopped unit. Green requires the
|
||||||
|
# unit's own MainPID to be alive behind the answer.
|
||||||
|
assert_unit_owns_port() {
|
||||||
|
local service="$1" port="$2" main_pid
|
||||||
|
main_pid="$(run_privileged systemctl show -p MainPID --value "$service.service")"
|
||||||
|
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
||||||
|
echo "FAIL: :$port answered but $service.service has no MainPID — a stray process holds the port" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
echo "$service: health answered by unit MainPID $main_pid"
|
||||||
|
}
|
||||||
|
|
||||||
deploy_llm_large() {
|
# start_root_unit — the tail every LLM root unit deploys: relink the unit
|
||||||
# Refuse rather than restart-loop; every gate below has a twin in
|
# file from the checkout into /etc/systemd/system, (re)enable, restart,
|
||||||
# serve-frank-large.sh and they must agree (2026-10-01 lesson).
|
# wait for /health, and prove the answer came from the unit's own process.
|
||||||
# Capture podman output before grepping: `podman images | grep -q`
|
# The three endpoints differ in gates, ports, and budgets, never in this.
|
||||||
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
|
start_root_unit() {
|
||||||
if ! grep -qF "${LLM_LARGE_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
|
local service="$1" port="$2" health_limit="$3"
|
||||||
echo "ERROR: halogen image absent: $LLM_LARGE_IMAGE" >&2
|
relink_unit "$service"
|
||||||
echo "frank-large: pull it: podman pull ${LLM_LARGE_IMAGE%@*}" >&2
|
run_privileged systemctl daemon-reload
|
||||||
return 1
|
run_privileged systemctl enable "$service.service"
|
||||||
fi
|
run_privileged systemctl restart "$service.service"
|
||||||
if [[ ! -f "$LLM_LARGE_CKPT" ]]; then
|
wait_for_health "$service" "http://127.0.0.1:${port}/health" "$health_limit" || return 1
|
||||||
echo "ERROR: halogen checkpoint absent at $LLM_LARGE_CKPT" >&2
|
assert_unit_owns_port "$service" "$port"
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
}
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ "$(stat -c %s "$LLM_LARGE_CKPT")" != "$LLM_LARGE_CKPT_BYTES" ]]; then
|
|
||||||
echo "ERROR: halogen checkpoint is not $LLM_LARGE_CKPT_BYTES bytes (partial download?)" >&2
|
|
||||||
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
for gate in "$LLM_LARGE_NGRAM:$LLM_LARGE_NGRAM_BYTES:n-gram table" \
|
|
||||||
"$LLM_LARGE_VISION:$LLM_LARGE_VISION_BYTES:vision tower"; do
|
|
||||||
IFS=: read -r gpath gbytes gname <<<"$gate"
|
|
||||||
if [[ ! -f "$gpath" ]]; then
|
|
||||||
echo "ERROR: halogen $gname absent at $gpath" >&2
|
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
|
|
||||||
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
|
|
||||||
echo "frank-large: re-check: bash scripts/download-frank-large-flashnext.sh --verify-only" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [[ -L "$LLM_LARGE_TOK_DIR" || ! -d "$LLM_LARGE_TOK_DIR" ]]; then
|
|
||||||
echo "ERROR: tokenizer dir missing or a symlink at $LLM_LARGE_TOK_DIR" >&2
|
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
for tf in $LLM_LARGE_TOK_FILES; do
|
|
||||||
if [[ ! -f "$LLM_LARGE_TOK_DIR/$tf" || -L "$LLM_LARGE_TOK_DIR/$tf" ]]; then
|
|
||||||
echo "ERROR: tokenizer file missing or a symlink: $LLM_LARGE_TOK_DIR/$tf" >&2
|
|
||||||
echo "frank-large: fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
retire_halogen_large_quadlet
|
deploy_llm_dragoon() {
|
||||||
|
# Refuse rather than restart-loop; every gate here is the SAME library
|
||||||
|
# call the serve script makes, so the two twins cannot drift (the
|
||||||
|
# 2026-10-01 crash-loop lesson).
|
||||||
|
# Capture podman output before grepping is moot: require_image is one
|
||||||
|
# command, no pipe (the 2026-09-30 SIGPIPE lesson).
|
||||||
|
export GATE_PREFIX=frank-dragoon
|
||||||
|
require_image "$LLM_DRAGOON_IMAGE" \
|
||||||
|
"pull it: podman pull docker://${LLM_DRAGOON_IMAGE%@*}" || return 1
|
||||||
|
require_bytes "$LLM_DRAGOON_CKPT" checkpoint "$LLM_DRAGOON_CKPT_BYTES" \
|
||||||
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
|
require_bytes "$LLM_DRAGOON_NGRAM" n-gram "$LLM_DRAGOON_NGRAM_BYTES" \
|
||||||
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
|
require_bytes "$LLM_DRAGOON_VISION" vision "$LLM_DRAGOON_VISION_BYTES" \
|
||||||
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
|
# shellcheck disable=SC2086
|
||||||
|
require_dir_files "$LLM_DRAGOON_TOK_DIR" tokenizer \
|
||||||
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
|
||||||
|
$LLM_DRAGOON_TOK_FILES || return 1
|
||||||
|
|
||||||
|
retire_halogen_dragoon_quadlet
|
||||||
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
||||||
# the container) before waiting on the port. The stray sweep is keyed
|
# the container) before waiting on the port. The stray sweep is keyed
|
||||||
# on the FULL container name, which appears in the argv of a manual
|
# on the FULL container name: it appears in the argv of a manual
|
||||||
# `podman run --name frank-large-halogen` / `podman wait` but NOT in
|
# `podman run --name frank-dragoon-halogen` / `podman wait`, while a
|
||||||
# this deployer's own command line (`bash deploy-remote.sh
|
# pattern naming only the runtime would ALSO match this deployer's own
|
||||||
# frank-large`) — so it cannot shoot itself in the foot (the fleet
|
# command line — a pkill whose pattern hits its own argv takes the
|
||||||
# pkill law), and it can never match frank-secondary's llama-server.
|
# running deploy down with the target. The full name can never match
|
||||||
run_privileged systemctl stop frank-large.service 2>/dev/null || true
|
# frank-zealot's llama-server either.
|
||||||
pkill -f 'frank-large-halogen' 2>/dev/null || true
|
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
||||||
podman rm -f frank-large frank-large-halogen >/dev/null 2>&1 || true
|
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
||||||
|
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
||||||
# Nothing may start until the predecessor has released the port
|
# Nothing may start until the predecessor has released the port
|
||||||
# (2026-09-16: "Deactivated successfully" was not "port free").
|
# (2026-09-16: "Deactivated successfully" was not "port free").
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-large
|
start_root_unit frank-dragoon 7770 1200
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-large.service
|
|
||||||
run_privileged systemctl restart frank-large.service
|
|
||||||
wait_for_health frank-large http://127.0.0.1:7770/health 1200
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_secondary — frank-secondary is llama.cpp (master build, CUDA sm_86) on
|
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
|
||||||
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
||||||
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
|
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
|
||||||
# no generated unit to avoid. relink_unit points
|
# no generated unit to avoid. relink_unit points
|
||||||
# /etc/systemd/system/frank-secondary.service at the repo file and boot
|
# /etc/systemd/system/frank-zealot.service at the repo file and boot
|
||||||
# persistence is that unit's [Install] section.
|
# persistence is that unit's [Install] section.
|
||||||
#
|
#
|
||||||
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
|
# It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
|
||||||
# not shared: frank-large's engine runs on the gfx1151 iGPU.
|
# not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
|
||||||
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
||||||
LLM_SECONDARY_RUNTIME="${LLM_SECONDARY_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
||||||
# The projector IS the vision capability here: the model is natively
|
# The projector IS the vision capability here: the model is natively
|
||||||
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
||||||
# its own BF16 projector in the same repo, so a deploy that starts without
|
# its own BF16 projector in the same repo, so a deploy that starts without
|
||||||
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
||||||
# frank-large's.
|
# frank-dragoon's.
|
||||||
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
||||||
|
|
||||||
deploy_llm_secondary() {
|
deploy_llm_zealot() {
|
||||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why. Same library calls
|
||||||
if [[ ! -x "$LLM_SECONDARY_RUNTIME" ]]; then
|
# as serve-frank-zealot.sh, so the deployer's twin refuses identically.
|
||||||
echo "ERROR: llama.cpp runtime absent at $LLM_SECONDARY_RUNTIME" >&2
|
export GATE_PREFIX=frank-zealot
|
||||||
echo "frank-secondary: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
|
require_exec "$LLM_ZEALOT_RUNTIME" runtime \
|
||||||
echo "frank-secondary: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
|
"build it: cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86" \
|
||||||
echo "frank-secondary: && cmake --build build-cuda --target llama-server) first." >&2
|
|| return 1
|
||||||
return 1
|
require_file "$LLM_ZEALOT_MODEL_FILE" gguf \
|
||||||
fi
|
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
|
||||||
for f in "$LLM_SECONDARY_MODEL_FILE" "$LLM_SECONDARY_MMPROJ"; do
|
require_file "$LLM_ZEALOT_MMPROJ" mmproj \
|
||||||
if [[ ! -f "$f" ]]; then
|
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
|
||||||
echo "ERROR: frank-secondary payload file absent at $f" >&2
|
|
||||||
echo "frank-secondary: run scripts/download-frank-secondary.sh first;" >&2
|
start_root_unit frank-zealot 7771 900
|
||||||
echo "frank-secondary: refusing to start a unit that cannot load." >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
relink_unit frank-secondary
|
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-secondary.service
|
|
||||||
run_privileged systemctl restart frank-secondary.service
|
|
||||||
wait_for_health frank-secondary http://127.0.0.1:7771/health 900 || return 1
|
|
||||||
# False-green hardening (frank-large's lesson): the port answering
|
|
||||||
# proves nothing about WHO answers — the unit must be active with a
|
|
||||||
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
|
|
||||||
local main_pid
|
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-secondary.service)"
|
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
|
||||||
echo "FAIL: :7771 answered but frank-secondary.service has no MainPID — a stray process holds the port" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "frank-secondary: health answered by unit MainPID $main_pid"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_primary — frank-primary is the radiance engine (StillDeadcode,
|
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
||||||
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
||||||
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
||||||
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
||||||
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
||||||
# the image entrypoint; the image lives in dank's rootless podman storage).
|
# the image entrypoint; the image lives in dank's rootless podman storage).
|
||||||
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
||||||
# frank-primary-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
||||||
# fork (tag frank-primary-vllm-shapeG-20261003). Same port and alias; the
|
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
||||||
# harness models.yml id is unchanged.
|
# harness models.yml id is unchanged.
|
||||||
#
|
#
|
||||||
# Gates: the image must exist under its digest ref in dank's rootless
|
# Gates: the image must exist under its digest ref in dank's rootless
|
||||||
# storage (FRANK_PRIMARY_IMAGE), the .rad container file must exist at the
|
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
||||||
# env's size floor (FRANK_PRIMARY_MODEL_FILE >= FRANK_PRIMARY_RAD_MIN_MIB —
|
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
||||||
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
||||||
# must be complete. Per-file sha256 verification is the fetch script's job
|
# must be complete. Per-file sha256 verification is the fetch script's job
|
||||||
# at download time and rad-convert/rad-info's job at build time, never
|
# at download time and rad-convert/rad-info's job at build time, never
|
||||||
# re-hashed here.
|
# re-hashed here.
|
||||||
LLM_PRIMARY_CONTAINER=frank-primary
|
LLM_ARCHON_CONTAINER=frank-archon
|
||||||
LLM_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
|
LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
|
||||||
|
|
||||||
deploy_llm_primary() {
|
deploy_llm_archon() {
|
||||||
# Refuse rather than restart-loop: a unit whose image or payload is
|
# Refuse rather than restart-loop: a unit whose image or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why. The gates are the
|
||||||
local env_file="/home/dank/serverAppConfigs/scripts/frank-primary.env"
|
# shared library's, called with the values the unit's own env file
|
||||||
local image model_dir model_file rad_min mib
|
# declares — so serve-frank-archon.sh and this handler cannot disagree,
|
||||||
|
# and the payload path lives in exactly one file.
|
||||||
|
export GATE_PREFIX=frank-archon
|
||||||
|
local env_file="${LLM_ARCHON_ENV_FILE:-$REMOTE_DIR/scripts/frank-archon.env}"
|
||||||
|
local image model_dir model_file rad_min
|
||||||
if [[ ! -f "$env_file" ]]; then
|
if [[ ! -f "$env_file" ]]; then
|
||||||
echo "ERROR: frank-primary env file absent: $env_file" >&2
|
echo "frank-archon: env: missing at $env_file" >&2
|
||||||
|
echo " hint: deploy the repo checkout first ($REMOTE_DIR)" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
image="$(sed -n 's/^FRANK_PRIMARY_IMAGE=//p' "$env_file" | tail -1)"
|
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
|
||||||
model_dir="$(sed -n 's/^FRANK_PRIMARY_MODEL_DIR=//p' "$env_file" | tail -1)"
|
model_dir="$(sed -n 's/^FRANK_ARCHON_MODEL_DIR=//p' "$env_file" | tail -1)"
|
||||||
model_file="$(sed -n 's/^FRANK_PRIMARY_MODEL_FILE=//p' "$env_file" | tail -1)"
|
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
|
||||||
rad_min="$(sed -n 's/^FRANK_PRIMARY_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
||||||
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
||||||
echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
echo "frank-archon: env: missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
||||||
return 1
|
echo " hint: scripts/frank-archon.env" >&2
|
||||||
fi
|
|
||||||
# The image gate is the digest ref itself: a name@sha256 ref exists in
|
|
||||||
# storage only if THAT manifest was pulled. One command — no
|
|
||||||
# pipefail/grep race (the 2026-09-30 lesson).
|
|
||||||
if ! podman image exists "$image"; then
|
|
||||||
echo "ERROR: runtime image absent from storage: $image" >&2
|
|
||||||
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ ! -f "$model_dir/$model_file" ]]; then
|
|
||||||
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
|
||||||
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
|
||||||
if [[ "$mib" -lt "$rad_min" ]]; then
|
|
||||||
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
|
||||||
echo "frank-primary: rebuild it (scripts/build-frank-primary-rad.sh --force)" >&2
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
require_image "$image" \
|
||||||
|
"pull it: podman pull docker://stilldeadcode/radiance:latest (as dank)" \
|
||||||
|
|| return 1
|
||||||
|
require_min_mib "$model_dir/$model_file" ".rad" "$rad_min" \
|
||||||
|
"convert it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" \
|
||||||
|
|| return 1
|
||||||
|
|
||||||
# A stale container with the same name makes `podman run --name` fail at
|
# A stale container with the same name makes `podman run --name` fail at
|
||||||
# start; remove any leftover from a failed previous attempt.
|
# start; remove any leftover from a failed previous attempt.
|
||||||
@@ -1343,50 +1294,24 @@ deploy_llm_primary() {
|
|||||||
# The legacy identities (the retired R9V container, the ad hoc
|
# The legacy identities (the retired R9V container, the ad hoc
|
||||||
# experiment container qwen-fp8, the retired glm-probe) hold the same
|
# experiment container qwen-fp8, the retired glm-probe) hold the same
|
||||||
# GPUs and die here too.
|
# GPUs and die here too.
|
||||||
run_privileged systemctl stop frank-primary.service 2>/dev/null || true
|
run_privileged systemctl stop frank-archon.service 2>/dev/null || true
|
||||||
local ctr
|
local ctr
|
||||||
for ctr in $LLM_PRIMARY_LEGACY_CONTAINERS "$LLM_PRIMARY_CONTAINER"; do
|
for ctr in $LLM_ARCHON_LEGACY_CONTAINERS "$LLM_ARCHON_CONTAINER"; do
|
||||||
podman rm -f "$ctr" >/dev/null 2>&1 || true
|
podman rm -f "$ctr" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
# Nothing starts until the port is released: the lucebox cutover
|
# Nothing starts until the port is released: the lucebox cutover
|
||||||
# (2026-09-26) and the frank-large cutover (2026-09-16) both failed
|
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
|
||||||
# exactly here when the next engine bound 7770 a second too early.
|
# exactly here when the next engine bound 7770 a second too early.
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-primary
|
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-primary.service
|
|
||||||
run_privileged systemctl restart frank-primary.service
|
|
||||||
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
||||||
# publisher's compose healthcheck allows a 15-minute start_period).
|
# publisher's compose healthcheck allows a 15-minute start_period) —
|
||||||
# Parity with every predecessor on this port. The HTTP listener binds
|
# the 900-iteration budget is 1800s, parity with every predecessor on
|
||||||
# AFTER the full load (main.cpp: engine.start() then engine.serve()),
|
# this port. The HTTP listener binds AFTER the full load (main.cpp:
|
||||||
# so during load the port REFUSES connections — a bare curl is honest
|
# engine.start() then engine.serve()), so during load the port REFUSES
|
||||||
# here, unlike FreeToken's 200-while-loading body. (The MainPID check
|
# connections — a bare curl is honest here, unlike FreeToken's
|
||||||
# below still guards the stray-port case.)
|
# 200-while-loading body. (start_root_unit's MainPID check still
|
||||||
local i ok=0
|
# guards the stray-port case.)
|
||||||
for ((i=1; i<=900; i++)); do
|
start_root_unit frank-archon 7770 900
|
||||||
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
|
|
||||||
echo "frank-primary engine healthy after approximately $((i * 2))s"
|
|
||||||
ok=1
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
sleep 2
|
|
||||||
done
|
|
||||||
if [[ "$ok" != 1 ]]; then
|
|
||||||
echo "FAIL: frank-primary /health never answered 200 in 1800s" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
|
|
||||||
# port cannot tell WHO answers — a stray probe holding :7770 satisfies
|
|
||||||
# it while the unit is stopped. The deploy is only green if the unit
|
|
||||||
# itself is active with a live MainPID behind the health answer.
|
|
||||||
local main_pid
|
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-primary.service)"
|
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
|
||||||
echo "FAIL: :7770 answered but frank-primary.service has no MainPID — a stray process holds the port" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "frank-primary: health answered by unit MainPID $main_pid"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
wait_for_health() {
|
wait_for_health() {
|
||||||
@@ -1573,9 +1498,9 @@ deploy_service() {
|
|||||||
anubis) deploy_anubis ;;
|
anubis) deploy_anubis ;;
|
||||||
seerr) deploy_seerr ;;
|
seerr) deploy_seerr ;;
|
||||||
marreta) deploy_marreta ;;
|
marreta) deploy_marreta ;;
|
||||||
frank-large) deploy_llm_large ;;
|
frank-dragoon) deploy_llm_dragoon ;;
|
||||||
frank-secondary) deploy_llm_secondary ;;
|
frank-zealot) deploy_llm_zealot ;;
|
||||||
frank-primary) deploy_llm_primary ;;
|
frank-archon) deploy_llm_archon ;;
|
||||||
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
|
hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
|
||||||
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
|
romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
|
||||||
jellyfin) deploy_jellyfin ;;
|
jellyfin) deploy_jellyfin ;;
|
||||||
@@ -1586,7 +1511,12 @@ deploy_service() {
|
|||||||
tautulli) deploy_tautulli ;;
|
tautulli) deploy_tautulli ;;
|
||||||
prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
|
prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
|
||||||
comfyui) deploy_comfyui ;;
|
comfyui) deploy_comfyui ;;
|
||||||
*) deploy_compose_service "$service" ;;
|
*)
|
||||||
|
# Every deployable service on every target has a handler above
|
||||||
|
# (inventory = allow-list, so nothing unreachable arrives).
|
||||||
|
echo "no handler for service: $service" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+8
-26
@@ -35,7 +35,7 @@ Options:
|
|||||||
Examples:
|
Examples:
|
||||||
./scripts/deploy.sh dankclaw
|
./scripts/deploy.sh dankclaw
|
||||||
./scripts/deploy.sh dankclaw --service searxng
|
./scripts/deploy.sh dankclaw --service searxng
|
||||||
./scripts/deploy.sh dankserver --service frank-primary
|
./scripts/deploy.sh dankserver --service frank-archon
|
||||||
./scripts/deploy.sh dankclaw --dry-run
|
./scripts/deploy.sh dankclaw --dry-run
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
@@ -131,31 +131,13 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
valid_service() {
|
valid_service() {
|
||||||
local service="$1"
|
# The inventory is the only allow-list (see scripts/service-map.py);
|
||||||
case "$TARGET:$service" in
|
# service-map's "allows" exit code is the answer. The former literal
|
||||||
dankclaw:caddy|dankclaw:gluetun|dankclaw:searxng-network|dankclaw:searxng-valkey|\
|
# case list here drifted from reality (it still listed the *arr stack
|
||||||
dankclaw:searxng|dankclaw:marreta|dankclaw:pihole|dankclaw:open-webui|dankclaw:gitea-postgres|\
|
# under dankserver, which moved to frankbook-pro, and a
|
||||||
dankclaw:gitea-server|dankclaw:homepage|\
|
# "searxng-network" name that is a generated systemd unit, not a
|
||||||
dankclaw:comfyui|dankclaw:hermes|dankclaw:hermes-dashboard|\
|
# deployable service).
|
||||||
dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|dankclaw:loki|dankclaw:alloy|\
|
python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$TARGET" --service "$1" >/dev/null
|
||||||
dankclaw:alertmanager|\
|
|
||||||
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
|
||||||
dankclaw:frank-large|dankclaw:frank-secondary|\
|
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
|
||||||
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
|
|
||||||
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
|
|
||||||
dankserver:frank-primary|\
|
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
|
||||||
frankbook-pro:readarr)
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for service in "${EXPLICIT_SERVICES[@]}"; do
|
for service in "${EXPLICIT_SERVICES[@]}"; do
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
||||||
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
||||||
|
|
||||||
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
The selection rule (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
||||||
an expert whose share of its layer's down-input energy is at least 3% AND
|
an expert whose share of its layer's down-input energy is at least 3% AND
|
||||||
at least three times its share of the routing is kept bf16; the count must
|
at least three times its share of the routing is kept bf16; the count must
|
||||||
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||||
|
|||||||
+9
-8
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary-swift15.sh — payload fetch/verify for the planned
|
# download-frank-archon-swift15.sh — payload fetch/verify for the planned
|
||||||
# frank-primary swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
|
# frank-archon swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
|
||||||
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
|
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
|
||||||
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
|
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
|
||||||
# template + licenses). Key set verified identical to the official
|
# template + licenses). Key set verified identical to the official
|
||||||
@@ -8,16 +8,17 @@
|
|||||||
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
|
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
|
||||||
# Repo is UNGATED — no token needed (never put one on the command line).
|
# Repo is UNGATED — no token needed (never put one on the command line).
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary-swift15.sh --fetch resume/complete
|
# bash scripts/download-frank-archon-swift15.sh --fetch resume/complete
|
||||||
# bash scripts/download-frank-primary-swift15.sh verify only
|
# bash scripts/download-frank-archon-swift15.sh verify only
|
||||||
#
|
#
|
||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
# Verify (default) is three independent agreements: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size (catches truncation), the shard count
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# matching the weight-map index (catches omissions), and every shard's
|
||||||
# only the public API.
|
# sha256 equal to the Hub's LFS oid (catches substituted content).
|
||||||
|
# Verify needs only the public API.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
||||||
@@ -1,21 +1,22 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-primary.sh — payload fetch/verify for frank-primary:
|
# download-frank-archon.sh — payload fetch/verify for frank-archon:
|
||||||
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
|
# the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint (131 safetensors
|
||||||
# shards + weight-map index + tokenizer + vision preprocessing configs,
|
# shards + weight-map index + tokenizer + vision preprocessing configs,
|
||||||
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
|
# ~172.8 GB). 2026-10-05: replaces the orcarouter uncensored payload of the
|
||||||
# FreeToken era (that script lived here; its repo was gated and needed
|
# FreeToken era (that script lived here; its repo was gated and needed
|
||||||
# HF_TOKEN — this one is UNGATED and needs no token at all).
|
# HF_TOKEN — this one is UNGATED and needs no token at all).
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-primary.sh --fetch resume/complete
|
# bash scripts/download-frank-archon.sh --fetch resume/complete
|
||||||
# bash scripts/download-frank-primary.sh verify only
|
# bash scripts/download-frank-archon.sh verify only
|
||||||
#
|
#
|
||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
# Verify (default) is three independent agreements: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size (catches truncation), the shard count
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# matching the weight-map index (catches omissions), and every shard's
|
||||||
# only the public API.
|
# sha256 equal to the Hub's LFS oid (catches substituted content).
|
||||||
|
# Verify needs only the public API.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="Qwen/Qwen3.8-Flash-Next-FP8"
|
REPO="Qwen/Qwen3.8-Flash-Next-FP8"
|
||||||
+13
-8
@@ -1,15 +1,18 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-large-flashnext.sh — fetch frank-large's flash-server v2
|
# download-frank-dragoon-flashnext.sh — fetch frank-dragoon's flash-server v2
|
||||||
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
|
# payload: the Quat3rnion abliterated Swift1.5 checkpoint of peonist's v2
|
||||||
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
||||||
# card pins), and the flat tokenizer the two repos share. All public, no
|
# card pins), and the flat tokenizer the two repos share. All public, no
|
||||||
# token — and per the fleet secret law, none would ever touch argv.
|
# token needed — and a secret would never be passed via argv anyway: the
|
||||||
|
# command line is world-readable through /proc, so any token placed there
|
||||||
|
# leaks to every process-listing tool on the host.
|
||||||
#
|
#
|
||||||
# The fetch law, same as every sibling script: curl -f (an access error
|
# The fetch rules, uniform across every sibling script: curl -f so an
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# access error fails loudly instead of writing the error body as a
|
||||||
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an
|
# "successful" tiny artifact; the exact byte count from the Hub tree API;
|
||||||
# existing payload without downloading.
|
# LFS sha256 verification; --verify-only to re-check an existing payload
|
||||||
|
# without downloading.
|
||||||
#
|
#
|
||||||
# File manifest (verified against the Hub tree API on 2026-10-06):
|
# File manifest (verified against the Hub tree API on 2026-10-06):
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn 66,687,678,464 B sha256
|
# qwen38-flash-next-v2-swift15-abliterated.hgn 66,687,678,464 B sha256
|
||||||
@@ -26,10 +29,12 @@
|
|||||||
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
||||||
# already on disk (tokenizer.json LFS
|
# already on disk (tokenizer.json LFS
|
||||||
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
||||||
# fetched fresh rather than symlinked, per the no-symlink law.
|
# fetched fresh, never symlinked: a snapshot link points into the
|
||||||
|
# host-side cache directory, which does not exist inside the
|
||||||
|
# container mount, so the link dangles exactly where it is consumed.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
||||||
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
|
ABLIT="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated/resolve/main"
|
||||||
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
|
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
|
||||||
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
|
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
|
||||||
+10
-8
@@ -1,16 +1,18 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-large-halogen.sh — fetch frank-large's halogen payload:
|
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
||||||
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
||||||
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
|
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and a token
|
||||||
# secret law, none would ever touch argv).
|
# would never travel in argv: command lines are world-readable through
|
||||||
|
# /proc, so a secret placed there leaks to any process-listing tool).
|
||||||
#
|
#
|
||||||
# The fetch law, same as every sibling script: curl -f (an access error
|
# The fetch rules, uniform across every sibling script: curl -f so an
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# access error fails loudly instead of writing the error body as a
|
||||||
# the model card, LFS sha256 verification, --verify-only to re-check an
|
# "successful" tiny artifact; the exact byte count from the model card;
|
||||||
# existing payload without downloading.
|
# LFS sha256 verification; --verify-only to re-check an existing payload
|
||||||
|
# without downloading.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
||||||
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
|
BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
|
||||||
|
|
||||||
FILES=(qwen3.8-27b-p1w4d-d2.hgn
|
FILES=(qwen3.8-27b-p1w4d-d2.hgn
|
||||||
+3
-3
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
|
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
|
||||||
# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token
|
# frank-dragoon (Dankclaw iGPU). Public repo — no auth header, and never a token
|
||||||
# on the command line (it is world-readable in /proc/<pid>/cmdline).
|
# on the command line (it is world-readable in /proc/<pid>/cmdline).
|
||||||
#
|
#
|
||||||
# The payload is the repo's Q6 AGENT rung:
|
# The payload is the repo's Q6 AGENT rung:
|
||||||
@@ -10,7 +10,7 @@
|
|||||||
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
|
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
|
||||||
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
|
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
|
||||||
#
|
#
|
||||||
# The other two files frank-large needs are NOT re-downloaded:
|
# The other two files frank-dragoon needs are NOT re-downloaded:
|
||||||
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
|
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
|
||||||
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
|
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
|
||||||
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
|
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
|
||||||
@@ -25,7 +25,7 @@
|
|||||||
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
|
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
|
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large
|
# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-dragoon
|
||||||
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
|
# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
|
||||||
# command line (it is world-readable in /proc/<pid>/cmdline).
|
# command line (it is world-readable in /proc/<pid>/cmdline).
|
||||||
#
|
#
|
||||||
@@ -24,7 +24,7 @@
|
|||||||
# come from the Hub LFS oids (api/models/<repo>/tree/main).
|
# come from the Hub LFS oids (api/models/<repo>/tree/main).
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-dragoon-kingjones.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
|
REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large
|
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-dragoon
|
||||||
# model) on Dankclaw.
|
# model) on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-orca.sh
|
# bash scripts/download-frank-dragoon-orca.sh
|
||||||
# bash scripts/download-frank-large-orca.sh --verify-only
|
# bash scripts/download-frank-dragoon-orca.sh --verify-only
|
||||||
#
|
#
|
||||||
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
|
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
|
||||||
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
|
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
|
||||||
@@ -105,4 +105,4 @@ fi
|
|||||||
echo "verifying assembled files against SHA256SUMS"
|
echo "verifying assembled files against SHA256SUMS"
|
||||||
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
|
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
|
||||||
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
|
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the Q4_K_XL frank-large payload on Dankclaw:
|
# Download the Q4_K_XL frank-dragoon payload on Dankclaw:
|
||||||
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
|
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-q4.sh
|
# bash scripts/download-frank-dragoon-q4.sh
|
||||||
# bash scripts/download-frank-large-q4.sh --verify-only
|
# bash scripts/download-frank-dragoon-q4.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: the file is checked against its declared byte count and sha256
|
# Integrity: the file is checked against its declared byte count and sha256
|
||||||
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
|
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
|
||||||
@@ -21,12 +21,12 @@
|
|||||||
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
|
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
|
||||||
#
|
#
|
||||||
# The speculative drafter is UNCHANGED and still fetched by its own script
|
# The speculative drafter is UNCHANGED and still fetched by its own script
|
||||||
# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not
|
# (scripts/download-frank-dragoon.sh, z-lab's Hub repo). That name must not
|
||||||
# appear here at all — the frank-large seam greps THIS file for it to prove
|
# appear here at all — the frank-dragoon seam greps THIS file for it to prove
|
||||||
# the drafter was not folded into the payload fetch.
|
# the drafter was not folded into the payload fetch.
|
||||||
#
|
#
|
||||||
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
|
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
|
||||||
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it —
|
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-dragoon does not need it —
|
||||||
# spec decode here is the block-diffusion drafter, not MTP.
|
# spec decode here is the block-diffusion drafter, not MTP.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the Swift-1.5 frank-large candidate on Dankclaw:
|
# Download the Swift-1.5 frank-dragoon candidate on Dankclaw:
|
||||||
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
||||||
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-swift.sh
|
# bash scripts/download-frank-dragoon-swift.sh
|
||||||
# bash scripts/download-frank-large-swift.sh --verify-only
|
# bash scripts/download-frank-dragoon-swift.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: the file is checked against its declared byte count and sha256
|
# Integrity: the file is checked against its declared byte count and sha256
|
||||||
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
|
# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
|
||||||
@@ -15,9 +15,9 @@
|
|||||||
# Layout after a successful run (MODEL_DIR):
|
# Layout after a successful run (MODEL_DIR):
|
||||||
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
|
# Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf 31457990272
|
||||||
#
|
#
|
||||||
# This is the CANDIDATE payload, not the deployed one. frank-large still
|
# This is the CANDIDATE payload, not the deployed one. frank-dragoon still
|
||||||
# serves scripts/download-frank-large.sh's output; do not repoint
|
# serves scripts/download-frank-dragoon.sh's output; do not repoint
|
||||||
# frank-large.env at MODEL_DIR until the gufo load test passes.
|
# frank-dragoon.env at MODEL_DIR until the gufo load test passes.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
|
MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
|
# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
|
||||||
# frank-large model) on Dankclaw.
|
# frank-dragoon model) on Dankclaw.
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large-unsloth.sh
|
# bash scripts/download-frank-dragoon-unsloth.sh
|
||||||
# bash scripts/download-frank-large-unsloth.sh --verify-only
|
# bash scripts/download-frank-dragoon-unsloth.sh --verify-only
|
||||||
#
|
#
|
||||||
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
|
# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
|
||||||
# 50 GB). The parts ARE the canonical payload: the serve script points -m at
|
# 50 GB). The parts ARE the canonical payload: the serve script points -m at
|
||||||
@@ -112,4 +112,4 @@ if [[ "$mode" == "download" ]]; then
|
|||||||
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
|
[[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the frank-large payload on Dankclaw:
|
# Download the frank-dragoon payload on Dankclaw:
|
||||||
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
||||||
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
||||||
#
|
#
|
||||||
# bash scripts/download-frank-large.sh
|
# bash scripts/download-frank-dragoon.sh
|
||||||
# bash scripts/download-frank-large.sh --verify-only
|
# bash scripts/download-frank-dragoon.sh --verify-only
|
||||||
#
|
#
|
||||||
# Integrity: every file is checked against its declared byte count and sha256
|
# Integrity: every file is checked against its declared byte count and sha256
|
||||||
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
||||||
@@ -86,4 +86,4 @@ if [[ "$mode" == "download" ]]; then
|
|||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "frank-large payload complete: $MODEL_DIR"
|
echo "frank-dragoon payload complete: $MODEL_DIR"
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
|
# Download the frank-zealot payload for llama.cpp on the RTX 3090:
|
||||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
||||||
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
||||||
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
||||||
@@ -21,7 +21,7 @@
|
|||||||
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
# scripts/download-frank-zealot.sh [DEST_DIR] [--verify-only]
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST=""
|
DEST=""
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Measure per-block / per-expert expert-routing activity on live frank-primary.
|
"""Measure per-block / per-expert expert-routing activity on live frank-archon.
|
||||||
|
|
||||||
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
|
Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
|
||||||
on the llama-server process while concurrent generations of mixed content run.
|
on the llama-server process while concurrent generations of mixed content run.
|
||||||
@@ -11,13 +11,13 @@ Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
|
|||||||
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
||||||
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
||||||
|
|
||||||
Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed
|
What this instrument settles: per-BLOCK expert read volume is FLAT across
|
||||||
streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT
|
all CPU-resident blocks, because MoE top-k routing fires every block on
|
||||||
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
|
every token — there are no "most active layers" at `-ot` granularity. The
|
||||||
routing fires every block every token — there are no "most active layers" at
|
real skew is per-EXPERT: a small fraction of (block,expert) cells carries
|
||||||
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
|
most of the samples, and tensor-level overrides cannot reach that level.
|
||||||
cells carry ~40% of samples), which tensor-level overrides cannot reach.
|
Any fixed number of GPU-resident blocks is therefore equivalent; COUNT is
|
||||||
Any 8 GPU-resident blocks are therefore equivalent; COUNT is the only lever.
|
the only lever.
|
||||||
|
|
||||||
Usage (needs passwordless sudo for perf only; model files must be readable):
|
Usage (needs passwordless sudo for perf only; model files must be readable):
|
||||||
fp-expert-activity.py <pid> [duration_s] # dump + sample + analyze
|
fp-expert-activity.py <pid> [duration_s] # dump + sample + analyze
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving
|
||||||
|
# ukisai/Swift1.5-Qwen3.8-Flash-Next converted to GPTQ w4 (MODEL_FILE below,
|
||||||
|
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
|
||||||
|
# dankserver :7770.
|
||||||
|
#
|
||||||
|
# SHAPE (2026-10-11): 2 decode slots x 200,000 tokens with an UNQUANTIZED
|
||||||
|
# (bf16) KV cache — the pool is sized for the pair, and the weights slab pays
|
||||||
|
# for it (see the KV block). The fp8-native payload this file was written for
|
||||||
|
# is the rollback target and keeps its own trio under the fp8-native runbook.
|
||||||
|
#
|
||||||
|
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
||||||
|
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
|
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
|
||||||
|
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
|
||||||
|
# frank-archon-vllm-shapeG-20261003).
|
||||||
|
#
|
||||||
|
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
|
||||||
|
# gfx1201, drives `podman run` directly from the digest-pinned
|
||||||
|
# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
|
||||||
|
# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
|
||||||
|
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
||||||
|
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
||||||
|
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
||||||
|
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
|
||||||
|
# --host-pool-mib 12288) remains the documented fallback if the native
|
||||||
|
# container proves unbootable.
|
||||||
|
#
|
||||||
|
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
|
||||||
|
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
|
||||||
|
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
|
||||||
|
# staging dir whose config.json drops `quantization_config`: the descriptor
|
||||||
|
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
|
||||||
|
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
|
||||||
|
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
||||||
|
# containers carry. Per-weight encodings are data, not the descriptor —
|
||||||
|
# removing the key changes nothing about what the weights hold. Built by
|
||||||
|
# scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
|
||||||
|
#
|
||||||
|
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
||||||
|
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
||||||
|
# then engine.serve()) — during load the port refuses connections, so a
|
||||||
|
# bare curl /health gate is honest. /health == /ping: 200 JSON while
|
||||||
|
# serving, 503 only while stopping (core/server/admin.cpp:385).
|
||||||
|
# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
|
||||||
|
# `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
|
||||||
|
# the chat format. Both need no parser flags.
|
||||||
|
# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
|
||||||
|
# typical_p, presence_penalty, frequency_penalty, repetition_penalty,
|
||||||
|
# repeat_last_n, chat_template_kwargs, reasoning_effort,
|
||||||
|
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
||||||
|
# models.yml's extraBody set is fully inside it. Unknown params are
|
||||||
|
# refused (400), which is why nothing unverified rides the wire.
|
||||||
|
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
|
||||||
|
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
|
||||||
|
# publisher's throughput peak for this model. Stated explicitly with
|
||||||
|
# --num-speculative-tokens 3.
|
||||||
|
# - The fused TP all-reduce is declared over max-num-seqs x
|
||||||
|
# (1 + --num-speculative-tokens) rows; 2 x 3 = 6 rows, under the 25-row
|
||||||
|
# exactness bound of the wht6 wire.
|
||||||
|
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
||||||
|
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
||||||
|
# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
|
||||||
|
# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
|
||||||
|
# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
|
||||||
|
# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
|
||||||
|
# Acceptance on the greedy harness is deterministic (the control
|
||||||
|
# reproduced the recorded 0.755 exactly), so the drop is a real numerics
|
||||||
|
# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
|
||||||
|
# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
|
||||||
|
# The vram-pair translation itself is exact (1.2.0 boot ledger took
|
||||||
|
# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
|
||||||
|
# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
|
||||||
|
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
|
||||||
|
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
|
||||||
|
# file already uses the pair, which BOTH pins accept (rollback symmetry).
|
||||||
|
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||||
|
# The .rad container FILE (mounted under /models) and the state dir for the
|
||||||
|
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
||||||
|
# .rad files; the raw HF download and its staging dir live beside it and
|
||||||
|
# are never mounted.
|
||||||
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
||||||
|
FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||||
|
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
||||||
|
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
||||||
|
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
||||||
|
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
|
||||||
|
# cache must ride a host mount — serve script mounts it, see the runbook's
|
||||||
|
# "Tuning levers").
|
||||||
|
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
|
||||||
|
# name across systemctl and podman ps. The runtime changes; the name does
|
||||||
|
# not. The legacy list retires every predecessor identity that could hold
|
||||||
|
# the GPUs or the port.
|
||||||
|
FRANK_ARCHON_CONTAINER=frank-archon
|
||||||
|
FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
|
||||||
|
# Served model id — Protoss name (wire rename 2026-10-09). The container,
|
||||||
|
# unit, scripts and this file keep the frank-archon name above; only the
|
||||||
|
# alias on the wire moved. Radiance validates the requested model name, so
|
||||||
|
# every client (harness, bench scripts, probes) MUST send this value.
|
||||||
|
FRANK_ARCHON_SERVED_NAME=frank-archon
|
||||||
|
FRANK_ARCHON_PORT=7770
|
||||||
|
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
|
||||||
|
# repo's root chat_template.jinja, vendored here; the checkpoint's own
|
||||||
|
# template is baked into the .rad by rad-convert and this overrides it).
|
||||||
|
# Radiance derives the reasoning/tool-call reply format from it.
|
||||||
|
FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
|
||||||
|
|
||||||
|
# --- engine shape (the boot ledger arbitrates every number below) ---------
|
||||||
|
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
|
||||||
|
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
|
||||||
|
# wire is libr4d's own, not RCCL.
|
||||||
|
FRANK_ARCHON_TP=2
|
||||||
|
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
|
||||||
|
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
||||||
|
# engine default.
|
||||||
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
|
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
|
||||||
|
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
||||||
|
# block below). Slot-count limits observed on other runtimes do NOT
|
||||||
|
# transfer: a "this slot count cannot boot" refusal is usually that
|
||||||
|
# runtime's compiler specializing shapes (e.g. a ConstraintViolationError
|
||||||
|
# from shape specialization on query_start_loc), not a property of the
|
||||||
|
# engine now running. This engine boots seqs 2, and seqs 2
|
||||||
|
# vs 3 decode measurements land inside each other's noise floor. Side
|
||||||
|
# benefit: the fused TP all-reduce is declared over seqs x (1 + spec) = 6
|
||||||
|
# rows, under the 25-row bound at which the wht6 wire stays EXACT
|
||||||
|
# (single-sequence decode and any step of <= 25 rows is bit-exact; only
|
||||||
|
# busier steps are lossy).
|
||||||
|
FRANK_ARCHON_MAX_SEQS=2
|
||||||
|
# Served window. Native is 262,144; 200,000 is the publisher's production
|
||||||
|
# value (compose) and what models.yml's contextWindow must quote — a longer
|
||||||
|
# prompt is refused by the engine, never truncated. The engine treats
|
||||||
|
# `kv group tokens >= max-model-len` as a HARD limit ("ONE request longer
|
||||||
|
# than N tokens is accepted by the API and then refused as too large to ever
|
||||||
|
# fit — this is a hard limit, not a backing ratio"), so the pool is sized
|
||||||
|
# for seqs x maxlen, not for one sequence.
|
||||||
|
FRANK_ARCHON_MAX_MODEL_LEN=200000
|
||||||
|
FRANK_ARCHON_PLACEMENT=expert_tiered
|
||||||
|
# Pinned host pool for the expert rows that do not fit VRAM. vram_weights and
|
||||||
|
# this pool serve the per-rank resident set TOGETHER: at the fp8-KV pair
|
||||||
|
# (26071/3543) the ledger showed the VRAM slab filled to 259 KiB of slack,
|
||||||
|
# i.e. the pair was exactly sized. Cutting vram-weights by 3.8 GiB to pay for
|
||||||
|
# the bf16 KV pool SPILLS ~3.8 GiB/rank here, so the pool moves with it —
|
||||||
|
# under-sizing it is a boot failure (36864 refused the boot in the fp8 era).
|
||||||
|
# The boot ledger's coverage line remains the referee, not this arithmetic.
|
||||||
|
FRANK_ARCHON_HOST_POOL_MIB=44928
|
||||||
|
# VRAM left unclaimed per card (publisher default 96 MiB).
|
||||||
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
|
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
||||||
|
# THE CEILING, measured on this host 2026-10-11: card 31,856 MiB, a boot
|
||||||
|
# consumes 1,710 MiB OUTSIDE both budgets (driver context + activation arena
|
||||||
|
# 990 + hardware queues 100 + headroom 96 + step staging), and the fp8 pair
|
||||||
|
# settled at 1,379 MiB free — so `vram_weights + vram_kv <= 29,120 MiB` keeps
|
||||||
|
# >= 1 GiB free. The pair below is exactly that sum.
|
||||||
|
# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
|
||||||
|
# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
|
||||||
|
# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
|
||||||
|
# so a bigger KV pool steals weight residency from the slab AND from the
|
||||||
|
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
|
||||||
|
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
|
||||||
|
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. Expect -3..-8%
|
||||||
|
# vs the fp8 control (136 t/s median at 44.5k depth, acceptance 0.7246,
|
||||||
|
# ~190 t/s on a live 512-token probe); re-tune from a boot ledger + probe,
|
||||||
|
# never from that estimate.
|
||||||
|
FRANK_ARCHON_VRAM_WEIGHTS_MIB=22208
|
||||||
|
FRANK_ARCHON_VRAM_KV_MIB=6912
|
||||||
|
# KV cache UNQUANTIZED (bf16). This engine's flag takes ONLY bf16|fp8 — there
|
||||||
|
# is ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
|
||||||
|
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
|
||||||
|
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
|
||||||
|
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
|
||||||
|
# 2026-10-08, bf16 boot 2026-10-11):
|
||||||
|
# kv_attn 13 layers, 4-token blocks, page 26.00 KiB (fp8) -> 52.00 KiB
|
||||||
|
# kv_qsa_bkey 13 layers, 4-token blocks, page 3.25 KiB at BOTH dtypes —
|
||||||
|
# the QSA block key is an index, not attention state, and does
|
||||||
|
# NOT follow --kv-cache-dtype
|
||||||
|
# => 7.3125 KiB/token fp8, 13.8125 KiB/token bf16: bf16 costs 1.89x fp8,
|
||||||
|
# not 2x, precisely because the bkey plane is dtype-fixed.
|
||||||
|
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
|
||||||
|
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
|
||||||
|
# checkpoint region 8 x 55.99 = 448 MiB. So:
|
||||||
|
# pool_mib >= tokens x 13.8125 KiB + 110 x seqs + 448
|
||||||
|
# 6912 -> (6912 - 220 - 448) x 1024 / 13.8125 = 462,946 tokens; the engine
|
||||||
|
# reported 462,752 (0.04% page rounding) — 2 x 200,000 with 62k of slack,
|
||||||
|
# and the ledger says it outright: "backs 2 sequences all at 200000 tokens
|
||||||
|
# at once, which is the worst case".
|
||||||
|
# (the fp8 pool of 3543 gave 386,904 — it never held 2 x 200k, it held 1.93
|
||||||
|
# sessions, and a third long conversation preempted to the host/disk tiers).
|
||||||
|
# Secondary cost: the prefix-cache entry grows with the attention plane
|
||||||
|
# (one entry 58.50 KiB at fp8 -> 110.50 KiB at bf16, measured on the two boot
|
||||||
|
# ledgers' `kv tier transfer` lines), so the host tier below doubles to keep
|
||||||
|
# the same number of parked conversations, out of RAM the host has
|
||||||
|
# (104 GiB available with the service loaded).
|
||||||
|
FRANK_ARCHON_KV_DTYPE=bf16
|
||||||
|
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
||||||
|
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
||||||
|
# flag states it so the ledger records the intent).
|
||||||
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
|
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
||||||
|
# The host tier is sized in ENTRIES, and one entry doubles with the cache
|
||||||
|
# dtype (58.50 KiB at fp8 -> 117 KiB at bf16, per the ledger's `kv tier
|
||||||
|
# transfer` line): 4096 MiB at bf16 parks ~1.4 sessions of 200k where it
|
||||||
|
# parked ~2.9 at fp8. Doubled to keep the capacity, at RAM the host has
|
||||||
|
# (104 GiB available with the service loaded).
|
||||||
|
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
||||||
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
|
# The PLE n-gram table (47.7 GiB) serves from pinned host RAM the cards
|
||||||
|
# read. `ram` REFUSES the start if the memlock limit is too small — a
|
||||||
|
# named refusal beats a mid-load allocator failure — and the unit's
|
||||||
|
# LimitMEMLOCK=infinity is the other half of the arrangement.
|
||||||
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||||
|
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
|
||||||
|
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
|
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
||||||
|
# takes the min over the knobs that exist). Below the plan the load dies
|
||||||
|
# mid-pin with an allocator error instead of this named refusal — raise the
|
||||||
|
# knob first (runtime sysfs write; kernel cmdline fallback + one reboot only
|
||||||
|
# if the file is read-only), see the runbook. Cap raised 2026-10-05 to
|
||||||
|
# 52,428,800 pages = 200 GiB, which still clears this floor.
|
||||||
|
FRANK_ARCHON_TTM_MIN_MIB=150000
|
||||||
|
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
||||||
|
# multi-flag values as one string; the serve script expands it unquoted on
|
||||||
|
# purpose.
|
||||||
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
|
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
||||||
|
# guaranteed long-timeout boot instead of a fast failure). The converted
|
||||||
|
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
||||||
|
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
||||||
|
# tail) - floor sits just under the payload.
|
||||||
|
FRANK_ARCHON_RAD_MIN_MIB=110000
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# frank-primary.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
||||||
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
||||||
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
||||||
# frank-primary-freetoken-pr587-20261005 is the pre-cutover state; the
|
# frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
|
||||||
# retired unit/env/serve trio and Dockerfile are in that tag).
|
# retired unit/env/serve trio and Dockerfile are in that tag).
|
||||||
#
|
#
|
||||||
# Same port, same served alias, same container identity as the unit name;
|
# Same port, same served alias, same container identity as the unit name;
|
||||||
@@ -19,7 +19,7 @@
|
|||||||
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
||||||
# engine still runs out of dank's rootless podman storage.
|
# engine still runs out of dank's rootless podman storage.
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=frank-primary — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
||||||
@@ -33,14 +33,14 @@ Group=dank
|
|||||||
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
|
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
|
||||||
# PID1 grants infinity here — parity with the fleet's other LLM units.
|
# PID1 grants infinity here — parity with the fleet's other LLM units.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
|
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
|
||||||
# systemd stop: wait alone cannot stop the container child; stop it
|
# systemd stop: wait alone cannot stop the container child; stop it
|
||||||
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
||||||
# holds no queue to drain — in-flight decodes are lost, which a stop
|
# holds no queue to drain — in-flight decodes are lost, which a stop
|
||||||
# during a bench window accepts; radiance takes SIGTERM on the engine
|
# during a bench window accepts; radiance takes SIGTERM on the engine
|
||||||
# process and podman reaps the container).
|
# process and podman reaps the container).
|
||||||
ExecStop=/usr/bin/podman stop -t 120 frank-primary
|
ExecStop=/usr/bin/podman stop -t 120 frank-archon
|
||||||
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
|
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
|
||||||
# publisher's compose healthcheck allows a 15-minute start_period; the
|
# publisher's compose healthcheck allows a 15-minute start_period; the
|
||||||
# vLLM-era budget (1800 s) carried forward.
|
# vLLM-era budget (1800 s) carried forward.
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
# frank-large.env — tunables for the frank-large endpoint on Dankclaw's
|
# frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
|
||||||
# Strix Halo iGPU (gfx1151), host port 7770.
|
# Strix Halo iGPU (gfx1151), host port 7770.
|
||||||
#
|
#
|
||||||
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
|
# RUNTIME (2026-10-06, Frank): halogen-flash-server 0.16.2 — the
|
||||||
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
|
# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
|
||||||
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
|
# digest-pinned below) serving the Quat3rnion ABLITERATED Swift1.5 finetune
|
||||||
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
|
# of peonist's Qwen3.8-Flash-Next v2 checkpoint, as a podman container under
|
||||||
# a root unit, driven by scripts/serve-frank-large.sh. This CUTS OVER the
|
# a root unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
|
||||||
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
|
# 2026-10-05 halogen 27B-dense shape (censored base, DFlash2, 31.7 t/s):
|
||||||
# rollback = revert the swap commit's trio; the 27B payload stays on disk at
|
# rollback = revert the swap commit's trio; the 27B payload stays on disk at
|
||||||
# /home/dank/models/halogen-qwen3.8-27b/.
|
# /home/dank/models/halogen-qwen3.8-27b/.
|
||||||
@@ -36,23 +36,24 @@
|
|||||||
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
||||||
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
||||||
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
||||||
# 10 % under-count). Compaction rides on frank-secondary: input ≈
|
# 10 % under-count). Compaction rides on frank-zealot: input ≈
|
||||||
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
||||||
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
||||||
# form of the engine's default, so the seam test can pin the product.
|
# form of the engine's default, so the seam test can pin the product.
|
||||||
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
||||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
# drafter is NOT slot-locked on this engine. A "slots > 1 disables the
|
||||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
# drafter" restriction is a property of the engine generation that
|
||||||
# quickstart ships 2 slots WITH the head), so the second slot is
|
# imposed it, not of speculation itself — batching and speculation
|
||||||
# concurrent capacity, not a speculation tax.
|
# coexist here, and the card's own quickstart ships 2 slots WITH the
|
||||||
|
# head — so the second slot is concurrent capacity, not a speculation tax.
|
||||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
||||||
# card's own quickstart arithmetic does not bill it):
|
# card's own quickstart arithmetic does not bill it):
|
||||||
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||||
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
||||||
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||||
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ)
|
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
|
||||||
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
||||||
# after deploy and re-size if the engine prints a clamp.
|
# after deploy and re-size if the engine prints a clamp.
|
||||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||||
@@ -88,50 +89,51 @@
|
|||||||
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
||||||
# amdgpu.noretry=0 (check at deploy).
|
# amdgpu.noretry=0 (check at deploy).
|
||||||
|
|
||||||
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
# Image: digest-pinned, not tag-pinned — a tag can be re-pointed to a
|
||||||
# is what was qualified). 0.16.2 = the version the model card names as its
|
# build that was never qualified here, while the digest pins exactly the
|
||||||
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
# build that was tested. 0.16.2 = the version the model card names as its
|
||||||
FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
# quickstart pin; the RepoDigest was captured at a pull of that tag.
|
||||||
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||||
# Container identity: kept from the 27B era (unit ExecStop names it).
|
# Container identity: kept from the 27B era (unit ExecStop names it).
|
||||||
FRANK_LARGE_CONTAINER=frank-large-halogen
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
||||||
# Old quadlet-era container identity; the serve script clears stale names.
|
# Old quadlet-era container identity; the serve script clears stale names.
|
||||||
FRANK_LARGE_LEGACY_CONTAINERS=frank-large
|
FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
|
||||||
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
|
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
|
||||||
# default); the engine port (8730) is never published.
|
# default); the engine port (8730) is never published.
|
||||||
FRANK_LARGE_PUBLISH=7770:8731
|
FRANK_DRAGOON_PUBLISH=7770:8731
|
||||||
|
|
||||||
FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
|
FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-flashnext-v2
|
||||||
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
|
# The checkpoint: Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
|
# qwen38-flash-next-v2-swift15-abliterated.hgn — compacted r2 of the card's
|
||||||
# append-only original (byte-identical live tensors, padding dropped:
|
# append-only original (byte-identical live tensors, padding dropped:
|
||||||
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
|
# 66,687,678,464 B vs 89,775,868,032 B). LFS sha256 verified at fetch.
|
||||||
FRANK_LARGE_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
|
FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-swift15-abliterated.hgn
|
||||||
FRANK_LARGE_CHECKPOINT_BYTES=66687678464
|
FRANK_DRAGOON_CHECKPOINT_BYTES=66687678464
|
||||||
FRANK_LARGE_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
|
FRANK_DRAGOON_CHECKPOINT_SHA256=9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037
|
||||||
# The n-gram lookup table: REQUIRED (the engine reads it beside the
|
# The n-gram lookup table: REQUIRED (the engine reads it beside the
|
||||||
# checkpoint; the card's quickstart names it explicitly). From
|
# checkpoint; the card's quickstart names it explicitly). From
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
|
# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
|
||||||
FRANK_LARGE_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
|
||||||
FRANK_LARGE_NGRAM_BYTES=51200246144
|
FRANK_DRAGOON_NGRAM_BYTES=51200246144
|
||||||
FRANK_LARGE_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
||||||
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
|
# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
|
||||||
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
|
# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
|
||||||
FRANK_LARGE_VISION_TOWER=qwen38-flash-next-vision.hgn
|
FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
|
||||||
FRANK_LARGE_VISION_BYTES=897916416
|
FRANK_DRAGOON_VISION_BYTES=897916416
|
||||||
FRANK_LARGE_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
||||||
# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer,
|
# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
|
||||||
# mounted whole at /tokenizer. The two repos ship the same one; fetched
|
# mounted whole at /tokenizer. The two repos ship the same one; fetched
|
||||||
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
|
# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
|
||||||
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
|
# the 27B-era copy; the rest are plain git blobs — size-checked only). The
|
||||||
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
|
# serve gate and the deployer refuse a missing file or ANY symlink: an HF
|
||||||
# cache snapshot symlinks each file into blobs/, and those links dangle
|
# cache snapshot symlinks each file into blobs/, and those links dangle
|
||||||
# inside the container mount.
|
# inside the container mount.
|
||||||
FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
|
||||||
|
|
||||||
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
||||||
# stays at the image default, which is what was qualified) ------------------
|
# stays at the image default, which is what was qualified) ------------------
|
||||||
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
||||||
# scripts and this file keep the frank-large name. OpenWebUI api_configs
|
# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
|
||||||
# model_ids for the :7770 backend MUST name this value.
|
# model_ids for the :7770 backend MUST name this value.
|
||||||
HALOGEN_MODEL_ID=frank-dragoon
|
HALOGEN_MODEL_ID=frank-dragoon
|
||||||
HALOGEN_CTX=262144
|
HALOGEN_CTX=262144
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
|
Description=halogen-flash-server container (Quat3rnion abliterated Swift1.5 of Qwen3.8-Flash-Next v2, MTP spec decode, 2 slots x 200192, vision ON), root systemd unit (dankclaw, frank-dragoon, Strix Halo iGPU)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
@@ -11,11 +11,11 @@ Group=dank
|
|||||||
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
|
||||||
# other LLM units on this fleet.
|
# other LLM units on this fleet.
|
||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
Environment="FRANK_DRAGOON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-dragoon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-dragoon.sh
|
||||||
# systemd stop alone cannot stop the container child; stop it explicitly
|
# systemd stop alone cannot stop the container child; stop it explicitly
|
||||||
# (frank-primary pattern). The engine drains on SIGTERM.
|
# (frank-archon pattern). The engine drains on SIGTERM.
|
||||||
ExecStop=/usr/bin/podman stop -t 60 frank-large-halogen
|
ExecStop=/usr/bin/podman stop -t 60 frank-dragoon-halogen
|
||||||
TimeoutStartSec=1200
|
TimeoutStartSec=1200
|
||||||
TimeoutStopSec=120
|
TimeoutStopSec=120
|
||||||
# always, not on-failure: an engine that exits 0 after an internal fault
|
# always, not on-failure: an engine that exits 0 after an internal fault
|
||||||
@@ -23,7 +23,7 @@ TimeoutStopSec=120
|
|||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
# The engine pins ~62 GiB of v2 weights and keeps a multi-GiB working set
|
||||||
# on a box that also hosts frank-secondary, ComfyUI and the Caddy/observability
|
# on a box that also hosts frank-zealot, ComfyUI and the Caddy/observability
|
||||||
# stack; make it a late OOM candidate, not the first.
|
# stack; make it a late OOM candidate, not the first.
|
||||||
OOMScoreAdjust=-500
|
OOMScoreAdjust=-500
|
||||||
|
|
||||||
@@ -1,168 +0,0 @@
|
|||||||
# frank-primary.env — radiance (StillDeadcode, Apache-2.0) serving the
|
|
||||||
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at NATIVE precision
|
|
||||||
# (modelopt block-FP8 throughout, no requantisation) on BOTH R9700s (TP=2),
|
|
||||||
# dankserver :7770.
|
|
||||||
#
|
|
||||||
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
|
||||||
# pinned host RAM; git tag frank-primary-freetoken-pr587-20261005 is the
|
|
||||||
# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
|
|
||||||
# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
|
|
||||||
# frank-primary-vllm-shapeG-20261003).
|
|
||||||
#
|
|
||||||
# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
|
|
||||||
# gfx1201, drives `podman run` directly from the digest-pinned
|
|
||||||
# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
|
|
||||||
# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
|
|
||||||
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
|
||||||
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
|
||||||
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
|
||||||
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
|
|
||||||
# --host-pool-mib 12288) remains the documented fallback if the native
|
|
||||||
# container proves unbootable.
|
|
||||||
#
|
|
||||||
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
|
|
||||||
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
|
|
||||||
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
|
|
||||||
# staging dir whose config.json drops `quantization_config`: the descriptor
|
|
||||||
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
|
|
||||||
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
|
|
||||||
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
|
||||||
# containers carry. Per-weight encodings are data, not the descriptor —
|
|
||||||
# removing the key changes nothing about what the weights hold. Built by
|
|
||||||
# scripts/build-frank-primary-rad.sh (staging + convert + rad-info).
|
|
||||||
#
|
|
||||||
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
|
||||||
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
|
||||||
# then engine.serve()) — during load the port refuses connections, so a
|
|
||||||
# bare curl /health gate is honest. /health == /ping: 200 JSON while
|
|
||||||
# serving, 503 only while stopping (core/server/admin.cpp:385).
|
|
||||||
# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
|
|
||||||
# `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
|
|
||||||
# the chat format. Both need no parser flags.
|
|
||||||
# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
|
|
||||||
# typical_p, presence_penalty, frequency_penalty, repetition_penalty,
|
|
||||||
# repeat_last_n, chat_template_kwargs, reasoning_effort,
|
|
||||||
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
|
||||||
# models.yml's extraBody set is fully inside it. Unknown params are
|
|
||||||
# refused (400), which is why nothing unverified rides the wire.
|
|
||||||
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
|
|
||||||
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
|
|
||||||
# publisher's throughput peak for this model. Stated explicitly with
|
|
||||||
# --num-speculative-tokens 3.
|
|
||||||
# - The fused TP all-reduce is declared over max-num-seqs x
|
|
||||||
# (1 + --num-speculative-tokens) rows; 3 x 3 = 9 rows fits either wire.
|
|
||||||
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
|
||||||
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
|
||||||
# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
|
|
||||||
# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
|
|
||||||
# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
|
|
||||||
# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
|
|
||||||
# Acceptance on the greedy harness is deterministic (the control
|
|
||||||
# reproduced the recorded 0.755 exactly), so the drop is a real numerics
|
|
||||||
# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
|
|
||||||
# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
|
|
||||||
# The vram-pair translation itself is exact (1.2.0 boot ledger took
|
|
||||||
# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
|
|
||||||
# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
|
|
||||||
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
|
|
||||||
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
|
|
||||||
# file already uses the pair, which BOTH pins accept (rollback symmetry).
|
|
||||||
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
|
||||||
# The .rad container FILE (mounted under /models) and the state dir for the
|
|
||||||
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
|
||||||
# .rad files; the raw HF download and its staging dir live beside it and
|
|
||||||
# are never mounted.
|
|
||||||
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad
|
|
||||||
FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
|
||||||
FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
|
|
||||||
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
|
||||||
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
|
||||||
# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
|
|
||||||
# cache must ride a host mount — serve script mounts it, see the runbook's
|
|
||||||
# "Tuning levers").
|
|
||||||
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
|
|
||||||
# name across systemctl and podman ps. The runtime changes; the name does
|
|
||||||
# not. The legacy list retires every predecessor identity that could hold
|
|
||||||
# the GPUs or the port.
|
|
||||||
FRANK_PRIMARY_CONTAINER=frank-primary
|
|
||||||
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
|
|
||||||
# Served model id — Protoss name (wire rename 2026-10-09). The container,
|
|
||||||
# unit, scripts and this file keep the frank-primary name above; only the
|
|
||||||
# alias on the wire moved. Radiance validates the requested model name, so
|
|
||||||
# every client (harness, bench scripts, probes) MUST send this value.
|
|
||||||
FRANK_PRIMARY_SERVED_NAME=frank-archon
|
|
||||||
FRANK_PRIMARY_PORT=7770
|
|
||||||
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
|
|
||||||
# repo's root chat_template.jinja, vendored here; the checkpoint's own
|
|
||||||
# template is baked into the .rad by rad-convert and this overrides it).
|
|
||||||
# Radiance derives the reasoning/tool-call reply format from it.
|
|
||||||
FRANK_PRIMARY_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
|
|
||||||
|
|
||||||
# --- engine shape (first-boot conservative; the boot ledger arbitrates) --
|
|
||||||
# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
|
|
||||||
# x8/x8 on root ports) made collectives legal on this host; radiance's TP
|
|
||||||
# wire is libr4d's own, not RCCL.
|
|
||||||
FRANK_PRIMARY_TP=2
|
|
||||||
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
|
|
||||||
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
|
||||||
# engine default.
|
|
||||||
FRANK_PRIMARY_TP_WIRE=wht6
|
|
||||||
# Decode slots. 2 is Frank's first-boot value ("use parallelism = 2 to
|
|
||||||
# start"); raise only after a real-traffic look at slot occupancy and the
|
|
||||||
# KV pool's headroom line. The recurrent-state cost scales with it (54
|
|
||||||
# MiB/copy/card, 2 copies with speculation).
|
|
||||||
FRANK_PRIMARY_MAX_SEQS=3
|
|
||||||
# Served window. Native is 262,144; 200,000 is the publisher's production
|
|
||||||
# value (compose) and what models.yml's contextWindow must quote — a longer
|
|
||||||
# prompt is refused by the engine, never truncated.
|
|
||||||
FRANK_PRIMARY_MAX_MODEL_LEN=200000
|
|
||||||
FRANK_PRIMARY_PLACEMENT=expert_tiered
|
|
||||||
# Pinned host pool for the expert rows that do not fit VRAM. Native FP8
|
|
||||||
# experts are 112.8 GiB total (56.4 GiB per rank slice) vs the 4-bit
|
|
||||||
# publisher shape's ~29 GiB — this pool is the cost of native precision.
|
|
||||||
# The boot ledger prints the pool's coverage ("the pool backs N% of the
|
|
||||||
# worst case ... M full-length sessions"); resize from that line, not from
|
|
||||||
# arithmetic.
|
|
||||||
FRANK_PRIMARY_HOST_POOL_MIB=40960
|
|
||||||
# VRAM left unclaimed per card (publisher default 96 MiB).
|
|
||||||
FRANK_PRIMARY_GPU_HEADROOM_MIB=96
|
|
||||||
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
|
||||||
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
|
|
||||||
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
|
|
||||||
# vram_weights 25.46 GiB, vram_kv 3.46 GiB per card -> 26071 / 3543.
|
|
||||||
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
|
|
||||||
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
|
|
||||||
# boot ledger arbitrates; re-derive after any payload change.
|
|
||||||
FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071
|
|
||||||
FRANK_PRIMARY_VRAM_KV_MIB=3543
|
|
||||||
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
|
|
||||||
# 2 KV heads, 256 head_dim, K+V).
|
|
||||||
FRANK_PRIMARY_KV_DTYPE=fp8
|
|
||||||
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
|
||||||
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
|
||||||
# flag states it so the ledger records the intent).
|
|
||||||
FRANK_PRIMARY_SPEC_TOKENS=2
|
|
||||||
FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048
|
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
|
||||||
FRANK_PRIMARY_PREFIX_HOST_MIB=4096
|
|
||||||
FRANK_PRIMARY_PREFIX_DISK_MIB=65536
|
|
||||||
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
|
||||||
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
|
|
||||||
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
|
|
||||||
FRANK_PRIMARY_NGRAM_PLACEMENT=ram
|
|
||||||
# TTM preflight: the pinned ledger (2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
|
||||||
# prefix host) is ~132 GiB. ttm's pages_limit/pages_max cap the GPU-mappable
|
|
||||||
# pinned pages; the current cap (32,252,917 pages = ~123 GiB) is BELOW the
|
|
||||||
# plan, so the serve script refuses to start until both are raised. This is
|
|
||||||
# the floor it enforces (in MiB), with ~8 GiB of margin.
|
|
||||||
FRANK_PRIMARY_TTM_MIN_MIB=140000
|
|
||||||
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
|
||||||
# multi-flag values as one string; the serve script expands it unquoted on
|
|
||||||
# purpose.
|
|
||||||
FRANK_PRIMARY_EXTRA_ARGS=
|
|
||||||
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
|
||||||
# guaranteed long-timeout boot instead of a fast failure). The converted
|
|
||||||
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
|
||||||
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
|
||||||
# tail) - floor sits just under the payload.
|
|
||||||
FRANK_PRIMARY_RAD_MIN_MIB=110000
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
|
|
||||||
After=network-online.target
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=simple
|
|
||||||
User=dank
|
|
||||||
Group=dank
|
|
||||||
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
|
||||||
# though the weights themselves are copied into VRAM.
|
|
||||||
LimitMEMLOCK=infinity
|
|
||||||
Environment="FRANK_SECONDARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-secondary.env"
|
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-secondary.sh
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=10
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
# frank-secondary — llama.cpp (master, CUDA sm_86) serving
|
# frank-zealot — llama.cpp (master, CUDA sm_86) serving
|
||||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
||||||
# :7771.
|
# :7771.
|
||||||
#
|
#
|
||||||
@@ -16,11 +16,11 @@
|
|||||||
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
||||||
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
||||||
#
|
#
|
||||||
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
|
# The KV rate follows from the architecture: 8 full layers × 4 KV heads ×
|
||||||
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
|
# 256 dim = 8,192 elements per token per cache. f16 K + f16 V = 32,768
|
||||||
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
|
# B/token = 32 KiB/token, so the 401,408-token pool is 12.25 GiB. The 24
|
||||||
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
|
# DeltaNet layers cost a fixed recurrent state per SLOT (not per token) —
|
||||||
# where the 26B MoE fit only one.
|
# that is why two 200k slots fit where the 26B MoE fit only one.
|
||||||
#
|
#
|
||||||
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
||||||
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
||||||
@@ -38,7 +38,7 @@ MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensore
|
|||||||
# Served model id — the harness/family name (Protoss rename completed to the
|
# Served model id — the harness/family name (Protoss rename completed to the
|
||||||
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
||||||
# this day). The SERVICE, unit, scripts and this env file keep the
|
# this day). The SERVICE, unit, scripts and this env file keep the
|
||||||
# frank-secondary name; only the alias on the wire changed. Consumers that
|
# frank-zealot name; only the alias on the wire changed. Consumers that
|
||||||
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
||||||
# model_ids) and the harness models.yml.
|
# model_ids) and the harness models.yml.
|
||||||
ALIAS=frank-zealot
|
ALIAS=frank-zealot
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=dank
|
||||||
|
Group=dank
|
||||||
|
# CUDA pins its staging buffers, so the process needs the memlock budget even
|
||||||
|
# though the weights themselves are copied into VRAM.
|
||||||
|
LimitMEMLOCK=infinity
|
||||||
|
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
|
||||||
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
|
||||||
|
# Restart=always is the right policy for a serving daemon: a process that
|
||||||
|
# exits on its own — exit 0 — is a CLEAN stop to systemd, so on-failure
|
||||||
|
# never fires and the endpoint stays dark (with the unit reporting
|
||||||
|
# success) until the next boot. frank-zealot is the fleet's compaction
|
||||||
|
# hub — every endpoint's compaction dies when it does.
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -24,7 +24,7 @@ KLD_OUT=/home/dank/radiance-state/kld-gate.json
|
|||||||
: > "$KLD_OUT"
|
: > "$KLD_OUT"
|
||||||
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
||||||
/home/dank/models/tpl.jinja 2>/dev/null || true
|
/home/dank/models/tpl.jinja 2>/dev/null || true
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
podman run --rm --name "$C" \
|
podman run --rm --name "$C" \
|
||||||
|
|||||||
Executable
+133
@@ -0,0 +1,133 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# llm-gates.sh — the artifact gates every LLM endpoint runs before it starts.
|
||||||
|
#
|
||||||
|
# Why this file exists: every LLM endpoint gates its payload twice — the
|
||||||
|
# serve script at start, the deployer before dispatch. Two hand-written
|
||||||
|
# copies of the same checks drift apart in wording and strictness, and the
|
||||||
|
# drift surfaces as deploy-time disagreement: one side passes, the other
|
||||||
|
# refuses, and the failure lands in production instead of in the gate.
|
||||||
|
# Both sides source this library, so the refusal is one text and one rule.
|
||||||
|
#
|
||||||
|
# Contract (public interface):
|
||||||
|
# require_exec PATH LABEL HINT
|
||||||
|
# require_file PATH LABEL HINT
|
||||||
|
# require_bytes PATH LABEL WANT_BYTES HINT
|
||||||
|
# require_min_mib PATH LABEL MIN_MIB HINT
|
||||||
|
# require_dir_files DIR LABEL HINT FILE...
|
||||||
|
# require_image REF HINT
|
||||||
|
#
|
||||||
|
# Each returns 0 and prints nothing on success. On failure it prints two
|
||||||
|
# lines to stderr — the first naming the endpoint ($GATE_PREFIX) and the
|
||||||
|
# artifact, the second the remedy — and returns 1. It NEVER exits: the serve
|
||||||
|
# scripts run under `set -e` (so a refusal is terminal), while the deployer
|
||||||
|
# wants `|| return 1` so a failed gate reports a red deploy for that one
|
||||||
|
# service and the dispatcher's state marker stays pending.
|
||||||
|
# Lines are labelled by ARTIFACT CLASS, and the class is the whole label
|
||||||
|
# (the path rides in the same line): the class + path is what an operator
|
||||||
|
# acts on, so no wording order and no payload rename can make the two twins
|
||||||
|
# disagree. The deployer's old "halogen vision tower" and the serve script's
|
||||||
|
# "vision-tower" were one gate in two vocabularies that no grep could join.
|
||||||
|
|
||||||
|
# Size checks use equality, not a floor, for `require_bytes`: a fetch that
|
||||||
|
# truncates looks like a success (an HTTP error exit is often swallowed, and
|
||||||
|
# the error body is written in the artifact's place), so "exists and is big
|
||||||
|
# enough" is not proof of the right artifact. The declared byte count is a
|
||||||
|
# contract in both directions — a bigger file is a DIFFERENT artifact.
|
||||||
|
|
||||||
|
# GATE_PREFIX names the endpoint in every refusal (e.g. frank-dragoon). The
|
||||||
|
# caller exports it; shellcheck cannot follow it across the source boundary,
|
||||||
|
# so the read site carries the directive.
|
||||||
|
# shellcheck disable=SC2034
|
||||||
|
: "${GATE_PREFIX:=gate}"
|
||||||
|
|
||||||
|
# Two bash behaviors make the `return 1` at every call site load-bearing:
|
||||||
|
# * a function whose body ends inside an `if` returns the CONDITION's
|
||||||
|
# status (true = 0), so a gate that ends on its own refusal prints the
|
||||||
|
# refusal and reports SUCCESS;
|
||||||
|
# * `set -e` is suspended for any command in a `… || …` chain, so a gate
|
||||||
|
# that runs inside one must not be trusted to abort the caller.
|
||||||
|
# Every gate's refusal therefore carries an explicit `|| return 1`, and the
|
||||||
|
# caller consumes the status (`|| exit 1`, `|| return 1`).
|
||||||
|
_gate_fail() {
|
||||||
|
printf '%s: %s\n' "$GATE_PREFIX" "$1" >&2
|
||||||
|
printf ' hint: %s\n' "$2" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_file — PATH must be a regular file. A directory (an unpacked
|
||||||
|
# staging dir wearing the payload's name) and a symlink (a HuggingFace cache
|
||||||
|
# snapshot's entry, which dangles inside the container mount) both refuse.
|
||||||
|
require_file() {
|
||||||
|
local path="$1" label="$2" hint="$3"
|
||||||
|
if [[ -L "$path" || ! -f "$path" ]]; then
|
||||||
|
_gate_fail "$label: missing at $path" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_exec — PATH must be an executable regular file. The native runtimes
|
||||||
|
# (llama.cpp's llama-server) are binaries the operator builds; a tree that
|
||||||
|
# compiled but lost its +x, or a directory wearing the binary's name, must
|
||||||
|
# refuse the same way a missing one does.
|
||||||
|
require_exec() {
|
||||||
|
local path="$1" label="$2" hint="$3"
|
||||||
|
if [[ -L "$path" || ! -x "$path" ]]; then
|
||||||
|
_gate_fail "$label: missing at $path" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_bytes — PATH must exist AND be exactly WANT_BYTES bytes.
|
||||||
|
require_bytes() {
|
||||||
|
local path="$1" label="$2" want="$3" hint="$4" got
|
||||||
|
require_file "$path" "$label" "$hint" || return 1
|
||||||
|
got="$(stat -c %s "$path")"
|
||||||
|
if [[ "$got" != "$want" ]]; then
|
||||||
|
_gate_fail "$label: is $got bytes, expected $want" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_min_mib — PATH must be at least MIN_MIB mebibytes. The floor form
|
||||||
|
# is for build products (a .rad container is a range of legal sizes; a
|
||||||
|
# truncated convert is not).
|
||||||
|
require_min_mib() {
|
||||||
|
local path="$1" label="$2" min="$3" hint="$4" mib
|
||||||
|
require_file "$path" "$label" "$hint" || return 1
|
||||||
|
mib=$(( $(stat -c %s "$path") / 1048576 ))
|
||||||
|
if (( mib < min )); then
|
||||||
|
_gate_fail "$label: is $mib MiB, expected >= $min MiB" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_dir_files — DIR must be a directory and every named member a
|
||||||
|
# regular file, never a symlink: snapshot layouts link files into place, and
|
||||||
|
# a link dangles inside the container mount even when the directory itself
|
||||||
|
# is real. A symlinked DIRECTORY is refused too — the same layout can link
|
||||||
|
# the directory itself, and the link dangles identically.
|
||||||
|
require_dir_files() {
|
||||||
|
local dir="$1" label="$2" hint="$3"
|
||||||
|
shift 3
|
||||||
|
local f
|
||||||
|
if [[ -L "$dir" || ! -d "$dir" ]]; then
|
||||||
|
_gate_fail "$label: missing at $dir" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
for f in "$@"; do
|
||||||
|
if [[ -L "$dir/$f" || ! -f "$dir/$f" ]]; then
|
||||||
|
_gate_fail "$label: $dir/$f is missing or a symlink" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_image — the image ref must be in podman storage. A name@sha256 ref
|
||||||
|
# exists only if THAT manifest was pulled, so the gate is the ref itself, in
|
||||||
|
# one command: no pipefail/grep race (the 2026-09-30 lesson).
|
||||||
|
require_image() {
|
||||||
|
local ref="$1" hint="$2"
|
||||||
|
if ! podman image exists "$ref"; then
|
||||||
|
_gate_fail "image: $ref is not in podman storage" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -44,17 +44,17 @@ JSONL_EXTS = (".jsonl", ".json")
|
|||||||
DOC_DIR = "serverAppConfigs-docs"
|
DOC_DIR = "serverAppConfigs-docs"
|
||||||
|
|
||||||
FLEET_CMDS = [
|
FLEET_CMDS = [
|
||||||
"systemctl status frank-primary --no-pager",
|
"systemctl status frank-archon --no-pager",
|
||||||
"journalctl -u frank-primary -n 100 --no-pager",
|
"journalctl -u frank-archon -n 100 --no-pager",
|
||||||
"podman ps --format '{{.Names}} {{.Status}}'",
|
"podman ps --format '{{.Names}} {{.Status}}'",
|
||||||
"podman logs --tail 50 frank-primary",
|
"podman logs --tail 50 frank-archon",
|
||||||
"curl -s http://192.168.0.235:7770/health",
|
"curl -s http://192.168.0.235:7770/health",
|
||||||
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
|
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
|
||||||
"df -h /home/dank | tail -1",
|
"df -h /home/dank | tail -1",
|
||||||
"free -g",
|
"free -g",
|
||||||
"sudo systemctl restart frank-primary",
|
"sudo systemctl restart frank-archon",
|
||||||
"cd /home/dank/serverAppConfigs && git pull --ff-only",
|
"cd /home/dank/serverAppConfigs && git pull --ff-only",
|
||||||
"./scripts/deploy.sh dankserver --service frank-primary",
|
"./scripts/deploy.sh dankserver --service frank-archon",
|
||||||
"./tests/run.sh 2>&1 | tail -5",
|
"./tests/run.sh 2>&1 | tail -5",
|
||||||
"jq . /home/dank/models/rad/index.json | head -40",
|
"jq . /home/dank/models/rad/index.json | head -40",
|
||||||
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
|
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
#
|
#
|
||||||
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
|
# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
|
||||||
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
|
# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
|
||||||
# the deployed frank-primary shape. Nothing in scripts/ mounts these files.
|
# the deployed frank-archon shape. Nothing in scripts/ mounts these files.
|
||||||
#
|
#
|
||||||
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
|
# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
|
||||||
#
|
#
|
||||||
|
|||||||
@@ -29,10 +29,11 @@
|
|||||||
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
||||||
# getattr_static()s each one off the instance, breaking on the first miss.
|
# getattr_static()s each one off the instance, breaking on the first miss.
|
||||||
# So BOTH wrappers need all three attributes.
|
# So BOTH wrappers need all three attributes.
|
||||||
# * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with
|
# * Why not inheritance: adding SupportsEagle3 as a BASE class dies at
|
||||||
# "Cannot create a consistent method resolution order (MRO) for bases ..."
|
# class creation with "Cannot create a consistent method resolution
|
||||||
# (GLM's wrappers already carry four/five Protocol-derived bases). The
|
# order (MRO) for bases ..." (GLM's wrappers already carry four/five
|
||||||
# attribute route avoids the MRO entirely. Also proven by simulation:
|
# Protocol-derived bases; the base ordering cannot take another). The
|
||||||
|
# attribute route avoids the MRO entirely; simulation confirms that
|
||||||
# the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
|
# the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
|
||||||
# position.
|
# position.
|
||||||
# * Method placement: without inheriting, the Protocol's default
|
# * Method placement: without inheriting, the Protocol's default
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
|
# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
|
||||||
#
|
#
|
||||||
# STATUS: NOT part of the deployed frank-primary shape. It exists because the
|
# STATUS: NOT part of the deployed frank-archon shape. It exists because the
|
||||||
# fork's plan under-models this model's KV, and it is the enabler for a future
|
# fork's plan under-models this model's KV, and it is the enabler for a future
|
||||||
# MTP attempt (the production GLM shape runs without MTP and boots fine on the
|
# MTP attempt (the production GLM shape runs without MTP and boots fine on the
|
||||||
# plan's own reserve).
|
# plan's own reserve).
|
||||||
|
|||||||
Executable
+149
@@ -0,0 +1,149 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""probe-frank-archon-kv.py — verify the 2 x 200,000 KV contract on a LIVE
|
||||||
|
frank-archon (radiance) service. Run it ON dankserver: it talks to
|
||||||
|
127.0.0.1:7770 and builds its prompts from the repo checkout there.
|
||||||
|
No scratch container, no mask: the service under test is the service.
|
||||||
|
|
||||||
|
ssh dankserver 'python3 /home/dank/serverAppConfigs/scripts/probe-frank-archon-kv.py'
|
||||||
|
|
||||||
|
Three probes, in increasing cost:
|
||||||
|
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
||||||
|
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
||||||
|
Method: >=128 generated tokens (below that, first-token latency
|
||||||
|
dominates the decode rate), greedy (sampling noise is unattributable),
|
||||||
|
real text (synthetic text inflates draft acceptance).
|
||||||
|
B the capacity contract — TWO concurrent chats of ~197k prompt tokens
|
||||||
|
(~394k tokens resident) each carrying a unique marker at its head. Both
|
||||||
|
must answer their own marker. This is what a 2 x 200,000 endpoint owes:
|
||||||
|
two full-height conversations resident at once, exact retrieval in both.
|
||||||
|
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
|
||||||
|
to give the blocks back (cache_n > 0, TTFT collapses).
|
||||||
|
|
||||||
|
PROBE PITFALL: radiance routes the thinking block to
|
||||||
|
`reasoning_content`, so a chat request whose budget is spent inside
|
||||||
|
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
|
||||||
|
reads `content` alone scores 0/2 on a healthy server. Every request here
|
||||||
|
sets chat_template_kwargs.enable_thinking=false and reads both fields.
|
||||||
|
|
||||||
|
A/B method for a shape change (no scratch container): write a variant env file
|
||||||
|
on the host, point a transient systemd drop-in at it
|
||||||
|
(`Environment=FRANK_ARCHON_ENV_FILE=...` — drop-ins are read AFTER the unit's
|
||||||
|
own Environment= lines, so they win), restart, probe, delete the drop-in,
|
||||||
|
daemon-reload, restart. `systemctl mask` is not available: the unit file in
|
||||||
|
/etc/systemd/system is a relink symlink into the checkout.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import glob
|
||||||
|
import time
|
||||||
|
import urllib.request
|
||||||
|
import concurrent.futures
|
||||||
|
|
||||||
|
BASE = "http://127.0.0.1:7770"
|
||||||
|
MODEL = "frank-archon"
|
||||||
|
REPO = "/home/dank/serverAppConfigs"
|
||||||
|
GLOBS = [f"{REPO}/scripts/*.sh", f"{REPO}/serverAppConfigs-docs/**/*.md"]
|
||||||
|
|
||||||
|
|
||||||
|
def post(payload, timeout=1800):
|
||||||
|
req = urllib.request.Request(
|
||||||
|
BASE + "/v1/chat/completions",
|
||||||
|
data=json.dumps(payload).encode(),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
)
|
||||||
|
t0 = time.time()
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as r:
|
||||||
|
body = json.loads(r.read().decode())
|
||||||
|
hdr = dict(r.headers.items())
|
||||||
|
return body, time.time() - t0, hdr
|
||||||
|
|
||||||
|
|
||||||
|
def get(path, timeout=60):
|
||||||
|
with urllib.request.urlopen(BASE + path, timeout=timeout) as r:
|
||||||
|
return json.loads(r.read().decode())
|
||||||
|
|
||||||
|
|
||||||
|
def corpus(min_chars):
|
||||||
|
files = []
|
||||||
|
for g in GLOBS:
|
||||||
|
files.extend(sorted(glob.glob(g, recursive=True)))
|
||||||
|
out, total = [], 0
|
||||||
|
for f in files:
|
||||||
|
try:
|
||||||
|
t = open(f, encoding="utf-8", errors="replace").read()
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
out.append(t)
|
||||||
|
total += len(t)
|
||||||
|
if total > min_chars:
|
||||||
|
break
|
||||||
|
return "".join(text for text in out)
|
||||||
|
|
||||||
|
|
||||||
|
def prompt(text, chars, salt, fact):
|
||||||
|
"""Real repo text, salted so the prefix cache cannot answer, marker planted
|
||||||
|
near the head, question at the tail."""
|
||||||
|
head = f"\n[SALT {salt}] confidential marker: {fact}\n"
|
||||||
|
body = text + text
|
||||||
|
p = (head + body * max(1, chars // len(body)))[:chars]
|
||||||
|
return p + "\n\nQuestion: what is the confidential marker in the " \
|
||||||
|
"SALT block near the top of this document? Answer with the marker " \
|
||||||
|
"alone, no other words.\n"
|
||||||
|
|
||||||
|
|
||||||
|
def chat(text, chars, salt, fact, max_tokens):
|
||||||
|
return {"model": MODEL,
|
||||||
|
"messages": [{"role": "user",
|
||||||
|
"content": prompt(text, chars, salt, fact)}],
|
||||||
|
"max_tokens": max_tokens, "temperature": 0.0,
|
||||||
|
"chat_template_kwargs": {"enable_thinking": False}}
|
||||||
|
|
||||||
|
|
||||||
|
def show(tag, j, wall, expect=None):
|
||||||
|
tim, u = j.get("timings") or {}, j.get("usage", {})
|
||||||
|
ch = (j.get("choices") or [{}])[0]
|
||||||
|
msg = ch.get("message") or {}
|
||||||
|
txt = (msg.get("content") or msg.get("reasoning_content") or "").strip()
|
||||||
|
acc = (tim["draft_n_accepted"] / tim["draft_n"]) if tim.get("draft_n") else None
|
||||||
|
line = (f"[{tag}] prompt={u.get('prompt_tokens')} gen={u.get('completion_tokens')} "
|
||||||
|
f"wall={wall:.1f}s prefill={tim.get('prompt_per_second', 0):.0f} t/s "
|
||||||
|
f"decode={tim.get('predicted_per_second', 0):.1f} t/s "
|
||||||
|
f"draft={tim.get('draft_n')}/{tim.get('draft_n_accepted')} "
|
||||||
|
f"accept={acc if acc is None else round(acc, 4)} "
|
||||||
|
f"cache_n={tim.get('cache_n')} finish={ch.get('finish_reason')}")
|
||||||
|
if expect:
|
||||||
|
hit = expect.split("-")[0] in txt.lower()
|
||||||
|
line += f" expected={expect} answered={txt[:80]!r} HIT={hit}"
|
||||||
|
print(line)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
print("health:", json.dumps(get("/health")))
|
||||||
|
print("models:", [m.get("id") for m in get("/v1/models").get("data", [])])
|
||||||
|
deep = corpus(160_000)
|
||||||
|
wide = corpus(1_100_000)
|
||||||
|
|
||||||
|
facts = {"B1": "cobalt-harbor-13", "B2": "quartz-meadow-42"}
|
||||||
|
bodies = {s: chat(wide, 646_000, s, f, 24) for s, f in facts.items()}
|
||||||
|
print(f"[B] {len(bodies['B1']['messages'][0]['content']):,} chars per "
|
||||||
|
f"request, x2 concurrent")
|
||||||
|
|
||||||
|
# A + C first (cheap), B last (the expensive pair).
|
||||||
|
for tag, salt in (("A-cold", "A1-9137"), ):
|
||||||
|
j, wall, _ = post(chat(deep, 140_000, salt, "zephyr-lantern-7", 512))
|
||||||
|
show(tag, j, wall, "zephyr-lantern-7")
|
||||||
|
t0 = time.time()
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as ex:
|
||||||
|
futs = {s: ex.submit(post, b) for s, b in bodies.items()}
|
||||||
|
for s, fut in futs.items():
|
||||||
|
try:
|
||||||
|
j, wall, _ = fut.result()
|
||||||
|
show(f"B-{s}", j, wall, facts[s])
|
||||||
|
except Exception as e: # a preemption/timeout lands here
|
||||||
|
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
||||||
|
print(f"[B] total {time.time() - t0:.1f}s")
|
||||||
|
j, wall, _ = post(chat(deep, 140_000, "A1-9137", "zephyr-lantern-7", 512))
|
||||||
|
show("C-prefix-hit", j, wall, "zephyr-lantern-7")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,10 +1,12 @@
|
|||||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
||||||
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
||||||
# base) as a radiance container for frank-primary: EVERY routed expert -- gate_up AND down --
|
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
||||||
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
# through GPTQ into the w4nl64a8h table. The converter declares ONE
|
||||||
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
# encoding per expert table (qwen4exp_fp8 rad_arch_declare): the served
|
||||||
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
# kernel covers the whole expert set, and the only legal mixed layout is a
|
||||||
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
# WHOLE expert kept plain bf16 beside it. That rules out the original
|
||||||
|
# split tier (down block-fp8 + gate_up w4), which asks one table for two
|
||||||
|
# encodings.
|
||||||
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
||||||
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
||||||
#
|
#
|
||||||
@@ -25,11 +27,14 @@
|
|||||||
|
|
||||||
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
||||||
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
||||||
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
# The Swift 1.5 finetune's routing distribution is not assumed equal: the
|
||||||
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
# derivation step rewrites this section from Swift's OWN calibration
|
||||||
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
# recording before this recipe is ever run. The selection rule is the base
|
||||||
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
# checkpoint's: an expert whose share of its layer's down-input energy is
|
||||||
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
# at least 3% AND at least three times its share of the routing; an even
|
||||||
|
# count on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||||
|
# as two equal tables by parity), padded with that half's next most
|
||||||
|
# energetic expert.
|
||||||
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
||||||
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
||||||
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
||||||
|
|||||||
@@ -30,9 +30,9 @@ KLD_REF_DIR=/home/dank/corpora/kld-ref
|
|||||||
CALIB_DIR=/home/dank/corpora/calib
|
CALIB_DIR=/home/dank/corpora/calib
|
||||||
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
|
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
|
||||||
|
|
||||||
restore() { sudo systemctl start frank-primary >/dev/null 2>&1 || true; }
|
restore() { sudo systemctl start frank-archon >/dev/null 2>&1 || true; }
|
||||||
trap restore EXIT
|
trap restore EXIT
|
||||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
sudo systemctl stop frank-archon >/dev/null 2>&1 || true
|
||||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
||||||
|
|||||||
Executable
+205
@@ -0,0 +1,205 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
||||||
|
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
||||||
|
# NATIVE block-FP8 precision from a rad-convert container
|
||||||
|
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
||||||
|
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
||||||
|
# one card; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
|
# pre-cutover state).
|
||||||
|
#
|
||||||
|
# The run line below IS the engine argv: the image's entrypoint is the
|
||||||
|
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
||||||
|
# there is no per-boot VRAM planner to second-guess: the boot ledger in the
|
||||||
|
# log states every budget before anything is allocated (weights vs KV
|
||||||
|
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||||
|
# the P2P answer per pair) — read it after every resize.
|
||||||
|
# Deployed shape (2026-10-11): 2 decode slots x 200,000 tokens, bf16
|
||||||
|
# (unquantized) KV in a 6,912 MiB pool, the weights slab cut to 22,208 MiB to
|
||||||
|
# pay for it, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
|
||||||
|
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
|
||||||
|
#
|
||||||
|
# The root unit (scripts/frank-archon.service, User=dank,
|
||||||
|
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
||||||
|
# the 47.7 GiB PLE table + 8 GiB prefix host tier — ~147 GiB of pinned
|
||||||
|
# pages, which is ALSO why this script refuses to start when ttm's
|
||||||
|
# pages_limit/pages_max cap is under FRANK_ARCHON_TTM_MIN_MIB (see below).
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
|
# The artifact gates are the shared library: the serve script and the
|
||||||
|
# deployer gate the same payload, and two hand-written copies of the same
|
||||||
|
# checks drift apart until one side passes what the other refuses. One
|
||||||
|
# library, one refusal text, one rule.
|
||||||
|
export GATE_PREFIX=frank-archon
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
|
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
|
||||||
|
|
||||||
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
|
echo "ERROR: frank-archon env file absent: $ENV_FILE" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
source "$ENV_FILE"
|
||||||
|
set +a
|
||||||
|
|
||||||
|
: "${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
|
||||||
|
: "${FRANK_ARCHON_MODEL_DIR:?FRANK_ARCHON_MODEL_DIR must name the .rad dir}"
|
||||||
|
: "${FRANK_ARCHON_MODEL_FILE:?FRANK_ARCHON_MODEL_FILE must name the .rad file}"
|
||||||
|
: "${FRANK_ARCHON_STATE_DIR:?FRANK_ARCHON_STATE_DIR must name the /data state dir}"
|
||||||
|
: "${FRANK_ARCHON_CONTAINER:?FRANK_ARCHON_CONTAINER must name the container}"
|
||||||
|
: "${FRANK_ARCHON_SERVED_NAME:?FRANK_ARCHON_SERVED_NAME must name the alias}"
|
||||||
|
: "${FRANK_ARCHON_PORT:?FRANK_ARCHON_PORT must be the port}"
|
||||||
|
: "${FRANK_ARCHON_TEMPLATE:?FRANK_ARCHON_TEMPLATE must name the chat template}"
|
||||||
|
: "${FRANK_ARCHON_TP:?FRANK_ARCHON_TP must be set}"
|
||||||
|
: "${FRANK_ARCHON_TP_WIRE:?FRANK_ARCHON_TP_WIRE must be set}"
|
||||||
|
: "${FRANK_ARCHON_MAX_SEQS:?FRANK_ARCHON_MAX_SEQS must be set}"
|
||||||
|
: "${FRANK_ARCHON_MAX_MODEL_LEN:?FRANK_ARCHON_MAX_MODEL_LEN must be set}"
|
||||||
|
: "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
|
||||||
|
: "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
|
||||||
|
: "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
|
||||||
|
: "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
||||||
|
: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
||||||
|
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
||||||
|
|
||||||
|
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
||||||
|
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
||||||
|
# newer latest does not satisfy it. One command, no pipefail/grep race (the
|
||||||
|
# 2026-09-30 lesson). Runs as dank: the unit's process uses dank's ROOTLESS
|
||||||
|
# podman storage (root unit, User=dank — the fleet's frank-archon shape).
|
||||||
|
require_image "$FRANK_ARCHON_IMAGE" \
|
||||||
|
"podman pull docker://stilldeadcode/radiance:latest (as dank)" || exit 1
|
||||||
|
|
||||||
|
# The payload gate: the .rad container file, sized. A truncated convert
|
||||||
|
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
||||||
|
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
||||||
|
# (the convert script built the file; its rad-info output is the record).
|
||||||
|
require_min_mib "$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
|
container "$FRANK_ARCHON_RAD_MIN_MIB" \
|
||||||
|
"build it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" || exit 1
|
||||||
|
|
||||||
|
require_file "$FRANK_ARCHON_TEMPLATE" template \
|
||||||
|
"the chat template is vendored: scripts/qwen38-froggeric-v22.5.jinja" || exit 1
|
||||||
|
|
||||||
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
|
||||||
|
# The engine's create is NOT recursive: with only the state dir mounted, its
|
||||||
|
# "cannot create /data/kvcache/frank-archon: No such file or directory"
|
||||||
|
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
|
||||||
|
# the full path on the host so the tier opens on first boot.
|
||||||
|
|
||||||
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
||||||
|
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
||||||
|
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
||||||
|
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
||||||
|
# the cache must ride a host mount or it dies with the --rm container
|
||||||
|
# that wrote it. The dir is pre-created for the same reason as kvcache
|
||||||
|
# (the engine's create is not recursive).
|
||||||
|
|
||||||
|
# GPU access is mandatory; the node paths are knobs so the seam can drive
|
||||||
|
# this gate on a box without AMD GPUs (production defaults).
|
||||||
|
DEV_KFD="${FRANK_ARCHON_DEV_KFD:-/dev/kfd}"
|
||||||
|
DEV_DRI="${FRANK_ARCHON_DEV_DRI:-/dev/dri}"
|
||||||
|
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
||||||
|
echo "ERROR: GPU device nodes absent ($DEV_KFD, $DEV_DRI)" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
||||||
|
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
|
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
||||||
|
# takes the min over the knobs that exist). Below FRANK_ARCHON_TTM_MIN_MIB
|
||||||
|
# the load dies mid-pin with an allocator error instead of this named
|
||||||
|
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
||||||
|
# fallback + one reboot only if the file is read-only), see the runbook.
|
||||||
|
# RAISED 2026-10-05 pre-cutover to 52,428,800 pages = 200 GiB.
|
||||||
|
ttm_cap_mib() {
|
||||||
|
# min over the knobs THIS kernel exposes — the current kernel has
|
||||||
|
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
||||||
|
# not a zero (a zero would refuse a healthy cap).
|
||||||
|
local v cur="" dir
|
||||||
|
dir="${FRANK_ARCHON_TTM_SYSFS:-/sys/module/ttm/parameters}"
|
||||||
|
for p in "$dir/pages_limit" "$dir/pages_max"; do
|
||||||
|
[ -r "$p" ] || continue
|
||||||
|
v="$(cat "$p" 2>/dev/null)" || continue
|
||||||
|
[ -n "$v" ] || continue
|
||||||
|
v=$(( v * 4 / 1024 ))
|
||||||
|
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
||||||
|
done
|
||||||
|
echo "${cur:-0}"
|
||||||
|
}
|
||||||
|
TTM_CAP="$(ttm_cap_mib)"
|
||||||
|
if [ "$TTM_CAP" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
|
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
|
echo "frank-archon: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_ARCHON_TTM_MIN_MIB in 4 KiB pages" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# A crashed/killed unit can leave the container name registered (systemd
|
||||||
|
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
||||||
|
# identities (the FreeToken-era frank-archon-qwen, the retired R9V
|
||||||
|
# container and the ad hoc experiment containers all hold this GPU); all
|
||||||
|
# idempotent.
|
||||||
|
for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
|
||||||
|
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
|
||||||
|
LOG="${FRANK_ARCHON_LOG:-/home/dank/logs/frank_archon.log}"
|
||||||
|
mkdir -p "$(dirname "$LOG")"
|
||||||
|
|
||||||
|
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
||||||
|
# --network host (the server binds the host's addresses directly, no NAT),
|
||||||
|
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
|
||||||
|
# (Docker/podman's default profile refuses io_uring, which the n-gram
|
||||||
|
# table's reads go through, and the NUMA binding calls the ROCm runtime
|
||||||
|
# makes for host memory), memlock unbounded (the pinned ledger — legal
|
||||||
|
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
|
||||||
|
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
||||||
|
# The template rides a single-file mount; the .rad dir is ro; /data holds
|
||||||
|
# the prefix-cache disk tier.
|
||||||
|
podman run -d --name "$FRANK_ARCHON_CONTAINER" \
|
||||||
|
--network host \
|
||||||
|
--init \
|
||||||
|
--security-opt seccomp=unconfined \
|
||||||
|
--ulimit memlock=-1:-1 \
|
||||||
|
--device /dev/kfd --device /dev/dri \
|
||||||
|
-v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
|
||||||
|
-v "$FRANK_ARCHON_STATE_DIR":/data \
|
||||||
|
-v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
||||||
|
-v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
|
||||||
|
"$FRANK_ARCHON_IMAGE" \
|
||||||
|
--model "/models/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
|
--host :: --port "$FRANK_ARCHON_PORT" \
|
||||||
|
--served-model-name "$FRANK_ARCHON_SERVED_NAME" \
|
||||||
|
--override-chat-template /templates/template.jinja \
|
||||||
|
--tp "$FRANK_ARCHON_TP" --tp-wire "$FRANK_ARCHON_TP_WIRE" \
|
||||||
|
--max-num-seqs "$FRANK_ARCHON_MAX_SEQS" \
|
||||||
|
--max-model-len "$FRANK_ARCHON_MAX_MODEL_LEN" \
|
||||||
|
--placement "$FRANK_ARCHON_PLACEMENT" \
|
||||||
|
--host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
|
||||||
|
--gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
|
||||||
|
--vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
|
||||||
|
--vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
|
||||||
|
--kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
|
||||||
|
--prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
|
||||||
|
--prefix-cache-dir /data/kvcache/frank-archon \
|
||||||
|
--prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
|
||||||
|
--ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
|
||||||
|
--num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
|
||||||
|
--max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
|
||||||
|
${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
|
||||||
|
> "$LOG" 2>&1
|
||||||
|
|
||||||
|
# From here on, this script's lifetime IS the container's: wait returns the
|
||||||
|
# container's exit code, so a crash is a nonzero ExecStart and the unit's
|
||||||
|
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
|
||||||
|
# podman wait alone cannot stop the container child.)
|
||||||
|
exec podman wait "$FRANK_ARCHON_CONTAINER"
|
||||||
@@ -1,13 +1,13 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# serve-frank-large.sh — frank-large endpoint: halogen-flash-server 0.16.2
|
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
|
||||||
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
|
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
|
||||||
# digest-pinned in frank-large.env) serving the Quat3rnion abliterated
|
# digest-pinned in frank-dragoon.env) serving the Quat3rnion abliterated
|
||||||
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
|
# Swift1.5 finetune of peonist's Qwen3.8-Flash-Next v2 checkpoint on
|
||||||
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
|
# Dankclaw's Strix Halo iGPU, host :7770. 2026-10-06 cutover from the
|
||||||
# halogen 27B-dense shape — revert the swap commit's trio to roll back
|
# halogen 27B-dense shape — revert the swap commit's trio to roll back
|
||||||
# (the 27B payload stays on disk).
|
# (the 27B payload stays on disk).
|
||||||
#
|
#
|
||||||
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-primary
|
# The engine is driven by `podman run` DIRECTLY (the fleet's frank-archon
|
||||||
# pattern): the image starts engine + OpenAI frontend in one container.
|
# pattern): the image starts engine + OpenAI frontend in one container.
|
||||||
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
|
# The frontend listens on 8731 inside (HALOGEN_API_PORT default) and is
|
||||||
# published as host 7770; the engine's own token port (8730, no auth)
|
# published as host 7770; the engine's own token port (8730, no auth)
|
||||||
@@ -26,13 +26,20 @@
|
|||||||
# Why containers instead of a bare binary: the payload is .hgn and only
|
# Why containers instead of a bare binary: the payload is .hgn and only
|
||||||
# the flash-server line reads it; it ships as a container; the root unit
|
# the flash-server line reads it; it ships as a container; the root unit
|
||||||
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
|
# keeps LimitMEMLOCK=infinity for parity and to outlive the login session,
|
||||||
# exactly like frank-primary's.
|
# exactly like frank-archon's.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
# The artifact gates are the shared library — the serve script and the
|
||||||
|
# deployer refuse the same missing/truncated payload with the same words
|
||||||
|
# (the 2026-10-01 crash-loop was the two hand-written twins disagreeing).
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
|
export GATE_PREFIX=frank-dragoon
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
|
|
||||||
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
echo "frank-large: env file not found: $ENV_FILE" >&2
|
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
set -a
|
set -a
|
||||||
@@ -40,81 +47,65 @@ set -a
|
|||||||
source "$ENV_FILE"
|
source "$ENV_FILE"
|
||||||
set +a
|
set +a
|
||||||
|
|
||||||
: "${FRANK_LARGE_IMAGE:?FRANK_LARGE_IMAGE must name the digest-pinned image}"
|
: "${FRANK_DRAGOON_IMAGE:?FRANK_DRAGOON_IMAGE must name the digest-pinned image}"
|
||||||
: "${FRANK_LARGE_CONTAINER:?FRANK_LARGE_CONTAINER must name the container}"
|
: "${FRANK_DRAGOON_CONTAINER:?FRANK_DRAGOON_CONTAINER must name the container}"
|
||||||
: "${FRANK_LARGE_PUBLISH:?FRANK_LARGE_PUBLISH must be host:container}"
|
: "${FRANK_DRAGOON_PUBLISH:?FRANK_DRAGOON_PUBLISH must be host:container}"
|
||||||
: "${FRANK_LARGE_MODEL_DIR:?FRANK_LARGE_MODEL_DIR must name the payload dir}"
|
: "${FRANK_DRAGOON_MODEL_DIR:?FRANK_DRAGOON_MODEL_DIR must name the payload dir}"
|
||||||
: "${FRANK_LARGE_CHECKPOINT:?FRANK_LARGE_CHECKPOINT must name the .hgn inside it}"
|
: "${FRANK_DRAGOON_CHECKPOINT:?FRANK_DRAGOON_CHECKPOINT must name the .hgn inside it}"
|
||||||
: "${FRANK_LARGE_CHECKPOINT_BYTES:?FRANK_LARGE_CHECKPOINT_BYTES must pin the size}"
|
: "${FRANK_DRAGOON_CHECKPOINT_BYTES:?FRANK_DRAGOON_CHECKPOINT_BYTES must pin the size}"
|
||||||
: "${FRANK_LARGE_NGRAM_TABLE:?FRANK_LARGE_NGRAM_TABLE must name the lookup table}"
|
: "${FRANK_DRAGOON_NGRAM_TABLE:?FRANK_DRAGOON_NGRAM_TABLE must name the lookup table}"
|
||||||
: "${FRANK_LARGE_NGRAM_BYTES:?FRANK_LARGE_NGRAM_BYTES must pin the table size}"
|
: "${FRANK_DRAGOON_NGRAM_BYTES:?FRANK_DRAGOON_NGRAM_BYTES must pin the table size}"
|
||||||
: "${FRANK_LARGE_VISION_TOWER:?FRANK_LARGE_VISION_TOWER must name the tower}"
|
: "${FRANK_DRAGOON_VISION_TOWER:?FRANK_DRAGOON_VISION_TOWER must name the tower}"
|
||||||
: "${FRANK_LARGE_VISION_BYTES:?FRANK_LARGE_VISION_BYTES must pin the tower size}"
|
: "${FRANK_DRAGOON_VISION_BYTES:?FRANK_DRAGOON_VISION_BYTES must pin the tower size}"
|
||||||
: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}"
|
: "${FRANK_DRAGOON_TOKENIZER_FILES:?FRANK_DRAGOON_TOKENIZER_FILES must list the tokenizer dir contents}"
|
||||||
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
|
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
|
||||||
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
|
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
|
||||||
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
|
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
|
||||||
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
|
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
|
||||||
: "${HALOGEN_SPEC_ADAPT:?HALOGEN_SPEC_ADAPT must be set (0 = head always on)}"
|
: "${HALOGEN_SPEC_ADAPT:?HALOGEN_SPEC_ADAPT must be set (0 = head always on)}"
|
||||||
|
|
||||||
# Capture podman's output BEFORE grepping: under `set -o pipefail` the
|
# The image gate is the digest-pinned ref itself: `podman image exists` takes
|
||||||
# `podman images | grep -q` pattern races (grep exits on first match,
|
# a name@sha256 ref in one command, so a retagged look-alike or a newer
|
||||||
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
|
# latest cannot satisfy it and there is no pipefail/SIGPIPE race with grep
|
||||||
# 2026-09-30 deploy refusal).
|
# (the 2026-09-30 deploy refusal). Runs as dank: the unit's process uses
|
||||||
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
|
# dank's rootless storage.
|
||||||
if ! grep -qF "${FRANK_LARGE_IMAGE#*@}" <<<"$DIGESTS"; then
|
require_image "$FRANK_DRAGOON_IMAGE" \
|
||||||
echo "frank-large: image $FRANK_LARGE_IMAGE not in podman storage;" >&2
|
"pull it: podman pull docker://$FRANK_DRAGOON_IMAGE" || exit 1
|
||||||
echo " pull it: podman pull ${FRANK_LARGE_IMAGE%@*}" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Size-exact gates on all three payload files: a truncated curl exits 0 on
|
# Size-exact gates on all three payload files: a truncated download can
|
||||||
# some failures and a partial .hgn loads far enough to look alive, so the
|
# exit 0, and a partial .hgn loads far enough to look alive, so existence
|
||||||
# byte counts from the Hub tree API are the contract (fleet fetch law).
|
# proves nothing. The byte count published by the Hub tree API is the
|
||||||
ck() {
|
# contract: any other size is a different (broken) artifact.
|
||||||
# ck <env-name> <path> <expected-bytes>
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" \
|
||||||
if [ ! -f "$2" ]; then
|
checkpoint "$FRANK_DRAGOON_CHECKPOINT_BYTES" \
|
||||||
echo "frank-large: $1 $2 missing;" >&2
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" \
|
||||||
exit 1
|
n-gram "$FRANK_DRAGOON_NGRAM_BYTES" \
|
||||||
fi
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
local sz
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" \
|
||||||
sz="$(stat -c %s "$2")"
|
vision "$FRANK_DRAGOON_VISION_BYTES" \
|
||||||
if [ "$sz" != "$3" ]; then
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
echo "frank-large: $1 is $sz bytes, expected $3 (partial download?)" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
ck checkpoint "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_CHECKPOINT" "$FRANK_LARGE_CHECKPOINT_BYTES"
|
|
||||||
ck n-gram-table "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_NGRAM_TABLE" "$FRANK_LARGE_NGRAM_BYTES"
|
|
||||||
ck vision-tower "$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_VISION_TOWER" "$FRANK_LARGE_VISION_BYTES"
|
|
||||||
|
|
||||||
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
||||||
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
||||||
# dangle inside the container mount — so the directory must be real and
|
# dangle inside the container mount — so the directory must be real and
|
||||||
# every named file must be a regular file, not a link.
|
# every named file must be a regular file, not a link.
|
||||||
TOK_DIR="$FRANK_LARGE_MODEL_DIR/tokenizer"
|
# The tokenizer file list is word-separated ON PURPOSE (one argument per
|
||||||
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
|
# file), so the unquoted expansion below is the interface, not a slip.
|
||||||
echo "frank-large: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
|
# shellcheck disable=SC2086
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
require_dir_files "$FRANK_DRAGOON_MODEL_DIR/tokenizer" tokenizer \
|
||||||
exit 1
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
|
||||||
fi
|
$FRANK_DRAGOON_TOKENIZER_FILES || exit 1
|
||||||
for TF in $FRANK_LARGE_TOKENIZER_FILES; do
|
|
||||||
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
|
|
||||||
echo "frank-large: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
|
|
||||||
echo " fetch it: bash scripts/download-frank-large-flashnext.sh" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
# GPU access is mandatory but the node paths are variables so the seam can
|
# GPU access is mandatory but the node paths are variables so the seam can
|
||||||
# drive the gate on a box without an AMD GPU.
|
# drive the gate on a box without an AMD GPU.
|
||||||
DEV_KFD="${FRANK_LARGE_DEV_KFD:-/dev/kfd}"
|
DEV_KFD="${FRANK_DRAGOON_DEV_KFD:-/dev/kfd}"
|
||||||
DEV_DRI="${FRANK_LARGE_DEV_DRI:-/dev/dri}"
|
DEV_DRI="${FRANK_DRAGOON_DEV_DRI:-/dev/dri}"
|
||||||
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
||||||
echo "frank-large: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
||||||
# The engine refuses to pin when MemAvailable is below the checkpoint plus
|
# The engine refuses to pin when MemAvailable is below the checkpoint plus
|
||||||
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
|
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
|
||||||
@@ -125,15 +116,15 @@ fi
|
|||||||
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
|
# self-resolved in ~2 min when the dying 27B engine freed RAM. Check here and
|
||||||
# WAIT for the RAM to clear, so a transient dip boots once instead of
|
# WAIT for the RAM to clear, so a transient dip boots once instead of
|
||||||
# crash-looping; if it never clears, refuse loudly.
|
# crash-looping; if it never clears, refuse loudly.
|
||||||
MEMINFO="${FRANK_LARGE_MEMINFO:-/proc/meminfo}"
|
MEMINFO="${FRANK_DRAGOON_MEMINFO:-/proc/meminfo}"
|
||||||
PIN_KB=$(( FRANK_LARGE_CHECKPOINT_BYTES / 1024 ))
|
PIN_KB=$(( FRANK_DRAGOON_CHECKPOINT_BYTES / 1024 ))
|
||||||
NEED_KB=$(( PIN_KB + 16 * 1048576 ))
|
NEED_KB=$(( PIN_KB + 16 * 1048576 ))
|
||||||
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
|
ram_avail_kb() { awk '/^MemAvailable:/{print $2; exit}' "$MEMINFO"; }
|
||||||
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
|
MEMAVAIL_KB="$(ram_avail_kb)" || MEMAVAIL_KB=""
|
||||||
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
||||||
WAIT_S="${FRANK_LARGE_RAM_WAIT:-600}"
|
WAIT_S="${FRANK_DRAGOON_RAM_WAIT:-600}"
|
||||||
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
|
GIB() { awk -v kb="$1" 'BEGIN{printf "%.2f", kb/1048576}'; }
|
||||||
echo "frank-large: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
|
echo "frank-dragoon: host RAM below the pin gate — MemAvailable $(GIB "$MEMAVAIL_KB") GiB < needed $(GIB "$NEED_KB") GiB ($(GIB "$PIN_KB") GiB checkpoint + 16.00 GiB floor); waiting up to ${WAIT_S}s for host RAM to clear" >&2
|
||||||
deadline=$(( $(date +%s) + WAIT_S ))
|
deadline=$(( $(date +%s) + WAIT_S ))
|
||||||
while [ "$(date +%s)" -lt "$deadline" ]; do
|
while [ "$(date +%s)" -lt "$deadline" ]; do
|
||||||
sleep 5
|
sleep 5
|
||||||
@@ -143,16 +134,16 @@ if [ -n "$MEMAVAIL_KB" ] && [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
if [ -z "$MEMAVAIL_KB" ] || [ "$MEMAVAIL_KB" -lt "$NEED_KB" ]; then
|
||||||
echo "frank-large: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
|
echo "frank-dragoon: MemAvailable $(GIB "${MEMAVAIL_KB:-0}") GiB after ${WAIT_S}s; refusing to start (the engine would refuse the pin)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "frank-large: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
|
echo "frank-dragoon: MemAvailable cleared to $(GIB "$MEMAVAIL_KB") GiB; starting" >&2
|
||||||
fi
|
fi
|
||||||
for CTR in ${FRANK_LARGE_LEGACY_CONTAINERS:-} "$FRANK_LARGE_CONTAINER"; do
|
for CTR in ${FRANK_DRAGOON_LEGACY_CONTAINERS:-} "$FRANK_DRAGOON_CONTAINER"; do
|
||||||
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
|
|
||||||
LOG="${FRANK_LARGE_LOG:-/home/dank/logs/frank_large.log}"
|
LOG="${FRANK_DRAGOON_LOG:-/home/dank/logs/frank_dragoon.log}"
|
||||||
mkdir -p "$(dirname "$LOG")"
|
mkdir -p "$(dirname "$LOG")"
|
||||||
# --ipc=host for the udev-granted supplementary group, seccomp unconfined
|
# --ipc=host for the udev-granted supplementary group, seccomp unconfined
|
||||||
# for the ROCm ioctl surface, and unbounded memlock (the unit's
|
# for the ROCm ioctl surface, and unbounded memlock (the unit's
|
||||||
@@ -163,17 +154,17 @@ mkdir -p "$(dirname "$LOG")"
|
|||||||
# and both sibling serve scripts pass no nofile either. No comment lines
|
# and both sibling serve scripts pass no nofile either. No comment lines
|
||||||
# inside the backslash chain — a mid-chain # terminates the continuation
|
# inside the backslash chain — a mid-chain # terminates the continuation
|
||||||
# (the 2026-10-05 exit-127 crash loop on the sibling serve script).
|
# (the 2026-10-05 exit-127 crash loop on the sibling serve script).
|
||||||
podman run --name "$FRANK_LARGE_CONTAINER" \
|
podman run --name "$FRANK_DRAGOON_CONTAINER" \
|
||||||
--publish "$FRANK_LARGE_PUBLISH" \
|
--publish "$FRANK_DRAGOON_PUBLISH" \
|
||||||
--ipc=host \
|
--ipc=host \
|
||||||
--ulimit memlock=-1:-1 \
|
--ulimit memlock=-1:-1 \
|
||||||
--device "$DEV_KFD" --device "$DEV_DRI" \
|
--device "$DEV_KFD" --device "$DEV_DRI" \
|
||||||
--group-add keep-groups \
|
--group-add keep-groups \
|
||||||
--security-opt seccomp=unconfined \
|
--security-opt seccomp=unconfined \
|
||||||
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
|
-e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
|
||||||
-e HALOGEN_CHECKPOINT="/models/$FRANK_LARGE_CHECKPOINT" \
|
-e HALOGEN_CHECKPOINT="/models/$FRANK_DRAGOON_CHECKPOINT" \
|
||||||
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_LARGE_NGRAM_TABLE" \
|
-e HALOGEN_NGRAM_TABLE="/models/$FRANK_DRAGOON_NGRAM_TABLE" \
|
||||||
-e HALOGEN_VISION_TOWER="/models/$FRANK_LARGE_VISION_TOWER" \
|
-e HALOGEN_VISION_TOWER="/models/$FRANK_DRAGOON_VISION_TOWER" \
|
||||||
-e HALOGEN_TOKENIZER=/tokenizer \
|
-e HALOGEN_TOKENIZER=/tokenizer \
|
||||||
-e HALOGEN_CTX="$HALOGEN_CTX" \
|
-e HALOGEN_CTX="$HALOGEN_CTX" \
|
||||||
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
|
-e HALOGEN_KV_POOL_POSITIONS="$HALOGEN_KV_POOL_POSITIONS" \
|
||||||
@@ -187,9 +178,9 @@ podman run --name "$FRANK_LARGE_CONTAINER" \
|
|||||||
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
|
-e HALOGEN_TOP_P="${HALOGEN_TOP_P:-0.95}" \
|
||||||
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
|
-e HALOGEN_TOP_K="${HALOGEN_TOP_K:-20}" \
|
||||||
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
|
${HALOGEN_WEIGHTS_LOCK:+-e HALOGEN_WEIGHTS_LOCK="$HALOGEN_WEIGHTS_LOCK"} \
|
||||||
-v "$FRANK_LARGE_MODEL_DIR:/models:ro" \
|
-v "$FRANK_DRAGOON_MODEL_DIR:/models:ro" \
|
||||||
-v "$FRANK_LARGE_MODEL_DIR/tokenizer:/tokenizer:ro" \
|
-v "$FRANK_DRAGOON_MODEL_DIR/tokenizer:/tokenizer:ro" \
|
||||||
"$FRANK_LARGE_IMAGE" \
|
"$FRANK_DRAGOON_IMAGE" \
|
||||||
> "$LOG" 2>&1
|
> "$LOG" 2>&1
|
||||||
|
|
||||||
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's
|
# FOREGROUND `podman run`: the engine's stdout/stderr are this script's
|
||||||
@@ -1,206 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# serve-frank-primary.sh — frank-primary endpoint: radiance (StillDeadcode,
|
|
||||||
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
|
||||||
# NATIVE block-FP8 precision from a rad-convert container
|
|
||||||
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
|
||||||
# one card; git tag frank-primary-freetoken-pr587-20261005 is the
|
|
||||||
# pre-cutover state).
|
|
||||||
#
|
|
||||||
# The run line below IS the engine argv: the image's entrypoint is the
|
|
||||||
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
|
||||||
# there is no per-boot VRAM planner to second-guess: the boot ledger in the
|
|
||||||
# log states every budget before anything is allocated (weights vs KV
|
|
||||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
|
||||||
# the P2P answer per pair) — read it after every resize. First-boot shape:
|
|
||||||
# 2 decode slots (Frank's call), 40 GiB/rank pinned expert pool, fp8 KV,
|
|
||||||
# MTP depth 3, prefix cache host 4 GiB + disk 64 GiB, PLE pinned in RAM.
|
|
||||||
#
|
|
||||||
# The root unit (scripts/frank-primary.service, User=dank,
|
|
||||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 40 GiB expert pools +
|
|
||||||
# the 47.7 GiB PLE table + 4 GiB prefix host tier — ~132 GiB of pinned
|
|
||||||
# pages, which is ALSO why this script refuses to start when ttm's
|
|
||||||
# pages_limit/pages_max cap is under FRANK_PRIMARY_TTM_MIN_MIB (see below).
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}"
|
|
||||||
|
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
|
||||||
echo "ERROR: frank-primary env file absent: $ENV_FILE" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
set -a
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
source "$ENV_FILE"
|
|
||||||
set +a
|
|
||||||
|
|
||||||
: "${FRANK_PRIMARY_IMAGE:?FRANK_PRIMARY_IMAGE must name the radiance image}"
|
|
||||||
: "${FRANK_PRIMARY_MODEL_DIR:?FRANK_PRIMARY_MODEL_DIR must name the .rad dir}"
|
|
||||||
: "${FRANK_PRIMARY_MODEL_FILE:?FRANK_PRIMARY_MODEL_FILE must name the .rad file}"
|
|
||||||
: "${FRANK_PRIMARY_STATE_DIR:?FRANK_PRIMARY_STATE_DIR must name the /data state dir}"
|
|
||||||
: "${FRANK_PRIMARY_CONTAINER:?FRANK_PRIMARY_CONTAINER must name the container}"
|
|
||||||
: "${FRANK_PRIMARY_SERVED_NAME:?FRANK_PRIMARY_SERVED_NAME must name the alias}"
|
|
||||||
: "${FRANK_PRIMARY_PORT:?FRANK_PRIMARY_PORT must be the port}"
|
|
||||||
: "${FRANK_PRIMARY_TEMPLATE:?FRANK_PRIMARY_TEMPLATE must name the chat template}"
|
|
||||||
: "${FRANK_PRIMARY_TP:?FRANK_PRIMARY_TP must be set}"
|
|
||||||
: "${FRANK_PRIMARY_TP_WIRE:?FRANK_PRIMARY_TP_WIRE must be set}"
|
|
||||||
: "${FRANK_PRIMARY_MAX_SEQS:?FRANK_PRIMARY_MAX_SEQS must be set}"
|
|
||||||
: "${FRANK_PRIMARY_MAX_MODEL_LEN:?FRANK_PRIMARY_MAX_MODEL_LEN must be set}"
|
|
||||||
: "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}"
|
|
||||||
: "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_VRAM_WEIGHTS_MIB:?FRANK_PRIMARY_VRAM_WEIGHTS_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_VRAM_KV_MIB:?FRANK_PRIMARY_VRAM_KV_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}"
|
|
||||||
: "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}"
|
|
||||||
: "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}"
|
|
||||||
: "${FRANK_PRIMARY_PREFIX_HOST_MIB:?FRANK_PRIMARY_PREFIX_HOST_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_PREFIX_DISK_MIB:?FRANK_PRIMARY_PREFIX_DISK_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_NGRAM_PLACEMENT:?FRANK_PRIMARY_NGRAM_PLACEMENT must be set}"
|
|
||||||
: "${FRANK_PRIMARY_TTM_MIN_MIB:?FRANK_PRIMARY_TTM_MIN_MIB must be set}"
|
|
||||||
: "${FRANK_PRIMARY_RAD_MIN_MIB:?FRANK_PRIMARY_RAD_MIN_MIB must be set}"
|
|
||||||
|
|
||||||
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
|
||||||
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
|
||||||
# newer latest does not satisfy it. `podman image exists` takes a
|
|
||||||
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
|
|
||||||
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
|
|
||||||
# storage (root unit, User=dank — the fleet's frank-primary shape).
|
|
||||||
if ! podman image exists "$FRANK_PRIMARY_IMAGE"; then
|
|
||||||
echo "ERROR: radiance image absent: $FRANK_PRIMARY_IMAGE" >&2
|
|
||||||
echo "frank-primary: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# The payload gate: the .rad container file, sized. A truncated convert
|
|
||||||
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
|
||||||
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
|
||||||
# (scripts/build-frank-primary-rad.sh built the file; its rad-info output
|
|
||||||
# is the record).
|
|
||||||
RAD="$FRANK_PRIMARY_MODEL_DIR/$FRANK_PRIMARY_MODEL_FILE"
|
|
||||||
if [ ! -f "$RAD" ]; then
|
|
||||||
echo "ERROR: .rad container missing: $RAD" >&2
|
|
||||||
echo "frank-primary: run scripts/build-frank-primary-rad.sh" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
|
|
||||||
if [ "$RAD_MIB" -lt "$FRANK_PRIMARY_RAD_MIN_MIB" ]; then
|
|
||||||
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_PRIMARY_RAD_MIN_MIB" >&2
|
|
||||||
echo "frank-primary: the convert was truncated — rebuild it" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ! -f "$FRANK_PRIMARY_TEMPLATE" ]; then
|
|
||||||
echo "ERROR: chat template missing: $FRANK_PRIMARY_TEMPLATE" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/kvcache/frank-primary"
|
|
||||||
# The engine's create is NOT recursive: with only the state dir mounted, its
|
|
||||||
# "cannot create /data/kvcache/frank-primary: No such file or directory"
|
|
||||||
# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
|
|
||||||
# the full path on the host so the tier opens on first boot.
|
|
||||||
|
|
||||||
mkdir -p "$FRANK_PRIMARY_STATE_DIR/tune"
|
|
||||||
# rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
|
|
||||||
# the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
|
|
||||||
# $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
|
|
||||||
# the cache must ride a host mount or it dies with the --rm container
|
|
||||||
# that wrote it. The dir is pre-created for the same reason as kvcache
|
|
||||||
# (the engine's create is not recursive).
|
|
||||||
|
|
||||||
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
|
||||||
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
|
||||||
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
|
||||||
# takes the min over the knobs that exist). Below FRANK_PRIMARY_TTM_MIN_MIB
|
|
||||||
# the load dies mid-pin with an allocator error instead of this named
|
|
||||||
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
|
||||||
# fallback + one reboot only if the file is read-only), see the runbook.
|
|
||||||
# RAISED 2026-10-05 pre-cutover to 52,428,800 pages = 200 GiB.
|
|
||||||
ttm_cap_mib() {
|
|
||||||
# min over the knobs THIS kernel exposes — the current kernel has
|
|
||||||
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
|
||||||
# not a zero (a zero would refuse a healthy cap).
|
|
||||||
local v cur=""
|
|
||||||
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
|
|
||||||
[ -r "$p" ] || continue
|
|
||||||
v="$(cat "$p" 2>/dev/null)" || continue
|
|
||||||
[ -n "$v" ] || continue
|
|
||||||
v=$(( v * 4 / 1024 ))
|
|
||||||
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
|
||||||
done
|
|
||||||
echo "${cur:-0}"
|
|
||||||
}
|
|
||||||
TTM_CAP="$(ttm_cap_mib)"
|
|
||||||
if [ "$TTM_CAP" -lt "$FRANK_PRIMARY_TTM_MIN_MIB" ]; then
|
|
||||||
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_PRIMARY_TTM_MIN_MIB" >&2
|
|
||||||
echo "frank-primary: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_PRIMARY_TTM_MIN_MIB in 4 KiB pages" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# A crashed/killed unit can leave the container name registered (systemd
|
|
||||||
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
|
||||||
# identities (the FreeToken-era frank-primary-qwen, the retired R9V
|
|
||||||
# container and the ad hoc experiment containers all hold this GPU); all
|
|
||||||
# idempotent.
|
|
||||||
for CTR in $FRANK_PRIMARY_LEGACY_CONTAINERS "$FRANK_PRIMARY_CONTAINER"; do
|
|
||||||
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
|
||||||
done
|
|
||||||
|
|
||||||
LOG=/home/dank/logs/frank_primary.log
|
|
||||||
mkdir -p "$(dirname "$LOG")"
|
|
||||||
|
|
||||||
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
|
||||||
# --network host (the server binds the host's addresses directly, no NAT),
|
|
||||||
# --init (the engine releases the cards on SIGTERM), seccomp=unconfined
|
|
||||||
# (Docker/podman's default profile refuses io_uring, which the n-gram
|
|
||||||
# table's reads go through, and the NUMA binding calls the ROCm runtime
|
|
||||||
# makes for host memory), memlock unbounded (the pinned ledger — legal
|
|
||||||
# because the root unit grants LimitMEMLOCK=infinity to this dank process),
|
|
||||||
# /dev/kfd + /dev/dri (render/video groups are moot for a root container).
|
|
||||||
# The template rides a single-file mount; the .rad dir is ro; /data holds
|
|
||||||
# the prefix-cache disk tier.
|
|
||||||
podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
|
||||||
--network host \
|
|
||||||
--init \
|
|
||||||
--security-opt seccomp=unconfined \
|
|
||||||
--ulimit memlock=-1:-1 \
|
|
||||||
--device /dev/kfd --device /dev/dri \
|
|
||||||
-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
|
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR":/data \
|
|
||||||
-v "$FRANK_PRIMARY_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
|
|
||||||
-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro \
|
|
||||||
"$FRANK_PRIMARY_IMAGE" \
|
|
||||||
--model "/models/$FRANK_PRIMARY_MODEL_FILE" \
|
|
||||||
--host :: --port "$FRANK_PRIMARY_PORT" \
|
|
||||||
--served-model-name "$FRANK_PRIMARY_SERVED_NAME" \
|
|
||||||
--override-chat-template /templates/template.jinja \
|
|
||||||
--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE" \
|
|
||||||
--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS" \
|
|
||||||
--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN" \
|
|
||||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
|
||||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
|
||||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
|
||||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
|
||||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
|
||||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
|
||||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
|
||||||
--prefix-cache-dir /data/kvcache/frank-primary \
|
|
||||||
--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB" \
|
|
||||||
--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT" \
|
|
||||||
--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS" \
|
|
||||||
--max-num-batched-tokens "$FRANK_PRIMARY_MAX_BATCHED_TOKENS" \
|
|
||||||
${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS} \
|
|
||||||
> "$LOG" 2>&1
|
|
||||||
|
|
||||||
# From here on, this script's lifetime IS the container's: wait returns the
|
|
||||||
# container's exit code, so a crash is a nonzero ExecStart and the unit's
|
|
||||||
# Restart=always is real. (systemd stop goes through ExecStop in the unit —
|
|
||||||
# podman wait alone cannot stop the container child.)
|
|
||||||
exec podman wait "$FRANK_PRIMARY_CONTAINER"
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
||||||
# frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
# frank-zealot on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
||||||
#
|
#
|
||||||
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
||||||
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
||||||
@@ -24,11 +24,19 @@
|
|||||||
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
||||||
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides.
|
# Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
ENV_FILE="${FRANK_SECONDARY_ENV_FILE:-$SCRIPT_DIR/frank-secondary.env}"
|
|
||||||
|
# The artifact gates are the shared library (scripts/lib/llm-gates.sh): the
|
||||||
|
# serve script and the deployer gate the same payload, and two hand-written
|
||||||
|
# copies of the same checks drift apart until one side passes what the
|
||||||
|
# other refuses.
|
||||||
|
export GATE_PREFIX=frank-zealot
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
|
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
# shellcheck disable=SC1090
|
# shellcheck disable=SC1090
|
||||||
@@ -62,9 +70,9 @@ fi
|
|||||||
# FP8 class; no true fp8 KV dtype exists in this build).
|
# FP8 class; no true fp8 KV dtype exists in this build).
|
||||||
: "${CACHE_TYPE_K:=f16}"
|
: "${CACHE_TYPE_K:=f16}"
|
||||||
: "${CACHE_TYPE_V:=f16}"
|
: "${CACHE_TYPE_V:=f16}"
|
||||||
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
|
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors,
|
||||||
# there is no in-file head (the law: MTP is a property of the file, not of the
|
# so there is no in-file head — a speculative head is a property of the
|
||||||
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
|
# model file, not of the flags; no flag creates tensors that are absent.
|
||||||
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
||||||
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
||||||
: "${SPEC_TYPE:=none}"
|
: "${SPEC_TYPE:=none}"
|
||||||
@@ -81,13 +89,18 @@ fi
|
|||||||
|
|
||||||
# Fail loudly rather than half-loading: a missing file is a download that
|
# Fail loudly rather than half-loading: a missing file is a download that
|
||||||
# has not finished, and Restart=always would turn it into a crash loop that
|
# has not finished, and Restart=always would turn it into a crash loop that
|
||||||
# reads like a runtime fault.
|
# reads like a runtime fault. The refusal text is the shared gate library's,
|
||||||
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
|
# so the deployer's twin refuses identically.
|
||||||
[[ -e "$path" ]] || { echo "frank-secondary: missing $path" >&2; exit 1; }
|
require_file "$LLAMA_BIN" runtime \
|
||||||
done
|
"build it: scripts/build-llama-master-cuda.sh" || exit 1
|
||||||
|
require_file "$MODEL_FILE" gguf \
|
||||||
|
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
|
||||||
|
if [[ -n "$MMPROJ_FILE" ]]; then
|
||||||
|
require_file "$MMPROJ_FILE" mmproj \
|
||||||
|
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
|
||||||
|
fi
|
||||||
EXTRA_ARGS=()
|
EXTRA_ARGS=()
|
||||||
if [[ -n "$MMPROJ_FILE" ]]; then
|
if [[ -n "$MMPROJ_FILE" ]]; then
|
||||||
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-secondary: missing $MMPROJ_FILE" >&2; exit 1; }
|
|
||||||
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -97,7 +110,7 @@ fi
|
|||||||
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
||||||
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
|
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
|
||||||
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
|
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
|
||||||
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-secondary: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
|
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-zealot: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
|
||||||
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
|
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
@@ -108,7 +121,7 @@ nvidia-modprobe -u -c=0 2>/dev/null || true
|
|||||||
|
|
||||||
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
|
# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
|
||||||
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
|
# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
|
||||||
# the q8_0 KV contract in frank-secondary.env is a real path — without this
|
# the q8_0 KV contract in frank-zealot.env is a real path — without this
|
||||||
# flag or that build option the engine silently falls back and the
|
# flag or that build option the engine silently falls back and the
|
||||||
# contract becomes a lie.
|
# contract becomes a lie.
|
||||||
ARGS=(
|
ARGS=(
|
||||||
Executable
+750
@@ -0,0 +1,750 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""service-map.py — the only reader of the service inventory.
|
||||||
|
|
||||||
|
Four lists used to agree by discipline (the planner's path rules, the
|
||||||
|
deployer's allow-list, the changed-test mapper, and the scrape/probe/DNS
|
||||||
|
facts written by hand in three config files). They now all read
|
||||||
|
`shared/services.yml` through this tool:
|
||||||
|
|
||||||
|
hosts the deploy targets
|
||||||
|
services [--host H] deployables a target accepts, canonical order
|
||||||
|
allows --host H --service S exit 0 when H accepts S (silent)
|
||||||
|
plan --host H PATH... services selected by changed paths, order-safe
|
||||||
|
seams [--host H] PATH... the Bats seams a changed path must re-run
|
||||||
|
owner --host H PATH the service that owns a file (empty: none)
|
||||||
|
units --host H --service S "<unit> <unit_dir>" lines the handler installs
|
||||||
|
stack --host H --service S the deployable + members one handler deploys
|
||||||
|
get --service S [--host H] F one field as JSON
|
||||||
|
render pihole-hosts the Pi-hole addn-hosts file
|
||||||
|
check [NAME...] integrity + drift gates (exit 1 on drift)
|
||||||
|
|
||||||
|
Two rules the engine enforces structurally:
|
||||||
|
|
||||||
|
* A path selects only services that RUN on the planned target. Path
|
||||||
|
matching that is host-blind (match first, filter later) lets a file on
|
||||||
|
one host plan a service on another; selection is scoped to the host the
|
||||||
|
path belongs to.
|
||||||
|
* The netns family is DERIVED, never declared: a unit file that says
|
||||||
|
`Network=gluetun.container` rides that tunnel, so a gluetun change selects
|
||||||
|
every sharer. The unit files are the source of that coupling.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from functools import lru_cache
|
||||||
|
from pathlib import Path, PurePosixPath
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
DEFAULT_INVENTORY = REPO_ROOT / "shared" / "services.yml"
|
||||||
|
|
||||||
|
REQUIRED_KEYS = ("name", "hosts", "kind", "tests")
|
||||||
|
KINDS = {"quadlet", "root-unit", "vm-quadlet", "host"}
|
||||||
|
UNIT_SUFFIXES = (".container", ".network")
|
||||||
|
|
||||||
|
# Repo material that is never a deployed runtime's state.
|
||||||
|
NON_RUNTIME_ROOTS = {"docs", "report", "old", ".githooks", ".obsidian", ".github",
|
||||||
|
"tests", "serverAppConfigs-docs"}
|
||||||
|
NON_RUNTIME_NAMES = {".env.example"}
|
||||||
|
NON_RUNTIME_PATHS = {"README.md", "AGENTS.md", ".gitignore"}
|
||||||
|
|
||||||
|
|
||||||
|
def is_trash(path: str) -> bool:
|
||||||
|
return ".AGENTTRASH" in PurePosixPath(path).parts
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# inventory
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
class Inventory:
|
||||||
|
def __init__(self, doc: dict[str, Any], origin: str) -> None:
|
||||||
|
self.origin = origin
|
||||||
|
self.hosts: dict[str, dict[str, Any]] = doc["hosts"]
|
||||||
|
self.services: list[dict[str, Any]] = doc["services"]
|
||||||
|
self.unowned: dict[str, str] = doc.get("unowned", {})
|
||||||
|
for index, svc in enumerate(self.services):
|
||||||
|
svc["_index"] = index
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def load(cls, path: str | Path = DEFAULT_INVENTORY) -> "Inventory":
|
||||||
|
doc = yaml.safe_load(Path(path).read_text())
|
||||||
|
if not isinstance(doc, dict) or "services" not in doc:
|
||||||
|
raise SystemExit(f"{path}: not a service inventory document")
|
||||||
|
return cls(doc, str(path))
|
||||||
|
|
||||||
|
def on(self, host: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if host in s["hosts"]]
|
||||||
|
|
||||||
|
def named(self, host: str, name: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if s["name"] == name and host in s["hosts"]]
|
||||||
|
|
||||||
|
def any_named(self, name: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if s["name"] == name]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def order(self) -> list[str]:
|
||||||
|
seen: list[str] = []
|
||||||
|
for svc in self.services:
|
||||||
|
if svc["name"] not in seen:
|
||||||
|
seen.append(svc["name"])
|
||||||
|
return seen
|
||||||
|
|
||||||
|
def edge_ip(self) -> str:
|
||||||
|
"""The single address every DNS name resolves at (Caddy's edge)."""
|
||||||
|
for host in self.hosts.values():
|
||||||
|
if "dns_edge" in host:
|
||||||
|
return str(host["dns_edge"])
|
||||||
|
raise SystemExit(f"{self.origin}: no host declares dns_edge")
|
||||||
|
|
||||||
|
def service(self, name: str) -> dict | None:
|
||||||
|
for svc in self.services:
|
||||||
|
if svc["name"] == name:
|
||||||
|
return svc
|
||||||
|
return None
|
||||||
|
|
||||||
|
def unit_dirs(self, svc: dict[str, Any], host: str | None = None) -> list[str]:
|
||||||
|
"""The repo dirs holding this service's unit files, for one host."""
|
||||||
|
dirs = svc.get("unit_dir")
|
||||||
|
if not dirs:
|
||||||
|
return []
|
||||||
|
if isinstance(dirs, dict):
|
||||||
|
if host is not None:
|
||||||
|
return [dirs[host]] if host in dirs else []
|
||||||
|
return [dirs[h] for h in svc["hosts"] if h in dirs]
|
||||||
|
return [dirs]
|
||||||
|
|
||||||
|
def owns_unit(self, svc: dict[str, Any], host: str, unit: str, directory: str) -> bool:
|
||||||
|
return unit in svc.get("units", []) and directory in self.unit_dirs(svc, host)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# repo facts
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def tracked_files() -> tuple[str, ...]:
|
||||||
|
# Tracked + untracked (a check that only saw HEAD would ignore the very
|
||||||
|
# files a work-in-progress commit is adding).
|
||||||
|
out = subprocess.run(
|
||||||
|
["git", "-C", str(REPO_ROOT), "ls-files", "--cached", "--others",
|
||||||
|
"--exclude-standard"],
|
||||||
|
text=True, stdout=subprocess.PIPE, check=True)
|
||||||
|
return tuple(line for line in out.stdout.splitlines() if line)
|
||||||
|
|
||||||
|
|
||||||
|
def _network_values(container: Path) -> list[str]:
|
||||||
|
"""Every `Network=` value a quadlet unit declares (comma lists included)."""
|
||||||
|
values: list[str] = []
|
||||||
|
try:
|
||||||
|
text = container.read_text()
|
||||||
|
except OSError:
|
||||||
|
return values
|
||||||
|
for line in text.splitlines():
|
||||||
|
if line.startswith("Network="):
|
||||||
|
values.extend(v.strip() for v in line[len("Network="):].split(",") if v.strip())
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
class Graph:
|
||||||
|
"""The unit-file coupling, read out of the tree: netns + netns-net users.
|
||||||
|
|
||||||
|
Quadlet installs every user unit flat, so `Network=gluetun.container`
|
||||||
|
names a unit by name, not by path; the same host is the scope, which is
|
||||||
|
also the deployment scope.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, hosts: list[str]) -> None:
|
||||||
|
self.units: dict[tuple[str, str], str] = {} # (host, unit) -> path
|
||||||
|
self.networks: dict[tuple[str, str], str] = {} # (host, net) -> path
|
||||||
|
self.netns_children: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||||
|
self.network_users: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||||
|
for path in tracked_files():
|
||||||
|
parts = PurePosixPath(path).parts
|
||||||
|
if not parts or parts[0] not in hosts or is_trash(path):
|
||||||
|
continue
|
||||||
|
host, directory, name = parts[0], str(PurePosixPath(*parts[1:-1])), parts[-1]
|
||||||
|
if name.endswith(".container"):
|
||||||
|
self.units[(host, name[: -len(".container")])] = path
|
||||||
|
elif name.endswith(".network"):
|
||||||
|
self.networks[(host, name[: -len(".network")])] = path
|
||||||
|
for (host, unit), path in self.units.items():
|
||||||
|
for value in _network_values(REPO_ROOT / path):
|
||||||
|
if value.endswith(".container"):
|
||||||
|
parent = (host, value[: -len(".container")])
|
||||||
|
self.netns_children.setdefault(parent, set()).add((host, unit))
|
||||||
|
elif value.endswith(".network"):
|
||||||
|
net = (host, value[: -len(".network")])
|
||||||
|
self.network_users.setdefault(net, set()).add((host, unit))
|
||||||
|
|
||||||
|
def rides(self, start: list[tuple[str, str]]) -> set[tuple[str, str]]:
|
||||||
|
"""The transitive netns closure: a unit, everything riding it, and
|
||||||
|
everything riding those (a netns inside a netns)."""
|
||||||
|
found: set[tuple[str, str]] = set()
|
||||||
|
frontier = list(start)
|
||||||
|
while frontier:
|
||||||
|
node = frontier.pop()
|
||||||
|
if node in found:
|
||||||
|
continue
|
||||||
|
found.add(node)
|
||||||
|
frontier.extend(self.netns_children.get(node, ()))
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=8)
|
||||||
|
def graph_for(hosts: tuple[str, ...]) -> Graph:
|
||||||
|
return Graph(list(hosts))
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# selection
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def _owner_index(inv: Inventory) -> dict[tuple[str, str, int]]:
|
||||||
|
"""(host, unit) -> service index, for unit-file ownership.
|
||||||
|
|
||||||
|
A quadlet unit name is unique per host (``check units`` enforces one
|
||||||
|
claim), so the coupling closure can resolve a riding unit that lives in
|
||||||
|
a DIFFERENT directory than the changed file — gluetun's netns children
|
||||||
|
are the point.
|
||||||
|
"""
|
||||||
|
index: dict[tuple[str, str], int] = {}
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
key = (host, unit)
|
||||||
|
if key in index and index[key] != svc["_index"]:
|
||||||
|
raise SystemExit(
|
||||||
|
f"{inv.origin}: unit {unit!r} on {host} claimed by "
|
||||||
|
f"{inv.services[index[key]]['name']} and {svc['name']}"
|
||||||
|
)
|
||||||
|
index[key] = svc["_index"]
|
||||||
|
return index
|
||||||
|
|
||||||
|
|
||||||
|
def select(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||||
|
"""Canonical-order service names the changed paths select for one host."""
|
||||||
|
if host not in inv.hosts:
|
||||||
|
raise SystemExit(f"{inv.origin}: no host {host!r}")
|
||||||
|
graph = graph_for(tuple(inv.hosts))
|
||||||
|
owners = _owner_index(inv)
|
||||||
|
hits: set[int] = set()
|
||||||
|
|
||||||
|
def add(index: int) -> None:
|
||||||
|
hits.add(index)
|
||||||
|
|
||||||
|
for raw in paths:
|
||||||
|
path = raw.replace("\\", "/")
|
||||||
|
parts = PurePosixPath(path).parts
|
||||||
|
if not parts or is_trash(path) or path in NON_RUNTIME_PATHS:
|
||||||
|
continue
|
||||||
|
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||||
|
continue
|
||||||
|
head = parts[0]
|
||||||
|
if head in NON_RUNTIME_ROOTS:
|
||||||
|
continue
|
||||||
|
# A host's directory is the scope: a frankbook-pro file can never
|
||||||
|
# select a dankserver service.
|
||||||
|
scoped_host = head if head in inv.hosts else None
|
||||||
|
if scoped_host is not None and scoped_host != host:
|
||||||
|
continue
|
||||||
|
|
||||||
|
name = parts[-1]
|
||||||
|
directory = str(PurePosixPath(*parts[1:-1])) if len(parts) > 1 else ""
|
||||||
|
|
||||||
|
if scoped_host and name.endswith(UNIT_SUFFIXES):
|
||||||
|
stem = name.rsplit(".", 1)[0]
|
||||||
|
key = (scoped_host, stem)
|
||||||
|
if name.endswith(".network") and key in graph.networks:
|
||||||
|
users = graph.network_users.get(key, set())
|
||||||
|
for unit in graph.rides(sorted(users)):
|
||||||
|
index = owners.get(unit)
|
||||||
|
if index is not None:
|
||||||
|
add(index)
|
||||||
|
elif key in graph.units:
|
||||||
|
for unit in graph.rides([key]):
|
||||||
|
index = owners.get(unit)
|
||||||
|
if index is not None:
|
||||||
|
add(index)
|
||||||
|
|
||||||
|
for index in list(hits):
|
||||||
|
for dependent in inv.services[index].get("dependents", []):
|
||||||
|
for svc in inv.named(host, dependent):
|
||||||
|
add(svc["_index"])
|
||||||
|
|
||||||
|
for svc in inv.on(host):
|
||||||
|
if path in svc.get("files", []):
|
||||||
|
add(svc["_index"])
|
||||||
|
if path in svc.get("retires", []):
|
||||||
|
add(svc["_index"])
|
||||||
|
for directory_rule in svc.get("dirs", []):
|
||||||
|
if path == directory_rule or path.startswith(directory_rule + "/"):
|
||||||
|
add(svc["_index"])
|
||||||
|
|
||||||
|
return [svc["name"] for svc in sorted((inv.services[i] for i in hits),
|
||||||
|
key=lambda s: s["_index"])]
|
||||||
|
|
||||||
|
|
||||||
|
def seams_for(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||||
|
"""The Bats seams the selected services require, deduplicated, in order."""
|
||||||
|
seams: list[str] = []
|
||||||
|
for name in select(inv, host, paths):
|
||||||
|
for svc in inv.named(host, name):
|
||||||
|
for seam in svc.get("tests", []):
|
||||||
|
if seam not in seams:
|
||||||
|
seams.append(seam)
|
||||||
|
if seams and "tests/deployment_scoping.bats" not in seams:
|
||||||
|
seams.append("tests/deployment_scoping.bats")
|
||||||
|
return seams
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# renderers + facts
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def render_pihole_hosts(inv: Inventory) -> str:
|
||||||
|
edge = inv.edge_ip()
|
||||||
|
lines = [
|
||||||
|
"# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts",
|
||||||
|
"# format: <IP> <name>). RENDERED — do not edit by hand:",
|
||||||
|
"#",
|
||||||
|
"# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts",
|
||||||
|
"#",
|
||||||
|
"# The source of truth is the `dns:` field of shared/services.yml; the",
|
||||||
|
"# drift gate in tests/service_inventory.bats fails when this file and",
|
||||||
|
"# the inventory disagree. Caddy is consolidated on Dankclaw, so every",
|
||||||
|
"# name — including the services running on dankserver and",
|
||||||
|
"# frankbook-pro — resolves at the Dankclaw edge.",
|
||||||
|
"#",
|
||||||
|
"# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing",
|
||||||
|
"# comment on a data line becomes a HOSTNAME token. No inline notes",
|
||||||
|
"# here; the service attribution lives in the inventory.",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
for svc in inv.services:
|
||||||
|
for name in svc.get("dns", []):
|
||||||
|
lines.append(f"{edge} {name}")
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def scrape_facts(inv: Inventory) -> list[dict[str, str]]:
|
||||||
|
facts = []
|
||||||
|
for svc in inv.services:
|
||||||
|
for job in svc.get("scrape", []):
|
||||||
|
facts.append({"job": job["job"], "target": job["target"],
|
||||||
|
"metrics_path": job.get("metrics_path", "/")})
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def prometheus_facts(path: Path) -> list[dict[str, str]]:
|
||||||
|
doc = yaml.safe_load(path.read_text())
|
||||||
|
facts = []
|
||||||
|
for job in doc.get("scrape_configs", []):
|
||||||
|
for sc in job.get("static_configs", []):
|
||||||
|
for target in sc.get("targets", []):
|
||||||
|
facts.append({"job": job["job_name"], "target": target,
|
||||||
|
"metrics_path": job.get("metrics_path", "/")})
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def probe_facts(inv: Inventory) -> list[str]:
|
||||||
|
facts: list[str] = []
|
||||||
|
for svc in inv.services:
|
||||||
|
probe = svc.get("probe")
|
||||||
|
if isinstance(probe, dict):
|
||||||
|
facts.extend(str(v) for v in probe.values())
|
||||||
|
elif probe:
|
||||||
|
facts.append(str(probe))
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def homepage_probes(path: Path) -> list[str]:
|
||||||
|
return sorted({s["probe"] for s in json.loads(path.read_text()) if s.get("probe")})
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# checks
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def check_ownership(inv: Inventory, graph: Graph) -> list[str]:
|
||||||
|
"""Every repo file belongs to a service, a rule, or the unowned list."""
|
||||||
|
problems: list[str] = []
|
||||||
|
exact: set[str] = set()
|
||||||
|
directories: set[str] = set()
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
exact.add(f"{directory}/{unit}.container")
|
||||||
|
for field in ("files", "retires"):
|
||||||
|
exact.update(svc.get(field, []))
|
||||||
|
directories.update(svc.get("dirs", []))
|
||||||
|
# A shared network file is claimed by the units that ride it; the unit
|
||||||
|
# graph is the referrer, and check_units proves every reference resolves.
|
||||||
|
exact.update(graph.networks.values())
|
||||||
|
|
||||||
|
def is_claimed(path: str) -> bool:
|
||||||
|
if path in exact:
|
||||||
|
return True
|
||||||
|
first = PurePosixPath(path).parts[0]
|
||||||
|
if first in NON_RUNTIME_ROOTS or path in NON_RUNTIME_PATHS or is_trash(path):
|
||||||
|
return True
|
||||||
|
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||||
|
return True
|
||||||
|
for owned in directories | set(inv.unowned):
|
||||||
|
if path == owned or path.startswith(owned + "/"):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
for path in tracked_files():
|
||||||
|
if not is_claimed(path):
|
||||||
|
problems.append(f"file belongs to no service and is not in unowned: {path}")
|
||||||
|
|
||||||
|
# The reverse direction: an unowned entry that names nothing is stale.
|
||||||
|
for directory in inv.unowned:
|
||||||
|
if not any(p == directory or p.startswith(directory + "/") for p in tracked_files()):
|
||||||
|
problems.append(f"unowned entry names no tracked file: {directory}")
|
||||||
|
|
||||||
|
# Every inventory file/dir claim names a tracked file.
|
||||||
|
for svc in inv.services:
|
||||||
|
# `retires` names a file whose DELETION selects the service, so by
|
||||||
|
# definition it is not in the tree.
|
||||||
|
for field in ("files", "dirs"):
|
||||||
|
for entry in svc.get(field, []):
|
||||||
|
if entry not in tracked_files() and not any(
|
||||||
|
p.startswith(entry + "/") for p in tracked_files()):
|
||||||
|
problems.append(f"{svc['name']}: {field} entry names no tracked file: {entry}")
|
||||||
|
del graph
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def check_units(inv: Inventory, graph: Graph) -> list[str]:
|
||||||
|
"""Every unit file has exactly one owner; every unit reference resolves."""
|
||||||
|
problems: list[str] = []
|
||||||
|
owners: dict[str, list[str]] = {}
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
path = f"{directory}/{unit}.container"
|
||||||
|
owners.setdefault(path, []).append(f"{svc['name']}@{host}")
|
||||||
|
if path not in tracked_files():
|
||||||
|
problems.append(f"{svc['name']}: unit file is not tracked: {path}")
|
||||||
|
for path in tracked_files():
|
||||||
|
if not path.endswith(".container") or is_trash(path):
|
||||||
|
continue
|
||||||
|
found = owners.get(path, [])
|
||||||
|
if not found:
|
||||||
|
problems.append(f"unit file claimed by no service: {path}")
|
||||||
|
elif len(found) > 1:
|
||||||
|
problems.append(f"unit file claimed by {len(found)} services: {path}: {found}")
|
||||||
|
|
||||||
|
# Root units (scripts/*.service) are claimed by exactly one `files:` list.
|
||||||
|
for path in tracked_files():
|
||||||
|
if not (path.startswith("scripts/") and path.endswith(".service")) or is_trash(path):
|
||||||
|
continue
|
||||||
|
found = [s["name"] for s in inv.services if path in s.get("files", [])]
|
||||||
|
if not found:
|
||||||
|
problems.append(f"root unit claimed by no service: {path}")
|
||||||
|
elif len(found) > 1:
|
||||||
|
problems.append(f"root unit claimed by {found}: {path}")
|
||||||
|
|
||||||
|
# Every Network= reference in a unit file names a file on the same host.
|
||||||
|
for (host, unit), path in graph.units.items():
|
||||||
|
for value in _network_values(REPO_ROOT / path):
|
||||||
|
if value.endswith(".container"):
|
||||||
|
target = (host, value[: -len(".container")])
|
||||||
|
if target not in graph.units:
|
||||||
|
problems.append(f"{path}: Network={value} names no unit on {host}")
|
||||||
|
elif value.endswith(".network"):
|
||||||
|
target = (host, value[: -len(".network")])
|
||||||
|
if target not in graph.networks:
|
||||||
|
problems.append(f"{path}: Network={value} names no network on {host}")
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def check_config(inv: Inventory) -> list[str]:
|
||||||
|
problems: list[str] = []
|
||||||
|
for svc in inv.services:
|
||||||
|
label = f"{'+'.join(svc['hosts'])}/{svc['name']}"
|
||||||
|
for key in REQUIRED_KEYS:
|
||||||
|
if key not in svc:
|
||||||
|
problems.append(f"{label}: missing required key {key}")
|
||||||
|
if svc.get("kind") not in KINDS:
|
||||||
|
problems.append(f"{label}: kind {svc.get('kind')!r} not one of {sorted(KINDS)}")
|
||||||
|
unknown = set(svc.get("hosts", [])) - set(inv.hosts)
|
||||||
|
if unknown:
|
||||||
|
problems.append(f"{label}: undeclared hosts: {sorted(unknown)}")
|
||||||
|
if svc.get("units") and not svc.get("unit_dir"):
|
||||||
|
problems.append(f"{label}: units declared without unit_dir")
|
||||||
|
if svc.get("deployed_by") and not inv.any_named(svc["deployed_by"]):
|
||||||
|
problems.append(f"{label}: deployed_by names no service: {svc['deployed_by']}")
|
||||||
|
if (svc.get("health") and svc.get("wait") is None) or \
|
||||||
|
(svc.get("wait") is not None and not svc.get("health")):
|
||||||
|
problems.append(f"{label}: health and wait are a pair")
|
||||||
|
for seam in svc.get("tests", []):
|
||||||
|
if seam not in tracked_files():
|
||||||
|
problems.append(f"{label}: seam does not exist: {seam}")
|
||||||
|
for name in svc.get("dns", []):
|
||||||
|
if not str(name).replace(".", "").replace("-", "").isalnum():
|
||||||
|
problems.append(f"{label}: dns name is not a plain hostname: {name}")
|
||||||
|
if svc.get("kind") == "root-unit" and svc.get("unit_dir"):
|
||||||
|
problems.append(f"{label}: a root unit lives in scripts/, not unit_dir")
|
||||||
|
seen: set[str] = set()
|
||||||
|
for svc in inv.services:
|
||||||
|
key = (svc["name"], tuple(svc["hosts"]))
|
||||||
|
if key in seen:
|
||||||
|
problems.append(f"duplicate inventory entry: {svc['name']} on {svc['hosts']}")
|
||||||
|
seen.add(key)
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
# --- node-exporter quadlet template (rank 6) --------------------------------
|
||||||
|
# Parallel per-host copies kept in step by a comment drift apart; the
|
||||||
|
# template is the single source, and `check` drift-gates every tracked
|
||||||
|
# rendering against it.
|
||||||
|
NODE_EXPORTER_TEMPLATE = """\
|
||||||
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host {host}
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
|
#
|
||||||
|
# Install path on {host}: ~/.config/containers/systemd/node-exporter.container
|
||||||
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
|
#
|
||||||
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
|
#
|
||||||
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
|
# loopback, which is what the homepage status timer probes.
|
||||||
|
#
|
||||||
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
|
# dankclaw_default — scrapes http://{lan}:{port}/metrics
|
||||||
|
# (shared/prometheus/prometheus.yml, job "node-{host}"). node_exporter has no
|
||||||
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
|
# host, and both consumers are accounted for.
|
||||||
|
#
|
||||||
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=Prometheus node exporter ({title} host metrics)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName=node-exporter
|
||||||
|
Image=docker.io/prom/node-exporter:latest
|
||||||
|
Network=host
|
||||||
|
PodmanArgs=--pid=host
|
||||||
|
Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:{port} --web.listen-address={lan}:{port}
|
||||||
|
Volume=/proc:/host/proc:ro
|
||||||
|
Volume=/sys:/host/sys:ro
|
||||||
|
Volume=/:/rootfs:ro
|
||||||
|
HealthCmd=wget --spider -q http://localhost:{port}/metrics
|
||||||
|
HealthInterval=30s
|
||||||
|
HealthTimeout=10s
|
||||||
|
HealthRetries=3
|
||||||
|
HealthStartPeriod=30s
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def render_node_exporter_unit(inv: Inventory, host: str) -> str:
|
||||||
|
"""The node-exporter quadlet for one host: shared template, per-host
|
||||||
|
title, LAN address from the hosts map, port from the scrape job."""
|
||||||
|
svc = inv.service("node-exporter")
|
||||||
|
if svc is None:
|
||||||
|
raise SystemExit("inventory has no node-exporter entry")
|
||||||
|
job = f"node-{host}"
|
||||||
|
port = next((str(e["target"]).rsplit(":", 1)[1] for e in svc.get("scrape", [])
|
||||||
|
if e.get("job") == job), None)
|
||||||
|
if port is None:
|
||||||
|
raise SystemExit(f"node-exporter scrape declares no job {job}")
|
||||||
|
lan = inv.hosts[host]["address"]
|
||||||
|
return NODE_EXPORTER_TEMPLATE.format(host=host, title=host.capitalize(),
|
||||||
|
lan=lan, port=port)
|
||||||
|
|
||||||
|
|
||||||
|
def check_drift(inv: Inventory) -> list[str]:
|
||||||
|
problems: list[str] = []
|
||||||
|
rendered = render_pihole_hosts(inv)
|
||||||
|
hosts_file = REPO_ROOT / "dankclaw" / "pihole" / "addn-hosts"
|
||||||
|
if hosts_file.read_text() != rendered:
|
||||||
|
problems.append("dankclaw/pihole/addn-hosts is not the render of the inventory "
|
||||||
|
"(scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts)")
|
||||||
|
|
||||||
|
want = sorted(json.dumps(f, sort_keys=True) for f in scrape_facts(inv))
|
||||||
|
have = sorted(json.dumps(f, sort_keys=True) for f in
|
||||||
|
prometheus_facts(REPO_ROOT / "shared" / "prometheus" / "prometheus.yml"))
|
||||||
|
for extra in [f for f in have if f not in want]:
|
||||||
|
problems.append(f"prometheus scrapes a target the inventory does not declare: {extra}")
|
||||||
|
for missing in [f for f in want if f not in have]:
|
||||||
|
problems.append(f"inventory declares a scrape prometheus.yml lacks: {missing}")
|
||||||
|
|
||||||
|
svc = inv.service("node-exporter")
|
||||||
|
for host in (svc or {}).get("hosts", []):
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
path = REPO_ROOT / directory / "node-exporter.container"
|
||||||
|
if path.exists() and path.read_text() != render_node_exporter_unit(inv, host):
|
||||||
|
problems.append(
|
||||||
|
f"{directory}/node-exporter.container is not the render of the "
|
||||||
|
f"template (scripts/service-map.py render node-exporter-unit --host {host})")
|
||||||
|
|
||||||
|
probes_want = sorted(set(probe_facts(inv)))
|
||||||
|
probes_have = homepage_probes(REPO_ROOT / "shared" / "homepage" / "services.json")
|
||||||
|
for extra in [p for p in probes_have if p not in probes_want]:
|
||||||
|
problems.append(f"homepage probes a target the inventory does not declare: {extra}")
|
||||||
|
for missing in [p for p in probes_want if p not in probes_have]:
|
||||||
|
problems.append(f"inventory declares a probe homepage does not list: {missing}")
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
CHECKS = {
|
||||||
|
"config": lambda inv, graph: check_config(inv),
|
||||||
|
"units": check_units,
|
||||||
|
"ownership": check_ownership,
|
||||||
|
"drift": lambda inv, graph: check_drift(inv),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def check(inv: Inventory, names: list[str]) -> list[str]:
|
||||||
|
graph = graph_for(tuple(inv.hosts))
|
||||||
|
problems: list[str] = []
|
||||||
|
for name in names or list(CHECKS):
|
||||||
|
problems.extend(CHECKS[name](inv, graph))
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# CLI
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__,
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
parser.add_argument("--inventory", type=Path, default=DEFAULT_INVENTORY)
|
||||||
|
sub = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
sub.add_parser("hosts")
|
||||||
|
p = sub.add_parser("services")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p = sub.add_parser("allows")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("plan")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("paths", nargs="+")
|
||||||
|
p = sub.add_parser("seams")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p.add_argument("paths", nargs="+")
|
||||||
|
p = sub.add_parser("owner")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("path")
|
||||||
|
p = sub.add_parser("units")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("stack")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("get")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p.add_argument("field")
|
||||||
|
p = sub.add_parser("render")
|
||||||
|
p.add_argument("artifact", choices=("pihole-hosts", "node-exporter-unit"))
|
||||||
|
p.add_argument("--host", choices=list(Hosts) if False else None)
|
||||||
|
p = sub.add_parser("check")
|
||||||
|
p.add_argument("names", nargs="*")
|
||||||
|
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
inv = Inventory.load(args.inventory)
|
||||||
|
|
||||||
|
if args.command == "hosts":
|
||||||
|
print("\n".join(inv.hosts))
|
||||||
|
elif args.command == "services":
|
||||||
|
services = inv.on(args.host) if args.host else inv.services
|
||||||
|
seen: list[str] = []
|
||||||
|
for svc in services:
|
||||||
|
if svc["name"] not in seen:
|
||||||
|
seen.append(svc["name"])
|
||||||
|
print("\n".join(seen))
|
||||||
|
elif args.command == "allows":
|
||||||
|
return 0 if inv.named(args.host, args.service) else 1
|
||||||
|
elif args.command == "plan":
|
||||||
|
print("\n".join(select(inv, args.host, args.paths)))
|
||||||
|
elif args.command == "seams":
|
||||||
|
if args.host is not None:
|
||||||
|
seams = seams_for(inv, args.host, args.paths)
|
||||||
|
else:
|
||||||
|
# No host: the union across every host (the dev machine's
|
||||||
|
# pre-commit mapper has no deployment target).
|
||||||
|
seams = []
|
||||||
|
for host in inv.hosts:
|
||||||
|
for seam in seams_for(inv, host, args.paths):
|
||||||
|
if seam not in seams:
|
||||||
|
seams.append(seam)
|
||||||
|
print("\n".join(seams))
|
||||||
|
elif args.command == "owner":
|
||||||
|
found = select(inv, args.host, [args.path])
|
||||||
|
print(found[-1] if found else "")
|
||||||
|
elif args.command == "units":
|
||||||
|
matches = inv.named(args.host, args.service)
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
for svc in matches:
|
||||||
|
for directory in inv.unit_dirs(svc, args.host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
print(f"{unit} {directory}")
|
||||||
|
elif args.command == "stack":
|
||||||
|
matches = inv.named(args.host, args.service)
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
owner = matches[0].get("deployed_by") or args.service
|
||||||
|
print(owner)
|
||||||
|
for svc in inv.on(args.host):
|
||||||
|
if svc.get("deployed_by") == owner and svc["name"] != owner:
|
||||||
|
print(svc["name"])
|
||||||
|
elif args.command == "get":
|
||||||
|
matches = inv.on(args.host) if args.host else inv.services
|
||||||
|
matches = [s for s in matches if s["name"] == args.service]
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service}" + (f" on {args.host}" if args.host else ""),
|
||||||
|
file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
values = [s.get(args.field) for s in matches if args.field in s]
|
||||||
|
print(json.dumps(values[0] if len(values) == 1 else values))
|
||||||
|
elif args.command == "render":
|
||||||
|
if args.artifact == "pihole-hosts":
|
||||||
|
sys.stdout.write(render_pihole_hosts(inv))
|
||||||
|
else:
|
||||||
|
if not args.host:
|
||||||
|
print("render node-exporter-unit requires --host", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
sys.stdout.write(render_node_exporter_unit(inv, args.host))
|
||||||
|
elif args.command == "check":
|
||||||
|
problems = check(inv, args.names)
|
||||||
|
if problems:
|
||||||
|
for problem in problems:
|
||||||
|
print(f"FAIL: {problem}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print("OK: the service inventory is sound (schema, units, ownership, drift)")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -293,12 +293,12 @@ def main():
|
|||||||
ddt, dsh = meta[f"{pre0}.mlp.experts.0.down_proj.weight"][3:5]
|
ddt, dsh = meta[f"{pre0}.mlp.experts.0.down_proj.weight"][3:5]
|
||||||
gsd, gssh = meta[f"{pre0}.mlp.experts.0.gate_proj.weight_scale_inv"][3:5]
|
gsd, gssh = meta[f"{pre0}.mlp.experts.0.gate_proj.weight_scale_inv"][3:5]
|
||||||
dsd, dssh = meta[f"{pre0}.mlp.experts.0.down_proj.weight_scale_inv"][3:5]
|
dsd, dssh = meta[f"{pre0}.mlp.experts.0.down_proj.weight_scale_inv"][3:5]
|
||||||
scale_law = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
|
scale_contract = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
|
||||||
and list(gssh) == [(gsh[0] + 127) // 128, (gsh[1] + 127) // 128]
|
and list(gssh) == [(gsh[0] + 127) // 128, (gsh[1] + 127) // 128]
|
||||||
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
||||||
if not scale_law:
|
if not scale_contract:
|
||||||
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
||||||
f"down {ddt}{dsh} scale {dsd}{dssh} (scale law 128x128)")
|
f"down {ddt}{dsh} scale {dsd}{dssh} (fp8 block scales cover 128x128 tiles)")
|
||||||
return 2
|
return 2
|
||||||
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
||||||
"gs": tuple(gssh), "ds": tuple(dssh)}
|
"gs": tuple(gssh), "ds": tuple(dssh)}
|
||||||
|
|||||||
+57
-190
@@ -62,234 +62,101 @@ declare -a TESTS=()
|
|||||||
add_test() {
|
add_test() {
|
||||||
local test="$1"
|
local test="$1"
|
||||||
local existing
|
local existing
|
||||||
|
MAPPED=1
|
||||||
for existing in "${TESTS[@]:-}"; do
|
for existing in "${TESTS[@]:-}"; do
|
||||||
[ "$existing" != "$test" ] || return 0
|
[ "$existing" != "$test" ] || return 0
|
||||||
done
|
done
|
||||||
TESTS+=("$test")
|
TESTS+=("$test")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The path->test mapping has two layers:
|
||||||
|
# 1. cross-cutting seams that no single service owns (compose validity,
|
||||||
|
# env consistency, the deployer itself, docs, syntax);
|
||||||
|
# 2. the service inventory (shared/services.yml via service-map.py
|
||||||
|
# "seams"): every service the path selects on ANY host contributes its
|
||||||
|
# declared tests. Service seams are the services' own — the former
|
||||||
|
# shotgun fallbacks (every dankclaw quadlet unit ran the open-webui,
|
||||||
|
# pihole, and searxng seams) are retired; the inventory names each
|
||||||
|
# service's real seams, and adding a service is a data edit.
|
||||||
for path in "${CHANGED[@]}"; do
|
for path in "${CHANGED[@]}"; do
|
||||||
|
MAPPED=0
|
||||||
case "$path" in
|
case "$path" in
|
||||||
tests/*.bats)
|
tests/*.bats)
|
||||||
add_test "$path"
|
add_test "$path"
|
||||||
|
continue
|
||||||
;;
|
;;
|
||||||
tests/helpers.bash|tests/run.sh)
|
tests/helpers.bash|tests/run.sh)
|
||||||
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
||||||
;;
|
;;
|
||||||
.githooks/*)
|
|
||||||
add_test tests/changed_tests.bats
|
|
||||||
;;
|
|
||||||
dankserver/compose.yaml|dankclaw/compose.yaml)
|
|
||||||
add_test tests/compose_validity.bats
|
|
||||||
add_test tests/env_consistency.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankserver/Caddyfile|dankclaw/Caddyfile)
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
tests/e2e/*)
|
tests/e2e/*)
|
||||||
add_test tests/e2e_scaffold.bats
|
add_test tests/e2e_scaffold.bats
|
||||||
|
continue
|
||||||
;;
|
;;
|
||||||
|
.githooks/*)
|
||||||
|
add_test tests/changed_tests.bats
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
# Archive material and host-level facts: a path under any
|
||||||
|
# .AGENTTRASH/ directory is retired-by-policy (the planner plans
|
||||||
|
# nothing from it), udev rules are applied by manual sudo install,
|
||||||
|
# reports are frozen artifacts, and ignore-lists change nothing at
|
||||||
|
# runtime. They select no focused test; without this case they would
|
||||||
|
# trip the mapping gate below.
|
||||||
|
.AGENTTRASH/*|*/.AGENTTRASH/*|dankclaw/udev/*|report/*|.gitignore|.env.example)
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Cross-cutting seams.
|
||||||
|
case "$path" in
|
||||||
dankserver/.env.example|dankclaw/.env.example)
|
dankserver/.env.example|dankclaw/.env.example)
|
||||||
add_test tests/env_consistency.bats
|
add_test tests/env_consistency.bats
|
||||||
;;
|
;;
|
||||||
shared/homepage/*)
|
dankserver/Caddyfile|dankclaw/Caddyfile)
|
||||||
add_test tests/homepage_caddy.bats
|
|
||||||
add_test tests/homepage_config.bats
|
|
||||||
add_test tests/homepage_deploy.bats
|
|
||||||
add_test tests/homepage_pages.bats
|
|
||||||
add_test tests/homepage_checker.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
shared/searxng/*)
|
|
||||||
add_test tests/searxng_limiter.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# The five observability containers left compose for Quadlet units on
|
|
||||||
# 2026-09-11. Their seam pins boot persistence and the loopback-only
|
|
||||||
# publish posture that keeps them off every host interface. This case
|
|
||||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
||||||
# would otherwise swallow it into unrelated seams.
|
|
||||||
dankclaw/observability/*)
|
|
||||||
add_test tests/observability_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
add_test tests/caddyfile.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
# Caddy left compose.yaml on 2026-08-26 for a quadlet; caddyfile.bats
|
|
||||||
# now asserts against the quadlet itself. Route every change under
|
|
||||||
# dankclaw/caddy/ to both seams (this block must precede the
|
|
||||||
# dankclaw/*.container glob, which would otherwise swallow the unit).
|
|
||||||
dankclaw/caddy/*)
|
|
||||||
add_test tests/caddy_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/*.container|dankclaw/*.network)
|
|
||||||
add_test tests/open_webui_quadlet.bats
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# Anubis is a quadlet + Caddyfile pair: the chain (public listener ->
|
|
||||||
# anubis:8923 -> internal :8016 listener) spans both, so any change
|
|
||||||
# under the service dir must re-run the caddyfile seam too. This case
|
|
||||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
||||||
# would otherwise swallow it into unrelated seams.
|
|
||||||
dankclaw/anubis/*)
|
|
||||||
add_test tests/anubis.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/homepage_caddy.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# frank-large is a rootless quadlet again since 2026-09-16 (the
|
|
||||||
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
|
||||||
# podman posture, so it gets its own seam rather than the generic
|
|
||||||
# fallback below.
|
|
||||||
dankclaw/frank-large/*)
|
|
||||||
add_test tests/frank_large_unit.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
# Quadlets live under dankclaw/<svc>/quadlet/ (reorganized out of the
|
|
||||||
# top-level dankclaw/*.container globs above). Map the whole subtree so
|
|
||||||
# changes to the pihole, open-webui, searxng, and gitea quadlets hit
|
|
||||||
# their seam tests. The pihole-specific block must come first: shell
|
|
||||||
# case matches the first pattern, and dankclaw/pihole/quadlet/*.container
|
|
||||||
# also satisfies the dankclaw/*/quadlet/*.container fallback.
|
|
||||||
dankclaw/pihole/*)
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/*/quadlet/*.container|dankclaw/*/quadlet/*.network)
|
|
||||||
add_test tests/open_webui_quadlet.bats
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
;;
|
||||||
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
||||||
add_test tests/deployment_flow.bats
|
add_test tests/deployment_flow.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
|
scripts/service-map.py|shared/services.yml)
|
||||||
|
add_test tests/service_inventory.bats
|
||||||
|
add_test tests/deployment_scoping.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
scripts/lib/*.sh)
|
||||||
|
add_test tests/llm_gates.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
serverAppConfigs-docs/*|README.md|AGENTS.md)
|
||||||
|
add_test tests/docs_completeness.bats
|
||||||
|
add_test tests/deployment_scoping.bats
|
||||||
|
;;
|
||||||
scripts/mount-entropy.sh)
|
scripts/mount-entropy.sh)
|
||||||
add_test tests/mount_entropy.bats
|
add_test tests/mount_entropy.bats
|
||||||
;;
|
|
||||||
# Archive material: a path under any .AGENTTRASH/ directory is
|
|
||||||
# retired-by-policy (deploy-plan.py plans nothing from it), so it
|
|
||||||
# selects no focused test. It still needs its own case: without one
|
|
||||||
# the wrapper refuses retirement commits, whose trash moves show up
|
|
||||||
# as additions, with "no focused-test mapping".
|
|
||||||
.AGENTTRASH/*|*/.AGENTTRASH/*)
|
|
||||||
;;
|
|
||||||
# frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the
|
|
||||||
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
|
|
||||||
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
|
|
||||||
# live in scripts/, which is why this case has to precede the generic
|
|
||||||
# scripts/*.service and scripts/* fallbacks below.
|
|
||||||
scripts/frank-primary.*|scripts/serve-frank-primary.sh)
|
|
||||||
add_test tests/frank_primary_unit.bats
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
# frank-large's chat template lives in scripts/ (moved there on
|
old/scripts/*)
|
||||||
# 2026-09-26 when its old home was retired): a template edit changes
|
|
||||||
# what the live endpoint renders, so it selects the large unit seam.
|
|
||||||
scripts/qwen-sharp-v22.4.0.jinja)
|
|
||||||
add_test tests/frank_large_unit.bats
|
|
||||||
;;
|
|
||||||
# frank-secondary is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
|
||||||
# binary, so no generated quadlet is involved. Its env, unit and serve
|
|
||||||
# script all live in scripts/, which is why this case has to precede
|
|
||||||
# the generic scripts/*.service and scripts/* fallbacks below.
|
|
||||||
scripts/frank-secondary.*|scripts/serve-frank-secondary.sh)
|
|
||||||
add_test tests/frank_secondary_unit.bats
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
scripts/*.service)
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
scripts/*)
|
scripts/*)
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
serverAppConfigs-docs/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/hermes/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
README.md|AGENTS.md)
|
|
||||||
# Project guidance changes can affect the documented deployment
|
|
||||||
# contract; validate the docs vault and scoped deployment seams.
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/comfyui/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# loki and alloy images are built from this repo's Containerfiles; the
|
|
||||||
# loki one carries the healthcheck command (see its comment), so both
|
|
||||||
# seams read it.
|
|
||||||
dankclaw/loki/*|dankclaw/alloy/*)
|
|
||||||
add_test tests/observability_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/udev/*)
|
|
||||||
# Host-level udev rules are applied by manual sudo install + udevadm
|
|
||||||
# trigger, not dispatched as a service; nothing here is bats-testable.
|
|
||||||
;;
|
|
||||||
.AGENTTRASH/*)
|
|
||||||
# Trash moves are not testable; skip.
|
|
||||||
;;
|
|
||||||
.gitignore|.env.example)
|
|
||||||
# Ignore/ignore-list changes change nothing at runtime; not testable.
|
|
||||||
;;
|
|
||||||
old/scripts/*|old/scripts/**)
|
|
||||||
add_test tests/scripts_syntax.bats
|
|
||||||
;;
|
|
||||||
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
|
|
||||||
# copyparty, plex, tautulli): the seam checks boot-persistence +
|
|
||||||
# v1-generator contract; deploy-flow seams guard the deployer routing.
|
|
||||||
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
|
|
||||||
add_test tests/dankserver_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# FrankBook-Pro units live inside the podman machine VM; the seam
|
|
||||||
# pins boot persistence, the gluetun netns invariant, and the
|
|
||||||
# plan-locked volume layout. Deployer changes route here too.
|
|
||||||
frankbook-pro/*)
|
|
||||||
add_test tests/frankbook_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# Seerr unit on dankclaw: boot persistence, loopback-only port
|
|
||||||
# posture, and the deployer routing that installs it.
|
|
||||||
dankclaw/seerr/*)
|
|
||||||
add_test tests/seerr_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
report/*)
|
|
||||||
# Frozen audit reports are point-in-time artifacts; not testable.
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "no focused-test mapping for '$path'; add one before committing"
|
|
||||||
;;
|
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
# The inventory's declared seams for every service this path selects.
|
||||||
|
while IFS= read -r seam; do
|
||||||
|
[ -n "$seam" ] && add_test "$seam"
|
||||||
|
done < <(python3 "$REPO_ROOT/scripts/service-map.py" seams -- "$path" 2>/dev/null || true)
|
||||||
|
|
||||||
|
# Mapping gate: a path that selected no test at all — no service seam,
|
||||||
|
# no cross-cutting rule — is an unmapped area of the repo; refuse to
|
||||||
|
# commit it silently (the old catch-all `die`).
|
||||||
|
if [ "$MAPPED" -eq 0 ]; then
|
||||||
|
die "no focused-test mapping for '$path'; map it in the inventory (shared/services.yml) before committing"
|
||||||
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
||||||
|
|||||||
@@ -16,13 +16,13 @@ this file is repo-managed; edits are harness-side only.
|
|||||||
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
||||||
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
|
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
|
||||||
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
|
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
|
||||||
:7771, the 3090 compaction hub) — formerly `frank-primary`, `frank-large`,
|
:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
|
||||||
`frank-secondary`. The SERVICES keep the old names (units, scripts, env
|
`frank-zealot`. The SERVICES keep the old names (units, scripts, env
|
||||||
files, runbook pages); pages written in the old era name the service,
|
files, runbook pages); pages written in the old era name the service,
|
||||||
the harness names the model. The wire now matches: each endpoint's
|
the harness names the model. The wire now matches: each endpoint's
|
||||||
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
|
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
|
||||||
picker/grants, the homepage docs and the raw API all use one id per model
|
picker/grants, the homepage docs and the raw API all use one id per model
|
||||||
(see [[OpenWebUI]] for the grant re-key and the [[frank-primary-radiance-swift15-w4dn8]]
|
(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
|
||||||
note on radiance echoing the requested name).
|
note on radiance echoing the requested name).
|
||||||
|
|
||||||
## Compaction chain arithmetic (verify before ANY window change)
|
## Compaction chain arithmetic (verify before ANY window change)
|
||||||
@@ -45,14 +45,14 @@ The arithmetic (the coupled contract every endpoint's maxTokens comment
|
|||||||
encodes):
|
encodes):
|
||||||
|
|
||||||
1. **Source trigger** = `thresholdPercent × source contextWindow`
|
1. **Source trigger** = `thresholdPercent × source contextWindow`
|
||||||
(0.70 × 200,704 = 140,492 for frank-primary on the current shape)
|
(0.70 × 200,704 = 140,492 for frank-archon on the current shape)
|
||||||
2. **Output budget** = capped at the target model's `maxTokens`
|
2. **Output budget** = capped at the target model's `maxTokens`
|
||||||
(32,768 for frank-secondary; 36,864 frank-primary; 57,344 frank-large — the
|
(32,768 for frank-zealot; 36,864 frank-archon; 57,344 frank-dragoon — the
|
||||||
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
|
largest, because halogen's `HALOGEN_MAX_TOKENS_CAP` must stay ≥ it)
|
||||||
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
|
3. **REQUIREMENT**: prompt (≤ trigger) + budget ≤ the target's per-slot
|
||||||
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
|
window. Worked pairs from the live registry: 140,492 + 36,864 = 177,356 ≤
|
||||||
200,704 (frank-primary); 140,490 + 57,344 = 197,834 ≤ 200,700
|
200,704 (frank-archon); 140,490 + 57,344 = 197,834 ≤ 200,700
|
||||||
(frank-large); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-secondary).
|
(frank-dragoon); 140,492 + 32,768 = 173,260 ≤ 200,704 (frank-zealot).
|
||||||
|
|
||||||
Procedure (recompute from the LIVE files every time — windows rotate with
|
Procedure (recompute from the LIVE files every time — windows rotate with
|
||||||
every swap; the numbers above are the 2026-10-05 state):
|
every swap; the numbers above are the 2026-10-05 state):
|
||||||
@@ -112,19 +112,19 @@ Use **wire names**, not OMP camelCase. Two traps:
|
|||||||
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
|
`qwenPreserveThinking` is auto-derived in `buildOpenAICompat` from the
|
||||||
*auto-detected* `thinkingFormat` + local baseUrl, **before**
|
*auto-detected* `thinkingFormat` + local baseUrl, **before**
|
||||||
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
|
`applyCompatOverrides`. Setting `compat.thinkingFormat: qwen` in
|
||||||
models.yml does **not** enable it. Custom-named models (`frank-large`)
|
models.yml does **not** enable it. Custom-named models (`frank-dragoon`)
|
||||||
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
match no Qwen id pattern → use `extraBody: {preserve_thinking: true}`.
|
||||||
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
|
2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
|
||||||
also sets it — `Object.assign` is shallow and replaces the whole object,
|
also sets it — `Object.assign` is shallow and replaces the whole object,
|
||||||
silently dropping the `reasoning_effort` kwarg the compat policy set.
|
silently dropping the `reasoning_effort` kwarg the compat policy set.
|
||||||
Top-level `preserve_thinking` is honoured by halogen. (frank-primary's
|
Top-level `preserve_thinking` is honoured by halogen. (frank-archon's
|
||||||
FreeToken entry currently carries a `chat_template_kwargs` extraBody with
|
FreeToken entry currently carries a `chat_template_kwargs` extraBody with
|
||||||
a comment asking to verify-and-prune — the exact trap to check at that
|
a comment asking to verify-and-prune — the exact trap to check at that
|
||||||
endpoint's next swap.)
|
endpoint's next swap.)
|
||||||
|
|
||||||
**Gotcha: silent greedy decode.** halogen has **no server-side sampler
|
**Gotcha: silent greedy decode.** halogen has **no server-side sampler
|
||||||
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
|
defaults** — `temperature` absent or `0` = greedy. If OMP sends nothing, the
|
||||||
endpoint decodes greedy while looking configured. This is why frank-large's
|
endpoint decodes greedy while looking configured. This is why frank-dragoon's
|
||||||
entry carries the recipe on the wire and a comment saying "do not drop it".
|
entry carries the recipe on the wire and a comment saying "do not drop it".
|
||||||
|
|
||||||
**Reasoning-effort folding.** Qwen3.8's template accepts only
|
**Reasoning-effort folding.** Qwen3.8's template accepts only
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
|
|||||||
|
|
||||||
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
|
**Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
|
||||||
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
|
whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
|
||||||
a 100+ GiB payload — and frank-primary (dankserver:7770) is also the
|
a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
|
||||||
harness's own model backend, so a blind deploy is a self-inflicted outage for
|
harness's own model backend, so a blind deploy is a self-inflicted outage for
|
||||||
possibly zero behavioral change.
|
possibly zero behavioral change.
|
||||||
|
|
||||||
|
|||||||
@@ -62,8 +62,8 @@ service on the other host.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
|
| Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
|
||||||
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
||||||
| `frank-large` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
||||||
| `frank-secondary` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
||||||
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
|
| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
|
||||||
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: Service Inventory
|
||||||
|
description: shared/services.yml is the one inventory of deployable services; scripts/service-map.py is its only reader. The planner, both deploy allow-lists, the changed-test seam mapper, the Pi-hole hosts render, the Prometheus scrape facts, and the homepage probes all derive from it.
|
||||||
|
tags: [deployment, inventory, planner, quadlet, prometheus, pihole, homepage]
|
||||||
|
timestamp: 2026-10-13T00:00:00-07:00
|
||||||
|
---
|
||||||
|
|
||||||
|
# Service Inventory
|
||||||
|
|
||||||
|
Before 2026-10-13 the same facts lived in four hand-maintained lists — the
|
||||||
|
planner's path rules (`deploy-plan.py`), the `--service` allow-list in
|
||||||
|
`deploy.sh`, the near-copy in `deploy-remote.sh`, and the changed-test mapper
|
||||||
|
(`test-changed.sh`) — plus per-fact copies in `prometheus.yml`,
|
||||||
|
`addn-hosts`, and `services.json`. They drifted on every swap (the planner
|
||||||
|
still mapped the retired `qwen-sharp` template to frank-dragoon; the allow-list
|
||||||
|
still listed the *arr stack under Dankserver after it moved to FrankBook-Pro).
|
||||||
|
Now there is **one inventory and one reader**:
|
||||||
|
|
||||||
|
- `shared/services.yml` — the data: every deployable service, its hosts, unit
|
||||||
|
dir(s), unit files, path claims, tests, scrape/DNS/probe facts, aliases.
|
||||||
|
- `scripts/service-map.py` — the only reader. CLI subcommands:
|
||||||
|
`hosts`, `services`, `allows`, `plan`, `seams`, `owner`, `units`, `stack`,
|
||||||
|
`get`, `render`, `check`.
|
||||||
|
|
||||||
|
## 1. Consumers
|
||||||
|
|
||||||
|
|Consumer|Call|
|
||||||
|
|---|---|
|
||||||
|
|`scripts/deploy-plan.py`|thin wrapper → `plan --host TARGET [paths...]` (keeps the `--repo/--base/--head` and `--paths` CLIs)|
|
||||||
|
|`scripts/deploy.sh` (`--service` gate)|`allows --host T --service S`|
|
||||||
|
|`scripts/deploy-remote.sh` (`service_is_allowed`)|same|
|
||||||
|
|`scripts/test-changed.sh`|`seams [paths...]` — a service's declared `tests` are its seams|
|
||||||
|
|`dankclaw/pihole/addn-hosts`|`render pihole-hosts` — the tracked file is a rendered artifact|
|
||||||
|
|`*/observability/quadlet/node-exporter.container`|`render node-exporter-unit --host H` — the MIRROR pair became one template; per-host title, LAN address (hosts map), and port (scrape job) fill it|
|
||||||
|
|Prometheus scrape jobs, LAN DNS, homepage probes|`check` compares the tracked configs against the inventory's `scrape`/`dns`/`probe` fields (drift gate)|
|
||||||
|
|
||||||
|
Selection rules the engine encodes (see `tests/fixtures/service-plan-golden.tsv`,
|
||||||
|
the pinned characterization of every tracked path × every target):
|
||||||
|
|
||||||
|
- **Host scoping** — a host's top-level directory can only select that host's
|
||||||
|
services. `frankbook-pro/**` plans nothing on Dankserver (the old planner
|
||||||
|
had a stale Dankserver *arr list).
|
||||||
|
- **Quadlet coupling graph** — `Network=x.container` rides x's netns;
|
||||||
|
`Network=y.network` joins a shared network. Changing the parent (or the
|
||||||
|
network file) selects every rider transitively, in their own directories
|
||||||
|
(`dankclaw/searxng/quadlet/gluetun.container` → gluetun + searxng + marreta;
|
||||||
|
`frankbook-pro/gluetun/quadlet/gluetun.container` → the whole arr stack).
|
||||||
|
- **Stacks** — a service's `stack` members deploy and restart as one handler
|
||||||
|
(romm trio, hermes trio).
|
||||||
|
- **Retirement** — removing a service from `services.yml` removes it from the
|
||||||
|
planner, both allow-lists, and the seam mapper in one data edit; a pending
|
||||||
|
deploy-state entry for it is auto-triaged (see [[Deploy State Triage]]).
|
||||||
|
|
||||||
|
## 2. Integrity gates (`check`)
|
||||||
|
|
||||||
|
`python3 scripts/service-map.py check` runs, and
|
||||||
|
`tests/service_inventory.bats` pins:
|
||||||
|
|
||||||
|
- **schema** — required fields per entry, known keys only;
|
||||||
|
- **units** — every unit file named by an entry exists under its `unit_dir`
|
||||||
|
on the right host; each unit name is claimed by exactly one service per host;
|
||||||
|
- **ownership** — every tracked quadlet `.container`/`.network` file is claimed
|
||||||
|
by exactly one service (trash excluded);
|
||||||
|
- **paths** — every claimed `files`/`dirs`/`retires` glob matches ≥1 tracked file;
|
||||||
|
- **compose** — the container names in each host's `compose.yaml` have exactly
|
||||||
|
one quadlet owner (the [[Quadlet Lifecycle]] two-owners trap);
|
||||||
|
- **drift** — `addn-hosts` equals its render, each tracked node-exporter unit
|
||||||
|
equals its per-host render; the Prometheus job set, the DNS names, and the
|
||||||
|
homepage probe ports equal the inventory's fields.
|
||||||
|
|
||||||
|
## 3. Adding, moving, retiring a service
|
||||||
|
|
||||||
|
- **Add**: append an entry (name, hosts, kind, unit_dir, units, paths, tests,
|
||||||
|
…), add the quadlet file, run `check` — `plan`/`allows`/`seams` pick it up
|
||||||
|
with zero script edits.
|
||||||
|
- **Move between hosts**: edit `hosts` (and per-host `unit_dir`/`probe` maps).
|
||||||
|
- **Retire**: delete the entry, move files to `.AGENTTRASH/` (see
|
||||||
|
[[Testing Policy]] for the seam the mapper applies to trash paths).
|
||||||
|
|
||||||
|
Field notes: `alias` carries deployable names that differ from the service
|
||||||
|
name (`calibre-web` deploys the unit `calibre-server`); `probe` for multi-host
|
||||||
|
services is a `{host: port}` map; `dependents` names services whose config
|
||||||
|
embeds a reference to this one (config-only coupling — a `.env` edit selects
|
||||||
|
both, a unit edit selects the graph closure).
|
||||||
|
|
||||||
|
## 4. Known cutover deltas (2026-10-13)
|
||||||
|
|
||||||
|
The 17 deltas against the old planner are all in these classes: anubis paths
|
||||||
|
now select `anubis` (the old planner had no rule at all); cross-host paths
|
||||||
|
select nothing; `scripts/qwen-sharp-v22.4.0.jinja` (retired template) plans
|
||||||
|
nothing while `scripts/qwen38-froggeric-v22.5.jinja` (the live frank-archon
|
||||||
|
template, previously unmapped) plans frank-archon;
|
||||||
|
`dankserver/romm/romm.container` (top-level quadlet dir the old unit-file rule
|
||||||
|
missed) now plans the romm trio.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Related: [[Deployment Flow]], [[Testing Policy]], [[Quadlet Lifecycle]],
|
||||||
|
[[Container Network Namespaces]], [[Deploy State Triage]]
|
||||||
@@ -15,10 +15,9 @@ This repository tests its configuration at defined seams. The tests run with `ba
|
|||||||
|
|
||||||
| Seam | File | What it asserts |
|
| Seam | File | What it asserts |
|
||||||
|------|------|-----------------|
|
|------|------|-----------------|
|
||||||
| Compose validity | `tests/compose_validity.bats` | YAML parses, services have images, no named volumes |
|
|
||||||
| Env consistency | `tests/env_consistency.bats` | every `${VAR}` is defined in `.env.example`; no real secrets; the live gitea password never appears |
|
| Env consistency | `tests/env_consistency.bats` | every `${VAR}` is defined in `.env.example`; no real secrets; the live gitea password never appears |
|
||||||
| Caddyfile validity | `tests/caddyfile.bats` | per-host Caddyfile, `.local` convention, balanced braces, basic auth present |
|
| Caddyfile validity | `tests/caddyfile.bats` | per-host Caddyfile, `.local` convention, balanced braces, basic auth present |
|
||||||
| Docs completeness | `tests/docs_completeness.bats` | every compose service has a docs page; index links every page |
|
| Docs completeness | `tests/docs_completeness.bats` | every service has a docs page; index links every page; no unresolved vault links |
|
||||||
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
||||||
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
|
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
|
||||||
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
|
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
|
||||||
|
|||||||
@@ -95,7 +95,7 @@ host-side `workflows/` dir and the repo copy.
|
|||||||
## Recommended additions
|
## Recommended additions
|
||||||
|
|
||||||
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
|
Quantized preferred, sized for the 8060S iGPU (GTT-backed; the RTX 3090 is
|
||||||
frank-secondary's). All repos verified 2026-10-05. None are on disk yet.
|
frank-zealot's). All repos verified 2026-10-05. None are on disk yet.
|
||||||
|
|
||||||
| # | What | Repo | Grab | Size | Why |
|
| # | What | Repo | Grab | Size | Why |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ timestamp: 2026-08-26
|
|||||||
# Dankclaw Stack
|
# Dankclaw Stack
|
||||||
|
|
||||||
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
|
Dankclaw (192.168.0.143) runs the AI, search, development, and observability
|
||||||
stack. Its LLM inference endpoint `frank-large` (`:7770`) is a ROCmFPX
|
stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
|
||||||
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
|
llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
|
||||||
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
|
the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
|
||||||
FLM v1.0.1) serves sight+sound models. The completion endpoint
|
FLM v1.0.1) serves sight+sound models. The completion endpoint
|
||||||
@@ -30,7 +30,7 @@ graph LR
|
|||||||
S --> V[Gluetun VPN]
|
S --> V[Gluetun VPN]
|
||||||
O --> L[llama.cpp dankserver :7770]
|
O --> L[llama.cpp dankserver :7770]
|
||||||
O --> H[Hermes]
|
O --> H[Hermes]
|
||||||
O --> E[frank-large :7770]
|
O --> E[frank-dragoon :7770]
|
||||||
```
|
```
|
||||||
|
|
||||||
## Services
|
## Services
|
||||||
@@ -48,14 +48,14 @@ graph LR
|
|||||||
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
|
| Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
|
||||||
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
|
| Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
|
||||||
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
|
| Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
|
||||||
| frank-large (host service) | — | `7770` | [[deployment]] |
|
| frank-dragoon (host service) | — | `7770` | [[deployment]] |
|
||||||
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
|
| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
|
||||||
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
|
| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
|
||||||
|
|
||||||
The LLM rows are bare-metal systemd services, not containers. This
|
The LLM rows are bare-metal systemd services, not containers. This
|
||||||
repo owns their scripts, units, and knowledge base (see
|
repo owns their scripts, units, and knowledge base (see
|
||||||
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
|
[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
|
||||||
`frank-large` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
|
`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
|
||||||
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
|
on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
|
||||||
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
|
`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
|
||||||
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
|
`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ is not durable. The tracked, sanitized configuration baseline is
|
|||||||
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
|
`0.0.0.0:8080`, so the Signal HTTP API is reachable from the whole LAN.
|
||||||
- The data volume persists skills, memory, and session history.
|
- The data volume persists skills, memory, and session history.
|
||||||
- Hermes has two named local providers: `dankclaw-large` reaches
|
- Hermes has two named local providers: `dankclaw-large` reaches
|
||||||
`http://192.168.0.143:7770/v1` and exposes `frank-large`;
|
`http://192.168.0.143:7770/v1` and exposes `frank-dragoon`;
|
||||||
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
|
`dankserver-small` reaches `http://192.168.0.235:7770/v1` and exposes
|
||||||
`frank-medium` (renamed from `frank-small` on 2026-09-15).
|
`frank-medium` (renamed from `frank-small` on 2026-09-15).
|
||||||
- The live `config.yaml` contains secrets and user state and is not committed;
|
- The live `config.yaml` contains secrets and user state and is not committed;
|
||||||
@@ -64,7 +64,7 @@ is not durable. The tracked, sanitized configuration baseline is
|
|||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
- Hermes is under active development; pin the image tag for reproducibility.
|
- Hermes is under active development; pin the image tag for reproducibility.
|
||||||
- Model choice drives quality; the local `frank-large` (Dankclaw, 7770) is the intended backend.
|
- Model choice drives quality; the local `frank-dragoon` (Dankclaw, 7770) is the intended backend.
|
||||||
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
|
- Do not expose the gateway without auth — put it behind [[Caddy]] with basic auth.
|
||||||
|
|
||||||
## Browser automation (Chrome DevTools Protocol daemon)
|
## Browser automation (Chrome DevTools Protocol daemon)
|
||||||
@@ -189,7 +189,7 @@ What actually breaks, and the fix:
|
|||||||
auxiliary:
|
auxiliary:
|
||||||
title_generation:
|
title_generation:
|
||||||
provider: dankserver-small # names track the live config; diff first
|
provider: dankserver-small # names track the live config; diff first
|
||||||
model: frank-secondary
|
model: frank-zealot
|
||||||
api_mode: codex_responses # real per-task key
|
api_mode: codex_responses # real per-task key
|
||||||
extra_body:
|
extra_body:
|
||||||
reasoning:
|
reasoning:
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ https://github.com/open-webui/open-webui
|
|||||||
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
|
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
|
||||||
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
|
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
|
||||||
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
|
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
|
||||||
| Backends | Picker exposes exactly one model — `frank-secondary` (host.containers.internal:7771). The `frank-large` and `frank-primary` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
|
| Backends | Picker exposes exactly one model — `frank-zealot` (host.containers.internal:7771). The `frank-dragoon` and `frank-archon` connections remain configured in the DB, disabled (2026-10-07) — see [The LLM backends](#the-llm-backends) |
|
||||||
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
|
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
|
||||||
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
|
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
|
||||||
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
|
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
|
||||||
@@ -34,15 +34,15 @@ https://github.com/open-webui/open-webui
|
|||||||
The picker is built from the `model` table in `webui.db` (one row per model)
|
The picker is built from the `model` table in `webui.db` (one row per model)
|
||||||
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
|
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
|
||||||
connection discovery — so the picker shows exactly what is seeded there.
|
connection discovery — so the picker shows exactly what is seeded there.
|
||||||
As of 2026-10-07 the picker exposes **exactly one model: `frank-secondary`**.
|
As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`**.
|
||||||
|
|
||||||
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-large` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
|
||||||
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-primary` | radiance TP=2 (2× R9700) | Dankserver |
|
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
|
||||||
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-secondary` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
||||||
|
|
||||||
The `model` table carries exactly one decoration row, `frank-secondary`
|
The `model` table carries exactly one decoration row, `frank-zealot`
|
||||||
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
|
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
|
||||||
Q6_K + BF16 vision mmproj).
|
Q6_K + BF16 vision mmproj).
|
||||||
|
|
||||||
@@ -62,7 +62,7 @@ Two gotchas make the wiring non-obvious:
|
|||||||
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
|
- **llama.cpp `/v1/models` is Ollama-shaped.** llama.cpp returns
|
||||||
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
|
`{"models":[{"name":...}]}`; OpenWebUI's `extract_data()` only reads the
|
||||||
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
|
OpenAI `{"data":[{"id":...}]}` shape, so connection discovery silently misses
|
||||||
the llama.cpp endpoints (frank-large, frank-secondary). The fix is to set
|
the llama.cpp endpoints (frank-dragoon, frank-zealot). The fix is to set
|
||||||
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
|
`model_ids` in each connection's `api_config` — when non-empty, OpenWebUI
|
||||||
builds the model list directly and never fetches `/v1/models`. frank-medium
|
builds the model list directly and never fetches `/v1/models`. frank-medium
|
||||||
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
|
(vLLM) serves the OpenAI shape and would discover fine, but uses `model_ids`
|
||||||
@@ -116,7 +116,7 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
|||||||
models. Response key is `items`, not `data`.
|
models. Response key is `items`, not `data`.
|
||||||
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
|
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
|
||||||
with exactly one word: pong") → each returns content `pong`. Reasoning
|
with exactly one word: pong") → each returns content `pong`. Reasoning
|
||||||
models need a generous budget: frank-secondary (Qwen3.5 template) returns
|
models need a generous budget: frank-zealot (Qwen3.5 template) returns
|
||||||
empty content + `finish_reason: "length"` at `max_tokens 200` (the
|
empty content + `finish_reason: "length"` at `max_tokens 200` (the
|
||||||
thinking block eats it) and answers `pong` at 1500.
|
thinking block eats it) and answers `pong` at 1500.
|
||||||
3. A 500 "Server Connection Error" → `podman logs open-webui` for
|
3. A 500 "Server Connection Error" → `podman logs open-webui` for
|
||||||
@@ -191,8 +191,8 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
|
|||||||
run with host `python3` over ssh against
|
run with host `python3` over ssh against
|
||||||
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
|
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
|
||||||
Dankclaw; table is `user`, singular; no `active` column). No per-user model
|
Dankclaw; table is `user`, singular; no `active` column). No per-user model
|
||||||
grant is needed: the picker model `frank-secondary` carries the global
|
grant is needed: the picker model `frank-zealot` carries the global
|
||||||
`access_grant` row `(model, frank-secondary, user, *, read)`, so any
|
`access_grant` row `(model, frank-zealot, user, *, read)`, so any
|
||||||
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
|
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
|
||||||
not exist until the account signs in (2026-10-09: five provisioned accounts
|
not exist until the account signs in (2026-10-09: five provisioned accounts
|
||||||
had no row).
|
had no row).
|
||||||
@@ -271,18 +271,18 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
|
|||||||
|
|
||||||
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
|
- **2026-08-22 — slirp4netns → pinned bridge.** The unit ran on `Network=slirp4netns`. The slirp4netns helper process died (no OOM, no kill entry, no crash log found), and the container kept running with only `lo` in its netns: published port 3005 accepted TCP and reset every request, so chat served empty responses for ~2 days. Nothing logged an error. Moving to a kernel bridge removes the per-container helper entirely, and the healthcheck's interface check makes the zombie state visible if it ever returns.
|
||||||
|
|
||||||
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-large`/`frank-medium`/`frank-secondary` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
|
- **2026-09-21 — model picker rebuilt for the 3-model fleet.** The picker carried 39 stale rows (37 dead Ollama models + 2 `frank-default-*`) and was missing `frank-medium`. Rebuilt the `model` table to exactly `frank-dragoon`/`frank-medium`/`frank-zealot` and set `model_ids` on each connection (llama.cpp's Ollama-shaped `/v1/models` is invisible to OpenWebUI's discovery). Pointed the two on-host endpoints at `host.containers.internal` — the bridge refuses the host LAN IP for non-published ports. All three verified end-to-end via `/openai/chat/completions`.
|
||||||
|
|
||||||
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
|
- **2026-09-24 — SSO via authentik.** OIDC authorization-code flow wired per the [SSO (authentik)](#sso-authentik) section: application/provider/entitlements created in authentik, client secret in the host env file, roles from the `roles` scope claim. E2E coverage in `tests/e2e/specs/18-openwebui-sso.spec.ts`.
|
||||||
|
|
||||||
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
|
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
|
||||||
|
|
||||||
- **2026-10-07 — picker narrowed to `frank-secondary` only.** Per Frank: the only model accessible in OpenWebUI is `frank-secondary`. Rebuilt the `model` table to a single `frank-secondary` row (the old rows `frank-large`/`frank-primary`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-large) and 1 (frank-primary) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-secondary` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-secondary"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
|
- **2026-10-07 — picker narrowed to `frank-zealot` only.** Per Frank: the only model accessible in OpenWebUI is `frank-zealot`. Rebuilt the `model` table to a single `frank-zealot` row (the old rows `frank-dragoon`/`frank-archon`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-dragoon) and 1 (frank-archon) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-zealot` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-zealot"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
|
||||||
|
|
||||||
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
|
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
|
||||||
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
||||||
id-keyed objects had to move in the same window as the alias flip:
|
id-keyed objects had to move in the same window as the alias flip:
|
||||||
`model.id`+`name` (`frank-secondary` → `frank-zealot`, `params`
|
`model.id`+`name` (`frank-zealot` → `frank-zealot`, `params`
|
||||||
`{"context":{"length":200704}}` and the `meta.description` kept),
|
`{"context":{"length":200704}}` and the `meta.description` kept),
|
||||||
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
||||||
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
||||||
|
|||||||
Loaded 100 of 237 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user