dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

Browse Source 
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
This commit is contained in:
Frank
2026-10-10 17:03:14 -07:00
parent 5ced474eb9
commit 55f87fcdce
195 changed files with 2085 additions and 2064 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/halogen-flash-server.md
+23 -23
View File
@@ -1,12 +1,12 @@
---
type: reference
title: halogen-flash-server on Dankclaw (frank-large)
description: The active frank-large runbook — peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b) on the Dankclaw gfx1151 box: container shape, deployed flags, the memory model, the client contract, and the failure modes worth knowing.
title: halogen-flash-server on Dankclaw (frank-dragoon)
description: The active frank-dragoon runbook — peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b) on the Dankclaw gfx1151 box: container shape, deployed flags, the memory model, the client contract, and the failure modes worth knowing.
tags:
- llm
- halogen
- dankclaw
- frank-large
- frank-dragoon
- strix-halo
- gfx1151
- qwen4exp
@@ -14,16 +14,16 @@ tags:
timestamp: 2026-09-13
---
# halogen-flash-server on Dankclaw (frank-large)
# halogen-flash-server on Dankclaw (frank-dragoon)
> **ACTIVE again since 2026-09-16, and retuned.** It replaced the llama.cpp /
> kingjones30-ROCmFPX STRIX deployment on 2026-09-11 (see
> [[kingjones-strix-rocmfp4]]), was displaced for two days by ik_llama.cpp (see
> [[frank-large-ik-uncensored]], the current rollback), and came back as the
> [[frank-dragoon-ik-uncensored]], the current rollback), and came back as the
> shape the fleet asked for: two 200k-token conversations rather than the
> native-context pair. Deployed as a rootless Quadlet container,
> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770. The RTX 3090
> it does not touch now belongs to [[frank-secondary-qwen38-gsq-rco]].
> `dankclaw/frank-dragoon/quadlet/frank-dragoon.container`, port 7770. The RTX 3090
> it does not touch now belongs to [[frank-zealot-qwen38-gsq-rco]].
The engine is a from-scratch serving runtime for the Qwen3.8-Flash-Next
architecture on AMD's gfx1151 (Strix Halo) — the same silicon this host runs.
@@ -45,7 +45,7 @@ The `all` mode is upstream's default and the shape a single-container host
wants: the front-end talks to the engine over loopback inside the container, so
there is one process tree, one log, and one published port.
`HALOGEN_MODEL_ID=frank-large` sets the id reported at `/v1/models`, in
`HALOGEN_MODEL_ID=frank-dragoon` sets the id reported at `/v1/models`, in
`/health`, and in every response body. Requests are **never rejected for naming
a different model** — the server serves whatever asks. Keeping the fleet alias
is therefore free, and it means Hermes, the harness config, the OpenWebUI
@@ -94,13 +94,13 @@ upstream's defaults:
| flag | value | why |
|---|---|---|
| `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
| `HALOGEN_CTX` | `200192` | **set 2026-09-16**: exactly half the pool, so two conversations of the full window are always resident. Not a round 200,000 — pool/2 is. The model's native window is 262,144, so this is a real reduction in the per-conversation ceiling, and a client asking for more gets a 400 (never a silent truncation). The harness declares 200192 for `frank-large` |
| `HALOGEN_MODEL_ID` | `frank-dragoon` | fleet identity; no client rename |
| `HALOGEN_CTX` | `200192` | **set 2026-09-16**: exactly half the pool, so two conversations of the full window are always resident. Not a round 200,000 — pool/2 is. The model's native window is 262,144, so this is a real reduction in the per-conversation ceiling, and a client asking for more gets a 400 (never a silent truncation). The harness declares 200192 for `frank-dragoon` |
| `HALOGEN_KV_POOL_POSITIONS` | `400384` | **retuned 2026-09-16** (from 401,408): 2 x 200,192. Reserves ~11.0 GiB of KV, confirmed by the boot ledger as `400384 positions, ~27.2 GiB (16.2 GiB of it for everything but the pool, at max_tok 24576)` against a 33.5 GiB device budget — **no clamp**. Slots still oversubscribe the pool for any real request (admission reserves prompt + `max_tokens`), so a near-full pair that is both generating is pool-bound. `HALOGEN_KV_POOL_FIT` guards the boot: read the ledger, not this column |
| `HALOGEN_KV_SLOTS` | `2` | **lowered from 3 on 2026-09-16** — one slot per half of the pool, and two is the fleet's shape now that Dankclaw serves two endpoints. Each slot is 111.0 MiB of O(1) state, so this hands ~111 MiB back as well |
| `HALOGEN_MAX_TOK` | `24576` | **halved from the 32768 default on 2026-09-16** for frank-secondary, its co-tenant on the same host. The arena's variable term goes `8.40 x 2 → 8.40 x 1.5`, handing ~4.2 GiB back at roughly a 4-5 % prefill cost — the same lever at 16,384 costs 8-9 %. The vendor's own notes call 24,576 the largest arena that allocated with headroom to spare, so it stays inside their tested range. 65536 still cannot boot. It sizes a per-call PREFILL CHUNK, not the prompt ceiling: a 200k conversation is prefilled in arena-sized pieces |
| `HALOGEN_MAX_TOK` | `24576` | **halved from the 32768 default on 2026-09-16** for frank-zealot, its co-tenant on the same host. The arena's variable term goes `8.40 x 2 → 8.40 x 1.5`, handing ~4.2 GiB back at roughly a 4-5 % prefill cost — the same lever at 16,384 costs 8-9 %. The vendor's own notes call 24,576 the largest arena that allocated with headroom to spare, so it stays inside their tested range. 65536 still cannot boot. It sizes a per-call PREFILL CHUNK, not the prompt ceiling: a 200k conversation is prefilled in arena-sized pieces |
| `HALOGEN_HOST_RESERVE_GIB` | `20` | **raised from 15 with the same retune**, because the arena shrank: `budget = 121.2 − 67.7 − 20 = 33.5 GiB` against a measured need of ~27.2, so the fit check kept ~6 GiB of margin and did not clamp. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away (see the ceiling section below) |
| `HALOGEN_VISION_TOWER` | *unset* | **removed 2026-09-16.** The tower charges 1.97 GiB against the same device budget as the pool and the arena, and the fleet's image path moved to [[frank-secondary-qwen38-gsq-rco]] (llama.cpp with `--mmproj`). This server refuses image content parts, and `models.yml` declares `frank-large` text-only to match |
| `HALOGEN_VISION_TOWER` | *unset* | **removed 2026-09-16.** The tower charges 1.97 GiB against the same device budget as the pool and the arena, and the fleet's image path moved to [[frank-zealot-qwen38-gsq-rco]] (llama.cpp with `--mmproj`). This server refuses image content parts, and `models.yml` declares `frank-dragoon` text-only to match |
| `HALOGEN_DOWNLOAD` | `peonist-ai/halogen-qwen3.8-flash-next` | first-start fetch |
| `HALOGEN_VERBOSE` | `1` | verbose engine logging — the fullest startup account, on permanently so an incident report needs no restart-into-debug |
@@ -226,7 +226,7 @@ restart.
That is the failure mode to watch, and it is the least diagnosable of the
three: not a silent clamp, not a boot refusal, but a minutes-long freeze that
looks like a hang, for the neighbours as much as for frank-large. `free` is
looks like a hang, for the neighbours as much as for frank-dragoon. `free` is
useless for spotting it — `MemAvailable` still reads ~74 GiB because the
page cache counts as reclaimable. Watch the `host memory left for everything
else` line and the contiguous-block count at boot instead.
@@ -374,7 +374,7 @@ tables above.
**None here since 2026-09-16.** `HALOGEN_VISION_TOWER` is deliberately unset:
the tower is a 1.97 GiB charge against the same device budget as the pool and
the arena, and the fleet's image path moved to [[frank-secondary-qwen38-gsq-rco]],
the arena, and the fleet's image path moved to [[frank-zealot-qwen38-gsq-rco]],
which loads the BF16 projector alongside its own weights. This server refuses
image content parts and says why, rather than ignoring them:
@@ -393,11 +393,11 @@ anywhere in the fleet — that left with [[frank-npu]].
## Deploy and operate
```bash
./scripts/deploy.sh dankclaw --service frank-large
./scripts/deploy.sh dankclaw --service frank-dragoon
```
The handler retires the ik-era **root** unit first (stop, disable, move
`/etc/systemd/system/frank-large.service` into
`/etc/systemd/system/frank-dragoon.service` into
`~/.AGENTTRASH/20260916-halogen-restore/host-units` — never `rm`; this host's
`disable` may delete the file outright, which the handler now reports), stops
the Quadlet's own unit if it is running, then pulls
@@ -419,8 +419,8 @@ file plus linger on the deploy user — generated units **cannot** be
`systemctl --user enable`d on this host.
```bash
systemctl --user status frank-large.service # the generated unit
podman logs frank-large | head -40 # version banner + memory ledger
systemctl --user status frank-dragoon.service # the generated unit
podman logs frank-dragoon | head -40 # version banner + memory ledger
curl -s http://127.0.0.1:7770/health # 200 only after the engine PONGs
curl -s http://127.0.0.1:7770/v1/models
```
@@ -447,13 +447,13 @@ steady-state noise on a shared host — raise it only for a live OOM report.
The lines worth grepping first:
```bash
podman logs frank-large 2>&1 | grep -E '^(dmalloc|kv pool):'
podman logs frank-dragoon 2>&1 | grep -E '^(dmalloc|kv pool):'
```
Retrieval: `podman logs frank-large` covers the **current** instance only —
Retrieval: `podman logs frank-dragoon` covers the **current** instance only —
the Quadlet runs `--replace --rm`, so everything before the last recreate is
gone from the container log. `journalctl --user -u frank-large.service` is
the durable record. Both are rootless: `sudo podman logs frank-large` looks
gone from the container log. `journalctl --user -u frank-dragoon.service` is
the durable record. Both are rootless: `sudo podman logs frank-dragoon` looks
in root's storage and answers "no such container".
## Rollback
@@ -466,7 +466,7 @@ serve script and the Qwen-Sharp template are archived in the repo under
is a `git revert` of this change set plus a deploy — **but the Quadlet must be
removed from `~/.config/containers/systemd/` first**, or the restarted root
unit and the still-installed container both claim port 7770. The port gate in
the handler now covers exactly that race. See [[frank-large-ik-uncensored]].
the handler now covers exactly that race. See [[frank-dragoon-ik-uncensored]].
The `amd_iommu=off` kernel argument that came with this change is separate and
reverts on its own — see [[dankclaw-iommu-translation-off]].