serverAppConfigs/serverAppConfigs-docs/LLM/halogen-flash-server.md

---
type: reference
title: halogen-flash-server on Dankclaw (frank-large)
description: The active frank-large runbook — peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b) on the Dankclaw gfx1151 box: container shape, deployed flags, the memory model, the client contract, and the failure modes worth knowing.
tags:
  - llm
  - halogen
  - dankclaw
  - frank-large
  - strix-halo
  - gfx1151
  - qwen4exp
  - rocm
timestamp: 2026-09-13
---

# halogen-flash-server on Dankclaw (frank-large)

> **ACTIVE again since 2026-09-16, and retuned.** It replaced the llama.cpp /
> kingjones30-ROCmFPX STRIX deployment on 2026-09-11 (see
> [[kingjones-strix-rocmfp4]]), was displaced for two days by ik_llama.cpp (see
> [[frank-large-ik-uncensored]], the current rollback), and came back as the
> shape the fleet asked for: two 200k-token conversations rather than the
> native-context pair. Deployed as a rootless Quadlet container,
> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770. The RTX 3090
> it does not touch now belongs to [[frank-small-qwen38-gsq-rco]].

The engine is a from-scratch serving runtime for the Qwen3.8-Flash-Next
architecture on AMD's gfx1151 (Strix Halo) — the same silicon this host runs.
Upstream's own comparison table measures it at **roughly 4x the prefill
throughput** of the llama.cpp-family runtimes it ships against (EngramHalo.cpp,
ROCmFP4, CIRU-IU4) on a 32K prompt, and prefill is most of the wall clock on any
prompt with real context. That, not decode, is the reason for the swap.

## Shape

One container runs both halves of the stack:

| half | port | reachable |
|---|---|---|
| engine | 8730 | container loopback only — **unauthenticated, never publish it** |
| OpenAI front-end | 8731 | published as host **7770** |

The `all` mode is upstream's default and the shape a single-container host
wants: the front-end talks to the engine over loopback inside the container, so
there is one process tree, one log, and one published port.

`HALOGEN_MODEL_ID=frank-large` sets the id reported at `/v1/models`, in
`/health`, and in every response body. Requests are **never rejected for naming
a different model** — the server serves whatever asks. Keeping the fleet alias
is therefore free, and it means Hermes, the harness config, the OpenWebUI
discovery, the `services.json` probe and the docs needed no rename.

## Weights

`peonist-ai/halogen-qwen3.8-flash-next` (HuggingFace, public and ungated):

| file | size | role |
|---|---|---|
| `qwen38-flash-next-w4b.hgn` | 115.55 GiB | the checkpoint |
| `qwen38-flash-next-w4b.overlay.hgn` | 2.40 GiB | **quality sidecar** — picked up automatically; the engine warns at startup if it is missing. **Refreshed 2026-09-16**: the sidecars are versioned *in place*, so a stale local copy is invisible from its file name — this one had read 2.31 GiB since it was first fetched while the Hub had moved on |
| `qwen38-flash-next-vision.hgn` | 0.84 GiB | vision tower (see below) |
| `tokenizer/` | ~21 MB | flat tokenizer dir; found beside the checkpoint, so no extra mount |

`HALOGEN_DOWNLOAD` fetches the whole repo into the bind mount on **first start
only** — the entrypoint acts only when the checkpoint is absent, so a restart
never re-downloads, and `huggingface_hub` resumes an interrupted pull. The
mount must be read-write for this. Roughly 127 GB; on Dankclaw it lands in
`/home/dank/models/halogen`.

**Refreshing the weights.** Delete the mount's contents (or the whole
directory, then recreate it **empty**) and start the unit: the entrypoint sees
the checkpoint absent and re-fetches the repo, resuming if interrupted. Two
traps. First, the bind mount's source directory must *exist* — podman refuses
the container with `Error: statfs /home/dank/models/halogen: no such file or
directory` and the unit loops on `Restart=on-failure`; `mkdir -p` the empty
directory and the fetch starts on the next start. Second, "refresh" only means
"fetch what the Hub has now": on 2026-09-16 the re-fetch brought newer sidecars
(`overlay.hgn` 2,477,677,120 → **2,572,466,560** bytes, `overlay-speed.hgn`
2,383,306,048 → **2,478,095,488**) while the checkpoint returned
byte-identical in size (124,068,083,904) — so an update can live entirely in
the sidecars, and equal file sizes are not proof of an unchanged repo.

`overlay-speed.hgn` (2.31 GiB on the Hub; 2.22 GiB before the 2026-09-16
refresh) comes along with the repo but is unused: it trades the `o_proj`
calibration for about 2 % of decode, and the quality sidecar is the default for
a reason.

## Deployed flags

Everything else is configured by environment variable — there is no config
file, and the flags are read once at startup. These are the departures from
upstream's defaults:

| flag | value | why |
|---|---|---|
| `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
| `HALOGEN_CTX` | `200192` | **set 2026-09-16**: exactly half the pool, so two conversations of the full window are always resident. Not a round 200,000 — pool/2 is. The model's native window is 262,144, so this is a real reduction in the per-conversation ceiling, and a client asking for more gets a 400 (never a silent truncation). The harness declares 200192 for `frank-large` |
| `HALOGEN_KV_POOL_POSITIONS` | `400384` | **retuned 2026-09-16** (from 401,408): 2 x 200,192. Reserves ~11.0 GiB of KV, confirmed by the boot ledger as `400384 positions, ~27.2 GiB (16.2 GiB of it for everything but the pool, at max_tok 24576)` against a 33.5 GiB device budget — **no clamp**. Slots still oversubscribe the pool for any real request (admission reserves prompt + `max_tokens`), so a near-full pair that is both generating is pool-bound. `HALOGEN_KV_POOL_FIT` guards the boot: read the ledger, not this column |
| `HALOGEN_KV_SLOTS` | `2` | **lowered from 3 on 2026-09-16** — one slot per half of the pool, and two is the fleet's shape now that Dankclaw serves two endpoints. Each slot is 111.0 MiB of O(1) state, so this hands ~111 MiB back as well |
| `HALOGEN_MAX_TOK` | `24576` | **halved from the 32768 default on 2026-09-16** for frank-small, its co-tenant on the same host. The arena's variable term goes `8.40 x 2 → 8.40 x 1.5`, handing ~4.2 GiB back at roughly a 4-5 % prefill cost — the same lever at 16,384 costs 8-9 %. The vendor's own notes call 24,576 the largest arena that allocated with headroom to spare, so it stays inside their tested range. 65536 still cannot boot. It sizes a per-call PREFILL CHUNK, not the prompt ceiling: a 200k conversation is prefilled in arena-sized pieces |
| `HALOGEN_HOST_RESERVE_GIB` | `20` | **raised from 15 with the same retune**, because the arena shrank: `budget = 121.2 − 67.7 − 20 = 33.5 GiB` against a measured need of ~27.2, so the fit check kept ~6 GiB of margin and did not clamp. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away (see the ceiling section below) |
| `HALOGEN_VISION_TOWER` | *unset* | **removed 2026-09-16.** The tower charges 1.97 GiB against the same device budget as the pool and the arena, and the fleet's image path moved to [[frank-small-qwen38-gsq-rco]] (llama.cpp with `--mmproj`). This server refuses image content parts, and `models.yml` declares `frank-large` text-only to match |
| `HALOGEN_DOWNLOAD` | `peonist-ai/halogen-qwen3.8-flash-next` | first-start fetch |
| `HALOGEN_VERBOSE` | `1` | verbose engine logging — the fullest startup account, on permanently so an incident report needs no restart-into-debug |

Everything else is upstream's default, deliberately: prompt cache mode `2`
(two entries per conversation — end-of-system-prompt and end-of-history — which
already covers the shared-prefix case), `HALOGEN_CACHE_INPLACE=1` (the snapshot
stays ~115 MiB at any depth), the speculative drafter `1` (byte-identical at
temperature 0, and it stands down whenever a second conversation is
generating), the quality sidecar, `HALOGEN_ADMIT_CHUNK=0` (a non-zero value
gives up the byte-identity property), and the default `HALOGEN_MAX_TOKENS_CAP=65536`.

## Memory model

Three numbers explain the whole box:

- **~68 GiB pinned weights.** The checkpoint is mapped and registered with the
  GPU in place, never copied. The kernel counts the locked pages as reclaimable
  file cache, so `free`, `MemAvailable` and every dashboard built on them
  **overstate free memory by the size of the model**. The server prints the
  corrected figure at startup; believe that line, not Grafana.
- **~28 KiB per KV position** (measured: 262,144 positions = 6,656 MiB of KV,
  ~7.2 GiB with block scratch), plus the prefill arena. The 2026-09-16 ledger
  reads the two together in the engine's own words: `400384 positions, ~27.2
  GiB (16.2 GiB of it for everything but the pool, at max_tok 24576)` — so
  ~11.0 GiB of pool and 16.2 GiB of arena, working slack and (no longer) a
  vision tower. The engine's device-side budget stops at roughly 47 GiB on a
  128 GB machine *with the weights pinned* — and lower on a machine carrying
  more besides this server, which Dankclaw is.
- **`HALOGEN_HOST_RESERVE_GIB`** is what is left for the page cache the lookup
  table is read through. Too little and long prompts crawl with the disk busy
  and the process in uninterruptible sleep — that is a cache problem, not a
  memory problem, and the fix is a *smaller* pool, not a bigger one.

`HALOGEN_KV_POOL_FIT=1` (default) measures that budget at startup and lowers
the pool itself if the configured one will not fit, printing what it chose.
Read those lines on every restart; they are the ground truth for sizing.

**Measured ledger, seven restarts, read live 2026-09-12.** The engine's own
startup account, including three deliberate experiments:

| pool | max_tok | reserve | KV | arena | need | budget | verdict |
|---|---|---|---|---|---|---|---|
| 262144 (baseline) | 32768 | 24 | 7.2 | 20.6 | 29.8 | 29.5 | boots 0.3 over — prints **no** sizing lines |
| 524288 | 16384 | 24 | 14.4 | 12.2 | 28.6 | 29.5 | fits, 0.9 spare |
| 524288 | 32768 | 24 | 14.4 | 20.6 | 37.0 | 29.5 | clamped to 262144, deploy still green |
| 524288 | 65536 | 24 | — | 37.3 | 46.5 | 29.5 | **OOM — will not boot** |
| 524288 | 32768 | **15** | 14.4 | 20.6 | 37.0 | **38.5** | **deployed — both features** |
| 524288 | 32768 | 16 | 14.4 | 20.6 | 37.0 | 37.5 | fits, footprint unchanged — see the ceiling below |
| **400384** | **24576** | **20** | **11.0** | **16.4** | **27.2** | **33.5** | **deployed 2026-09-16 — ledger need 27.2, no clamp** |
| 401408 | 32768 | 15 | 11.0 | 20.6 | 33.4 | 38.5 | 2026-09-13 → 09-16, superseded by the retune |

### The sizing rule

One budget covers the pool, the arena **and** the vision tower together:

```
device budget = host RAM - resident weights - HALOGEN_HOST_RESERVE_GIB
              = 121.2    - 67.7             - reserve
                                    reserve 24 -> 29.5 GiB
                                    reserve 15 -> 38.5 GiB

need = KV (28 KiB/position) + arena + vision (1.97 measured)
arena = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384)
```

The `121` the engine prints is rounded **down** from `MemTotal 127104060 kB =
121.2159 GiB`, so subtracting the displayed figures gives 29.3 rather than 29.5.
Use the unrounded value. Two of the three terms are fixed hardware and
checkpoint facts; `HALOGEN_HOST_RESERVE_GIB` is the only one you own — which
buys the *fit margin*, not the host's memory. It is a term in this inequality,
not an allocation, and the section at the end of this page shows the live test
that proves it.

Trust the engine's printed requirement over this model wherever they disagree:
a boot clamped at reserve 24 stated `524288 positions need ~35.0 GiB (plus 1.5
of margin)`, i.e. **budget >= 36.5**, against a modelled 37.0 here. Close
enough to plan with, and the authoritative number to cut the other way.

The arena term is **linear in `HALOGEN_MAX_TOK`** and predicts every measured
point to 0.1 GiB — including upstream's three published "on the device" figures
(524,288 → ~35, 786,432 → ~42, 1,048,576 at `max_tok=16384` → ~41), which are
**pool + arena totals at the default `max_tok`**, not pool alone. That is why
their largest pool "needs `HALOGEN_MAX_TOK=16384`": at 32768 a 1M pool would be
49.4. Reading those figures as pool-only is the trap that makes a doubling look
affordable when it is not.

At the baseline reserve of 24 the doubled pool leaves `29.5 - 1.97 - 14.4 =
13.1` for the arena, so `max_tok <= 18,130` and **16384 is that reserve's
ceiling, not a conservative setting.** The corner check agrees: at 32768 only
7.0 GiB is left for the KV, ~254k positions, and the pool drops back toward
262144. Cut the reserve to 15 and the same arithmetic gives `38.5 - 1.97 - 14.4
= 22.1` for the arena — `max_tok <= 35,700`, which is exactly why 32768 now
fits unmodified.

### What the reserve actually costs

The device budget is not the whole story. Memory position by configuration
(401,408 is the 2026-09-13 deployed row, read from its boot ledger):

| config | held | host left | contiguous 2 MiB blocks |
|---|---|---|---|
| 262144 / 32768, reserve 24 | 96.3 | 12.1 GiB | 2081 |
| 524288 / 16384, reserve 24 | 94.6 | 13.8 GiB | 3742 |
| 524288 / 32768, reserve **16** | **103.5** | **7.1 GiB** | 293 |
| **401408 / 32768, reserve 15** | **100.0** | **9.1 GiB** | **1396** |
| **400384 / 24576, reserve 20** | **95.4** | **14.7 GiB** | **475** |

At the doubled setting (2026-09-12) the server held 103.5 of 121 GiB and the
engine volunteered the stall note. It did not print it at the 401,408 trim, and
it prints it again after the 2026-09-16 retune. Three data points, and they do
not line up with a "free memory below X" reading: 310 blocks printed, 1396 did
not, 475 did. So treat the note as "the machine is tight" and read the
`host memory left` line rather than waiting to be told:

```
halogen: NOTE this server now holds most of the machine, and 310 contiguous
2 MiB blocks remain. It runs best on a host of its own. If other large
processes share this machine they will compete for what is left, and when it
runs out BOTH they and this server can stall for minutes at a time at full
CPU with no disk activity and no output, which is not a crash and needs no
restart.
```

That is the failure mode to watch, and it is the least diagnosable of the
three: not a silent clamp, not a boot refusal, but a minutes-long freeze that
looks like a hang, for the neighbours as much as for frank-large. `free` is
useless for spotting it — `MemAvailable` still reads ~74 GiB because the
page cache counts as reclaimable. Watch the `host memory left for everything
else` line and the contiguous-block count at boot instead.

28 co-tenant containers (~4.3 GiB working set) stayed up through the change,
so it is survivable.

### The reserve cannot buy air — ceiling 16

The last row above is the experiment that proves it. Raising the reserve from
15 to 16 left `held` and `host left` **byte-for-byte identical** (103.5 / 7.1
GiB). `HALOGEN_HOST_RESERVE_GIB` feeds the fit-check budget and is *not* an
allocation; the actual footprint is `weights + KV + arena + vision`, none of
which the reserve touches. So the knob has exactly two regimes:

| reserve | budget | vs required 36.5 | effect |
|---|---|---|---|
| 15 | 38.5 | +2.0 | deployed |
| 16 | 37.5 | +1.0 | no change to footprint, thinner margin — pointless |
| 17 | 36.5 | +0.02 | knife edge, inside the noise of the 67.7 term |
| **18** | **35.5** | **−1.0** | **clamps the pool to 262144** |
| 24 | 29.5 | −7.0 | clamps the pool to 262144 |

The 36.5 requirement is the engine's own, read from a clamped boot:
`524288 positions need ~35.0 GiB (plus 1.5 of margin)`. Above 16 the only
thing a bigger reserve can do is take the doubling away.

**To actually free host memory, change the pool or the arena**, and the reserve
then becomes free to rise: `HALOGEN_KV_POOL_POSITIONS=262144` hands back
7.2 GiB from the doubled pool, `HALOGEN_MAX_TOK=16384` hands back 8.4. Both
are one-line edits that the fit check will confirm on the next boot. The
2026-09-13 trim to 401,408 was this remedy applied at half strength —
`HALOGEN_MAX_TOK` untouched, and on the host-left metric it bought +2.0 GiB
with a 4.5x improvement in contiguous blocks (310 → 1396), less than the
modelled 3.4 because the engine holds the freed KV as working slack.

### The `WARNING` line only appears when the fit check had to move

This one cost me a wrong conclusion, so it is worth stating plainly. When the
configured pool already fits, the engine reserves it **and prints nothing** —
the 09-11 baseline boot goes straight from the pre-flight line to
`checkpoint: mapped` with no `kv pool:` sizing block at all. The sizing block and
its `WARNING even 262144 positions ... host RAM cannot spare it` appear **only
after a clamp**.

So the absence of a warning in old logs is not evidence of headroom. Both
`262144/32768` rows above need 29.8 against a 29.5 budget — 0.27 over — and the
baseline has lived in that state since it was deployed without incident,
because the 24 GiB reserve is itself generous (the host reports 12–14 GiB free
at that point). The engine's budget is a conservative floor, not a hard wall.

### Two failure modes, and only one is dangerous

Raising `max_tok` past the ceiling does not shrink the pool — the arena alone
outgrows the budget, and the server cannot allocate at all:

```
kv pool: 524288 positions need ~51.7 GiB ... LOWERING THE POOL TO 262144 (~44.5 GiB).
kv pool: WARNING even 262144 positions (the context, the smallest pool that can
         serve one request) needs ~44.5 GiB and host RAM cannot spare it.
dmalloc: FAILED requesting 0.250 GiB after 39.970 GiB in 566 allocations (out of memory)
```

This is the quiet one: the unit stays `active`, the container reports
`Up (starting)`, and `deploy.sh` sits in its health wait for the whole
`LLM_WAIT` rather than failing fast. It confirms upstream's warning that a
large `HALOGEN_MAX_TOK` means "the server will not start", not "it will be
slower".

Under-running the ceiling is far more benign and far easier to miss. At 524288
with the arena back at 32768, `HALOGEN_KV_POOL_FIT` lowered the pool and
`deploy.sh` still reported healthy in 12 seconds:

```
kv pool: 524288 positions need ~35.0 GiB (plus 1.5 of margin) and host
         RAM cannot spare it. LOWERING THE POOL TO 262144 (~27.8 GiB).
```

A green health check does not mean the pool you asked for is the pool you got —
read the `kv pool:` lines after every restart.

The 47.7 GiB n-gram lookup table is never held in RAM; it is read through
the file cache. The 2.0 GiB the firmware carves for the iGPU appears in no
`/proc/meminfo` figure at all.

**Co-tenancy.** `HALOGEN_FLASH_PIN_TRUNK=1` (default) is what the published
decode numbers were measured with, and it means this container holds most of
the host. If a co-tenant then takes the remainder, both it and the engine can
stall for minutes at a time at full CPU with no disk activity and no output.
That is documented behaviour, not a crash, and it needs no restart.
`HALOGEN_FLASH_PIN_TRUNK=0` gives the memory back and costs several times the
decode speed — a last resort, not a tuning option.

## Client contract

The differences from llama.cpp that bite are all in the request, not the
deployment:

- **No server-side sampler defaults.** `temperature` absent or `0` is
  **greedy** decode; above 0 the request samples on the same drafter. Supported
  fields: `temperature`, `top_p`, `top_k`, `min_p`, `seed`,
  `presence_penalty`, `frequency_penalty`, `logit_bias`, `logprobs`. Refused
  with a 400: `top_logprobs`, `logprobs` with `stream: true`, `n > 1`, and any
  value outside its defined range (values are refused, never clamped).
  The fleet runs greedy for clients that stay silent — deterministic, and
  byte-identical with the drafter.
- **The token budget covers thinking.** The default is **8192**, and a budget
  that runs out mid-thought returns `finish_reason: "length"` with an **empty
  `content`** and the partial reasoning in `reasoning_content` — which most
  clients do not display. Above the cap (`HALOGEN_MAX_TOKENS_CAP`, 65536) a
  request is a 400 rather than a silent truncation. `max_tokens`,
  `max_completion_tokens` and `max_output_tokens` all work and must agree if
  more than one is sent.
- **A budget above (window − compaction-threshold) is a guaranteed 400 zone.**
  The fit check (`prompt + max_tokens <= HALOGEN_CTX`) refuses, and the harness
  only compacts at its own threshold — with OMP at 70 % of a 200,192 window
  (140,134), any prompt in (136,192, 140,134] with a 64,000 budget overflows
  the window while sitting below the compaction trigger. That collided live on
  2026-09-17 (prompt 138,898 + 64,000 = 202,898 > 200,192). The fix is
  harness-side: keep the declared `maxTokens` at or under
  `window − threshold` (57,344 chosen) so no pre-compaction request can miss
  the fit check. Do not fix it by raising `HALOGEN_CTX` past pool/2 — that
  breaks the two-resident-conversations property.
- **`reasoning_effort`** is accepted at the top level and inside
  `chat_template_kwargs`: `minimal`, `low`, `medium`, `high`, `xhigh`. The
  model's default is `xhigh`. `minimal` aliases the template's `low` and still
  reasons; `enable_thinking: false` turns thinking off; `preserve_thinking` is
  honoured. `preserve_reasoning` is a **400** — it is the llama.cpp-era key.
- `/v1/models`, `/health` and completions all report the served id, and
  `reasoning_content` carries the reasoning trace.

**Pool admission is the one arithmetic trap.** A request reserves its prompt
**plus** `max_tokens` positions when it is admitted; if the pool cannot hold it
the request waits in arrival order, and after the queue timeout (3600 s) it
gets a `503 engine_busy`. A prompt large enough that prompt + budget exceeds
the pool can therefore never be admitted at all. The 2026-09-16 shape is a
400,384-position pool with `HALOGEN_CTX` 200,192 and two slots: two
conversations at the ceiling fit the pool exactly, so a pair that is both
near-full and both generating is pool-bound — the second waits for the first to
release its region. The pool binds **how many** conversations stay resident,
not how long one may be; the older 524,288 / 401,408 numbers live in the ledger
tables above.

## Vision

**None here since 2026-09-16.** `HALOGEN_VISION_TOWER` is deliberately unset:
the tower is a 1.97 GiB charge against the same device budget as the pool and
the arena, and the fleet's image path moved to [[frank-small-qwen38-gsq-rco]],
which loads the BF16 projector alongside its own weights. This server refuses
image content parts and says why, rather than ignoring them:

```json
{"error": {"message": "this server does not accept images: the engine was
started without a vision tower (HALOGEN_VISION_TOWER)", "type":
"invalid_request_error"}}
```

The llama.cpp side handles images the same way this engine used to: a `data:`
URL or bare base64, about 2,040 tokens of context per image, and `http(s)` URLs
refused deliberately (fetching one would make the server issue outbound
requests to wherever a client pointed it). Audio and video have no path
anywhere in the fleet — that left with [[frank-npu]].

## Deploy and operate

```bash
./scripts/deploy.sh dankclaw --service frank-large
```

The handler retires the ik-era **root** unit first (stop, disable, move
`/etc/systemd/system/frank-large.service` into
`~/.AGENTTRASH/20260916-halogen-restore/host-units` — never `rm`; this host's
`disable` may delete the file outright, which the handler now reports), stops
the Quadlet's own unit if it is running, then pulls
`ghcr.io/peonist-ai/halogen-flash-server:0.5.8`, re-tags it to the
`localhost/` alias the unit references, and **waits for port 7770 to be free**
before installing and starting anything.

That last gate is not paranoia. On the first cutover attempt the ik server
logged "Deactivated successfully", and pasta — rootless podman's userspace
network stack, which binds the published port at container CREATE — still
reported `Address already in use` a second later, exiting the unit with code
126. Only `Restart=on-failure` recovered it, minutes later. A deploy that
starts a doomed container has not deployed anything, so `wait_port_free` polls
every socket on the port (listener, accepted, TIME_WAIT) and fails loudly with
the socket table if it never clears.

Boot persistence comes from `[Install] WantedBy=default.target` in the unit
file plus linger on the deploy user — generated units **cannot** be
`systemctl --user enable`d on this host.

```bash
systemctl --user status frank-large.service     # the generated unit
podman logs frank-large | head -40              # version banner + memory ledger
curl -s http://127.0.0.1:7770/health            # 200 only after the engine PONGs
curl -s http://127.0.0.1:7770/v1/models
```

`/health` asks the **engine** for a PONG before answering, so a 200 means
"ready to serve", not merely "process alive". During the first-start download
it stays unhealthy by design.

## Troubleshooting

| symptom | cause |
|---|---|
| `dmalloc: FAILED requesting … (out of memory)` then `HIP … out of memory` | the KV pool did not fit. `HALOGEN_KV_SLOTS` will not fix it (slots share the pool); lower `HALOGEN_KV_POOL_POSITIONS` to 262144, then `HALOGEN_MAX_TOK` to 16384 (~9 % prefill) |
| starts, answers short prompts, collapses to a few tok/s on a long one, disk busy, process in uninterruptible sleep | short of **file cache**, not memory — lower the pool and/or raise `HALOGEN_HOST_RESERVE_GIB` |
| long replies that look cut off | read `finish_reason`: `stop` means you have the whole answer, `length` means you ran out of budget (thinking counts) |
| a request that never returns, then `503 engine_busy` | the pool could not reserve prompt + `max_tokens`; see the admission trap above |
| both the engine and a co-tenant stall for minutes at full CPU | the pinned-trunk trade; see Memory model |
| `free`, `MemAvailable`, Grafana all wrong by ~68 GiB | expected — locked weights are counted as reclaimable |

`HALOGEN_VERBOSE=1` ships in the unit, so the fullest startup account is
already in the log when something goes wrong. `HALOGEN_DMALLOC_LOG` is
deliberately **not** set: it logs every allocation over 64 MB, which is
steady-state noise on a shared host — raise it only for a live OOM report.
The lines worth grepping first:

```bash
podman logs frank-large 2>&1 | grep -E '^(dmalloc|kv pool):'
```

Retrieval: `podman logs frank-large` covers the **current** instance only —
the Quadlet runs `--replace --rm`, so everything before the last recreate is
gone from the container log. `journalctl --user -u frank-large.service` is
the durable record. Both are rootless: `sudo podman logs frank-large` looks
in root's storage and answers "no such container".

## Rollback

The predecessor is intact: the ik payload stays at
`/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`, the
ik build at `/home/dank/GitPackages/ik_llama.cpp/build`, and its unit, env,
serve script and the Qwen-Sharp template are archived in the repo under
`scripts/.AGENTTRASH/*.ik-llama.cpp` and `*.ik-llama-qwen-sharp`. Rolling back
is a `git revert` of this change set plus a deploy — **but the Quadlet must be
removed from `~/.config/containers/systemd/` first**, or the restarted root
unit and the still-installed container both claim port 7770. The port gate in
the handler now covers exactly that race. See [[frank-large-ik-uncensored]].

The `amd_iommu=off` kernel argument that came with this change is separate and
reverts on its own — see [[dankclaw-iommu-translation-off]].

## Citations

- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README, `docs/FLAGS.md`, `deploy/entrypoint.sh`, `docker-compose.yml`, CHANGELOG (v0.5.8)
- [peonist-ai/halogen-qwen3.8-flash-next](https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next) — weights and file inventory
- [Qwen/Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) — base model