---
name: halogen-flash-server-ops
description: "Operate the frank-large halogen-flash-server deployment on Dankclaw (model swaps, restarts, memory sizing, health and client verification, rollback) — use for anything touching frank-large after the 2026-09-11 runtime swap"
---

# halogen-flash-server ops (frank-large, Dankclaw)

frank-large is a **rootless Quadlet container** since 2026-09-11:
`dankclaw/frank-large/quadlet/frank-large.container` in the serverAppConfigs repo,
installed to `~/.config/containers/systemd/`, generated unit `frank-large.service`
in the **user** manager. Full runbook: `serverAppConfigs-docs/LLM/halogen-flash-server.md`.

## Never use sudo systemctl on it

It is a user unit. `sudo systemctl stop frank-large` does nothing useful.

```bash
systemctl --user status frank-large.service
podman logs frank-large | head -40        # version banner + memory ledger
curl -s http://127.0.0.1:7770/health      # 200 only after the engine PONGs
curl -s http://127.0.0.1:7770/v1/models   # must list frank-large
```

## Deploy / change flags

1. Edit the `.container` file in the repo (all tunables are inline `Environment=` lines).
2. `./tests/run.sh` (the quadlet seam covers GPU devices, memlock, ipc/seccomp, the single published port, writable weights mount).
3. Commit, push, then `./scripts/deploy.sh dankclaw --service frank-large`.

The handler retires the old root unit (moving its symlink to the home trash), pulls and
re-tags the image to the `localhost/` alias, installs the unit, restarts it, and gates on
the download when the checkpoint is absent. Never `systemctl --user enable` a generated unit.

## Sizing a swap

Read the startup ledger before and after any change: it prints the locked weights, the KV
pool it chose, the working set, the BIOS iGPU carve-out, and the corrected free-memory
figure. `free`, MemAvailable and every monitor built on them overstate free memory by the
size of the model (~68 GiB) because the kernel counts the locked weights as reclaimable
page cache.

- KV is ~7.2 GiB per 262,144 positions; the prefill arena is 16.7 GiB at `HALOGEN_MAX_TOK=32768`.
- The device budget stops near 47 GiB *with the weights pinned*, and lower on a machine
  carrying more besides this server. `HALOGEN_KV_POOL_FIT=1` lowers the pool to fit and
  prints what it chose; a refused start needs a smaller `HALOGEN_KV_POOL_POSITIONS`
  (`HALOGEN_KV_SLOTS` will not help — slots share one pool).
- Long prompts crawling with the disk busy = file cache, not memory: lower the pool or
  raise `HALOGEN_HOST_RESERVE_GIB`.

## Client contract traps

- No server-side sampler defaults: a request without `temperature` decodes GREEDY.
- The per-request budget defaults to 8192 and **includes thinking**; a reasoning model
  plus a small budget returns EMPTY content with `finish_reason: "length"`. Cap is 65536
  (a 400 above it), and `max_tokens`/`max_completion_tokens`/`max_output_tokens` must agree.
- `response_format` is a 400 ("no constrained decoding") — as it is on FreeToken/frank-small.
  Any caller that needs structured output or a tiny budget needs a different backend.
- A request reserves prompt + `max_tokens` positions against the pool; larger requests queue
  and then fail with `503 engine_busy` after 3600 s.

## Verification after any change

Run a real completion (never just `/health`), a vision probe if the tower is enabled
(image content part with a `data:` URL; `http(s)` URLs are refused), then one call from the
client that consumes it. Rollback: the llama.cpp era is in `.AGENTTRASH/` (`.llama.cpp`),
the STRIX payload and ROCmFPX build stay on disk — but remove the Quadlet from
`~/.config/containers/systemd/` before restoring the root unit, or both claim port 7770.