model cache cleared, re-fetched, and the update turned out to be real

Operator instruction: purge the cache so no run can serve a stale payload. The two
live payloads were cleared and then re-fetched rather than left absent — deleting
them without a re-fetch means neither endpoint starts.

halogen's re-fetch proved the point: its sidecars had been stale *in place*
(overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn
2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned
byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes,
unchanged.

Trap recorded: podman refuses a bind mount whose source directory is missing
(statfs ... no such file or directory), so frank-large needs /models to exist
*empty* for its first-start fetch to run.

Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 /
pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs,
~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory
lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256
b4787265...d0574f9d) is pulled and verified for the next comparison.

Also repairs an AGENTS.md edit that landed in the wrong section: the peer
session's commits shifted that file between my read and my write, so the
frank-large text briefly split the Model Config section.
This commit is contained in:
Frank committed 2026-09-16 22:07:04 -07:00
1 parent 77a07ffc30
commit a40a1aad92
4 files changed
+103 -11

No files matched your search

+16 -7
View File
@@ -264,17 +264,26 @@ of 2026-09-16 the image path is `frank-small` (`input: [text, image]`), and
tower. There is **no audio** input anywhere in the fleet: that capability left
with the FLM NPU endpoint on 2026-09-11.
## Active Services
### frank-large — halogen-flash-server (Dankclaw, ACTIVE)
halogen-flash-server 0.5.8 as a **rootless Quadlet again since 2026-09-16**
(`dankclaw/frank-large/quadlet/frank-large.container`), serving
`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + the
2.31 GiB quality overlay) on port 7770 from the gfx1151 iGPU. Two conversations
of 200,192 tokens resident (pool 400,384, 2 slots), greedy default, reasoning
extracted to `reasoning_content`, and **no vision tower** — that moved to
frank-small. It replaced ik_llama.cpp, which had replaced it for two days.
`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + a
quality overlay that re-fetched to 2.40 GiB on 2026-09-16) on port 7770 from
the gfx1151 iGPU. Two conversations of 200,192 tokens resident (pool 400,384,
2 slots), greedy default, reasoning extracted to `reasoning_content`, and **no
vision tower** — that moved to frank-small. It replaced ik_llama.cpp, which had
replaced it for two days.
The engine **fetches its own weights on first start** (`HALOGEN_DOWNLOAD`, into
the `/home/dank/models/halogen` bind mount): clearing that directory and
restarting the unit is the supported way to pick up upstream updates, and it
needs the directory to exist **empty** — a missing directory fails the
container with `Error: statfs /home/dank/models/halogen: no such file or
directory` and the unit loops on `Restart=on-failure` until `mkdir -p` brings
it back. The sidecars are versioned in place, so a stale copy is invisible from
the file name (both overlays were ~95 MB larger upstream than the copies this
host had been serving).
Measured boot ledger (2026-09-16, `HALOGEN_VERBOSE=1` ships in the unit):
device budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `~27.2 GiB`, so