model cache cleared, re-fetched, and the update turned out to be real
Operator instruction: purge the cache so no run can serve a stale payload. The two live payloads were cleared and then re-fetched rather than left absent — deleting them without a re-fetch means neither endpoint starts. halogen's re-fetch proved the point: its sidecars had been stale *in place* (overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn 2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes, unchanged. Trap recorded: podman refuses a bind mount whose source directory is missing (statfs ... no such file or directory), so frank-large needs /models to exist *empty* for its first-start fetch to run. Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 / pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs, ~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256 b4787265...d0574f9d) is pulled and verified for the next comparison. Also repairs an AGENTS.md edit that landed in the wrong section: the peer session's commits shifted that file between my read and my write, so the frank-large text briefly split the Model Config section.
This commit is contained in:
1 parent
77a07ffc30
commit
a40a1aad92
4 files changed
+103
-11
No files matched your search
@@ -264,17 +264,26 @@ of 2026-09-16 the image path is `frank-small` (`input: [text, image]`), and
|
||||
tower. There is **no audio** input anywhere in the fleet: that capability left
|
||||
with the FLM NPU endpoint on 2026-09-11.
|
||||
|
||||
## Active Services
|
||||
|
||||
### frank-large — halogen-flash-server (Dankclaw, ACTIVE)
|
||||
|
||||
halogen-flash-server 0.5.8 as a **rootless Quadlet again since 2026-09-16**
|
||||
(`dankclaw/frank-large/quadlet/frank-large.container`), serving
|
||||
`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + the
|
||||
2.31 GiB quality overlay) on port 7770 from the gfx1151 iGPU. Two conversations
|
||||
of 200,192 tokens resident (pool 400,384, 2 slots), greedy default, reasoning
|
||||
extracted to `reasoning_content`, and **no vision tower** — that moved to
|
||||
frank-small. It replaced ik_llama.cpp, which had replaced it for two days.
|
||||
`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + a
|
||||
quality overlay that re-fetched to 2.40 GiB on 2026-09-16) on port 7770 from
|
||||
the gfx1151 iGPU. Two conversations of 200,192 tokens resident (pool 400,384,
|
||||
2 slots), greedy default, reasoning extracted to `reasoning_content`, and **no
|
||||
vision tower** — that moved to frank-small. It replaced ik_llama.cpp, which had
|
||||
replaced it for two days.
|
||||
|
||||
The engine **fetches its own weights on first start** (`HALOGEN_DOWNLOAD`, into
|
||||
the `/home/dank/models/halogen` bind mount): clearing that directory and
|
||||
restarting the unit is the supported way to pick up upstream updates, and it
|
||||
needs the directory to exist **empty** — a missing directory fails the
|
||||
container with `Error: statfs /home/dank/models/halogen: no such file or
|
||||
directory` and the unit loops on `Restart=on-failure` until `mkdir -p` brings
|
||||
it back. The sidecars are versioned in place, so a stale copy is invisible from
|
||||
the file name (both overlays were ~95 MB larger upstream than the copies this
|
||||
host had been serving).
|
||||
|
||||
Measured boot ledger (2026-09-16, `HALOGEN_VERBOSE=1` ships in the unit):
|
||||
device budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `~27.2 GiB`, so
|
||||
|
||||
Reference in new issue
Block a user