model cache cleared, re-fetched, and the update turned out to be real

Operator instruction: purge the cache so no run can serve a stale payload. The two
live payloads were cleared and then re-fetched rather than left absent — deleting
them without a re-fetch means neither endpoint starts.

halogen's re-fetch proved the point: its sidecars had been stale *in place*
(overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn
2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned
byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes,
unchanged.

Trap recorded: podman refuses a bind mount whose source directory is missing
(statfs ... no such file or directory), so frank-large needs /models to exist
*empty* for its first-start fetch to run.

Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 /
pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs,
~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory
lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256
b4787265...d0574f9d) is pulled and verified for the next comparison.

Also repairs an AGENTS.md edit that landed in the wrong section: the peer
session's commits shifted that file between my read and my write, so the
frank-large text briefly split the Model Config section.
This commit is contained in:
Frank
2026-09-16 22:07:04 -07:00
parent 77a07ffc30
commit a40a1aad92
4 changed files with 103 additions and 11 deletions
@@ -58,7 +58,7 @@ discovery, the `services.json` probe and the docs needed no rename.
| file | size | role |
|---|---|---|
| `qwen38-flash-next-w4b.hgn` | 115.55 GiB | the checkpoint |
| `qwen38-flash-next-w4b.overlay.hgn` | 2.31 GiB | **quality sidecar** — picked up automatically; the engine warns at startup if it is missing |
| `qwen38-flash-next-w4b.overlay.hgn` | 2.40 GiB | **quality sidecar** — picked up automatically; the engine warns at startup if it is missing. **Refreshed 2026-09-16**: the sidecars are versioned *in place*, so a stale local copy is invisible from its file name — this one had read 2.31 GiB since it was first fetched while the Hub had moved on |
| `qwen38-flash-next-vision.hgn` | 0.84 GiB | vision tower (see below) |
| `tokenizer/` | ~21 MB | flat tokenizer dir; found beside the checkpoint, so no extra mount |
@@ -68,9 +68,23 @@ never re-downloads, and `huggingface_hub` resumes an interrupted pull. The
mount must be read-write for this. Roughly 127 GB; on Dankclaw it lands in
`/home/dank/models/halogen`.
`overlay-speed.hgn` (2.22 GiB) comes along with the repo but is unused: it
trades the `o_proj` calibration for about 2 % of decode, and the quality
sidecar is the default for a reason.
**Refreshing the weights.** Delete the mount's contents (or the whole
directory, then recreate it **empty**) and start the unit: the entrypoint sees
the checkpoint absent and re-fetches the repo, resuming if interrupted. Two
traps. First, the bind mount's source directory must *exist* — podman refuses
the container with `Error: statfs /home/dank/models/halogen: no such file or
directory` and the unit loops on `Restart=on-failure`; `mkdir -p` the empty
directory and the fetch starts on the next start. Second, "refresh" only means
"fetch what the Hub has now": on 2026-09-16 the re-fetch brought newer sidecars
(`overlay.hgn` 2,477,677,120 → **2,572,466,560** bytes, `overlay-speed.hgn`
2,383,306,048 → **2,478,095,488**) while the checkpoint returned
byte-identical in size (124,068,083,904) — so an update can live entirely in
the sidecars, and equal file sizes are not proof of an unchanged repo.
`overlay-speed.hgn` (2.31 GiB on the Hub; 2.22 GiB before the 2026-09-16
refresh) comes along with the repo but is unused: it trades the `o_proj`
calibration for about 2 % of decode, and the quality sidecar is the default for
a reason.
## Deployed flags