model cache cleared, re-fetched, and the update turned out to be real
Operator instruction: purge the cache so no run can serve a stale payload. The two live payloads were cleared and then re-fetched rather than left absent — deleting them without a re-fetch means neither endpoint starts. halogen's re-fetch proved the point: its sidecars had been stale *in place* (overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn 2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes, unchanged. Trap recorded: podman refuses a bind mount whose source directory is missing (statfs ... no such file or directory), so frank-large needs /models to exist *empty* for its first-start fetch to run. Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 / pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs, ~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256 b4787265...d0574f9d) is pulled and verified for the next comparison. Also repairs an AGENTS.md edit that landed in the wrong section: the peer session's commits shifted that file between my read and my write, so the frank-large text briefly split the Model Config section.
This commit is contained in:
@@ -58,7 +58,7 @@ discovery, the `services.json` probe and the docs needed no rename.
|
||||
| file | size | role |
|
||||
|---|---|---|
|
||||
| `qwen38-flash-next-w4b.hgn` | 115.55 GiB | the checkpoint |
|
||||
| `qwen38-flash-next-w4b.overlay.hgn` | 2.31 GiB | **quality sidecar** — picked up automatically; the engine warns at startup if it is missing |
|
||||
| `qwen38-flash-next-w4b.overlay.hgn` | 2.40 GiB | **quality sidecar** — picked up automatically; the engine warns at startup if it is missing. **Refreshed 2026-09-16**: the sidecars are versioned *in place*, so a stale local copy is invisible from its file name — this one had read 2.31 GiB since it was first fetched while the Hub had moved on |
|
||||
| `qwen38-flash-next-vision.hgn` | 0.84 GiB | vision tower (see below) |
|
||||
| `tokenizer/` | ~21 MB | flat tokenizer dir; found beside the checkpoint, so no extra mount |
|
||||
|
||||
@@ -68,9 +68,23 @@ never re-downloads, and `huggingface_hub` resumes an interrupted pull. The
|
||||
mount must be read-write for this. Roughly 127 GB; on Dankclaw it lands in
|
||||
`/home/dank/models/halogen`.
|
||||
|
||||
`overlay-speed.hgn` (2.22 GiB) comes along with the repo but is unused: it
|
||||
trades the `o_proj` calibration for about 2 % of decode, and the quality
|
||||
sidecar is the default for a reason.
|
||||
**Refreshing the weights.** Delete the mount's contents (or the whole
|
||||
directory, then recreate it **empty**) and start the unit: the entrypoint sees
|
||||
the checkpoint absent and re-fetches the repo, resuming if interrupted. Two
|
||||
traps. First, the bind mount's source directory must *exist* — podman refuses
|
||||
the container with `Error: statfs /home/dank/models/halogen: no such file or
|
||||
directory` and the unit loops on `Restart=on-failure`; `mkdir -p` the empty
|
||||
directory and the fetch starts on the next start. Second, "refresh" only means
|
||||
"fetch what the Hub has now": on 2026-09-16 the re-fetch brought newer sidecars
|
||||
(`overlay.hgn` 2,477,677,120 → **2,572,466,560** bytes, `overlay-speed.hgn`
|
||||
2,383,306,048 → **2,478,095,488**) while the checkpoint returned
|
||||
byte-identical in size (124,068,083,904) — so an update can live entirely in
|
||||
the sidecars, and equal file sizes are not proof of an unchanged repo.
|
||||
|
||||
`overlay-speed.hgn` (2.31 GiB on the Hub; 2.22 GiB before the 2026-09-16
|
||||
refresh) comes along with the repo but is unused: it trades the `o_proj`
|
||||
calibration for about 2 % of decode, and the quality sidecar is the default for
|
||||
a reason.
|
||||
|
||||
## Deployed flags
|
||||
|
||||
|
||||
Reference in New Issue
Block a user