model cache cleared, re-fetched, and the update turned out to be real

Operator instruction: purge the cache so no run can serve a stale payload. The two
live payloads were cleared and then re-fetched rather than left absent — deleting
them without a re-fetch means neither endpoint starts.

halogen's re-fetch proved the point: its sidecars had been stale *in place*
(overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn
2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned
byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes,
unchanged.

Trap recorded: podman refuses a bind mount whose source directory is missing
(statfs ... no such file or directory), so frank-large needs /models to exist
*empty* for its first-start fetch to run.

Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 /
pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs,
~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory
lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256
b4787265...d0574f9d) is pulled and verified for the next comparison.

Also repairs an AGENTS.md edit that landed in the wrong section: the peer
session's commits shifted that file between my read and my write, so the
frank-large text briefly split the Model Config section.
This commit is contained in:
Frank
2026-09-16 22:07:04 -07:00
parent 77a07ffc30
commit a40a1aad92
4 changed files with 103 additions and 11 deletions
+41
View File
@@ -707,3 +707,44 @@ timestamp: 2026-09-14
- Both endpoints restored and healthy: frank-large `400384 positions, ~27.2 GiB`
against a 33.5 GiB budget (no clamp), frank-small 200 on 7771. Page:
[[x4-pair-experiment]] (rewritten).
## 2026-09-16 (night) — model cache cleared on instruction, then re-fetched, and the update was real
- **Operator instruction:** purge the model cache outright so no run can serve a
stale payload. Done with one-time `rm` authorisation, and the two *live*
payloads were cleared and immediately re-fetched — because deleting them
without a re-fetch means neither endpoint can start.
- **Cleared:** the Uncensored i1-Q4_K_M payload (112G), the Q4_K_S part-fragments
(105G), the retired kingjones STRIX and agentionai ROCmFP4 payloads (114G +
863M), an unreferenced Qwen3.5 vision dir (8.8G), and all of `~/.AGENTTRASH`
including the ik part-files (119G). Disk went **298 GB → 895 GB free**; 814 GB
after re-fetching what the services need.
- **The re-fetch produced genuine updates.** halogen's sidecars had been stale —
`overlay.hgn` 2,477,677,120 → **2,572,466,560** B, `overlay-speed.hgn`
2,383,306,048 → **2,478,095,488** B — while `w4b.hgn` came back the same size
(124,068,083,904). So "clear and re-pull" *is* an update mechanism here, and
the sidecars are versioned **in place**: a stale copy is invisible from the
file name. frank-small's 27B + mmproj re-fetched to hashes matching the Hub
exactly and are unchanged.
- **Trap:** podman refuses a bind mount whose source directory is missing
(`Error: statfs /home/dank/models/halogen: no such file or directory`), so
frank-large looped on `Restart=on-failure` until `mkdir -p` recreated the
**empty** directory — then the engine fetched the repo itself
(`HALOGEN_DOWNLOAD`), which is the documented refresh path.
- **Q4_K_S vs Q4_K_M, measured on the same host and build:** +2.1 / +1.6 / +2.6 %
(iGPU-only pp512 / pp16384 / tg64), +2.7 % tg32@64k, +4.1 % and +2.0 % for the
two layer-split configs, ~0 on both pp512@64k. Host RAM bottomed at **10.0 GiB
available** in A\* against ~5 GiB for Q4_K_M. A memory lever with a small speed
dividend — nothing structural moves.
- **IQ4_XS pulled in parallel with the bench**, as asked: 97,473,155,360 bytes,
sha256 `b4787265…d0574f9d`, matching the Hub's LFS digest. Kept as the fresh
payload for the next comparison; Q4_K_S was cleared once the bench was done.
- **Held back deliberately:** `models/checkpoints` (23 GB of Flux 2 Klein — no
unit mounts it, but its upstream is licence-gated, so it is not mine to
destroy) plus the small tool dirs under `models/` and `models/logs`, which is
the evidence for this page.
- **Verified after the refresh:** `/v1/models` correct on 7770/7771, real
completions returning `pong` on both, halogen's ledger `68.0 GiB weights
locked / 11.0 GiB KV pool`, 2 slots over one 400384-position pool, and **307**
contiguous 2 MiB blocks left (240 before the refresh).
- Runbook: [[halogen-flash-server]] gains a "Refreshing the weights" note; the
quant comparison is appended to [[x4-pair-experiment]].