model cache cleared, re-fetched, and the update turned out to be real
Operator instruction: purge the cache so no run can serve a stale payload. The two live payloads were cleared and then re-fetched rather than left absent — deleting them without a re-fetch means neither endpoint starts. halogen's re-fetch proved the point: its sidecars had been stale *in place* (overlay.hgn 2,477,677,120 -> 2,572,466,560 bytes, overlay-speed.hgn 2,383,306,048 -> 2,478,095,488) while the 115.55 GiB checkpoint returned byte-identical. frank-small's 27B and mmproj re-fetched to Hub-matching hashes, unchanged. Trap recorded: podman refuses a bind mount whose source directory is missing (statfs ... no such file or directory), so frank-large needs /models to exist *empty* for its first-start fetch to run. Q4_K_S against the recorded Q4_K_M numbers: +2.1/+1.6/+2.6 % (iGPU-only pp512 / pp16384 / tg64), +2.7 % tg32@64k, +4.1 %/+2.0 % for the two layer-split configs, ~0 on both pp512@64k, host RAM bottoming at 10.0 GiB against ~5 GiB. A memory lever with a small speed dividend. IQ4_XS (97,473,155,360 bytes, sha256 b4787265...d0574f9d) is pulled and verified for the next comparison. Also repairs an AGENTS.md edit that landed in the wrong section: the peer session's commits shifted that file between my read and my write, so the frank-large text briefly split the Model Config section.
This commit is contained in:
@@ -707,3 +707,44 @@ timestamp: 2026-09-14
|
||||
- Both endpoints restored and healthy: frank-large `400384 positions, ~27.2 GiB`
|
||||
against a 33.5 GiB budget (no clamp), frank-small 200 on 7771. Page:
|
||||
[[x4-pair-experiment]] (rewritten).
|
||||
|
||||
## 2026-09-16 (night) — model cache cleared on instruction, then re-fetched, and the update was real
|
||||
- **Operator instruction:** purge the model cache outright so no run can serve a
|
||||
stale payload. Done with one-time `rm` authorisation, and the two *live*
|
||||
payloads were cleared and immediately re-fetched — because deleting them
|
||||
without a re-fetch means neither endpoint can start.
|
||||
- **Cleared:** the Uncensored i1-Q4_K_M payload (112G), the Q4_K_S part-fragments
|
||||
(105G), the retired kingjones STRIX and agentionai ROCmFP4 payloads (114G +
|
||||
863M), an unreferenced Qwen3.5 vision dir (8.8G), and all of `~/.AGENTTRASH`
|
||||
including the ik part-files (119G). Disk went **298 GB → 895 GB free**; 814 GB
|
||||
after re-fetching what the services need.
|
||||
- **The re-fetch produced genuine updates.** halogen's sidecars had been stale —
|
||||
`overlay.hgn` 2,477,677,120 → **2,572,466,560** B, `overlay-speed.hgn`
|
||||
2,383,306,048 → **2,478,095,488** B — while `w4b.hgn` came back the same size
|
||||
(124,068,083,904). So "clear and re-pull" *is* an update mechanism here, and
|
||||
the sidecars are versioned **in place**: a stale copy is invisible from the
|
||||
file name. frank-small's 27B + mmproj re-fetched to hashes matching the Hub
|
||||
exactly and are unchanged.
|
||||
- **Trap:** podman refuses a bind mount whose source directory is missing
|
||||
(`Error: statfs /home/dank/models/halogen: no such file or directory`), so
|
||||
frank-large looped on `Restart=on-failure` until `mkdir -p` recreated the
|
||||
**empty** directory — then the engine fetched the repo itself
|
||||
(`HALOGEN_DOWNLOAD`), which is the documented refresh path.
|
||||
- **Q4_K_S vs Q4_K_M, measured on the same host and build:** +2.1 / +1.6 / +2.6 %
|
||||
(iGPU-only pp512 / pp16384 / tg64), +2.7 % tg32@64k, +4.1 % and +2.0 % for the
|
||||
two layer-split configs, ~0 on both pp512@64k. Host RAM bottomed at **10.0 GiB
|
||||
available** in A\* against ~5 GiB for Q4_K_M. A memory lever with a small speed
|
||||
dividend — nothing structural moves.
|
||||
- **IQ4_XS pulled in parallel with the bench**, as asked: 97,473,155,360 bytes,
|
||||
sha256 `b4787265…d0574f9d`, matching the Hub's LFS digest. Kept as the fresh
|
||||
payload for the next comparison; Q4_K_S was cleared once the bench was done.
|
||||
- **Held back deliberately:** `models/checkpoints` (23 GB of Flux 2 Klein — no
|
||||
unit mounts it, but its upstream is licence-gated, so it is not mine to
|
||||
destroy) plus the small tool dirs under `models/` and `models/logs`, which is
|
||||
the evidence for this page.
|
||||
- **Verified after the refresh:** `/v1/models` correct on 7770/7771, real
|
||||
completions returning `pong` on both, halogen's ledger `68.0 GiB weights
|
||||
locked / 11.0 GiB KV pool`, 2 slots over one 400384-position pool, and **307**
|
||||
contiguous 2 MiB blocks left (240 before the refresh).
|
||||
- Runbook: [[halogen-flash-server]] gains a "Refreshing the weights" note; the
|
||||
quant comparison is appended to [[x4-pair-experiment]].
|
||||
|
||||
Reference in New Issue
Block a user