fix(qwen38): the preflight asks whether the pin is ONE of an image's digests
A by-digest pull leaves the image carrying every digest it was pulled under, so reading RepoDigests[0] reported a healthy pinned image as wrong and refused a window that was ready to run. Membership, not position. Also records the measured step optimum: 106 tok/s at an 8,192-token step against 76 at 16,384 on a 32,768 calibration window, so the wide step has a ceiling, not a plateau.
This commit is contained in:
1 parent
5c678b804c
commit
44d72b9e78
3 files changed
+24
-9
No files matched your search
@@ -571,6 +571,16 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
|
||||
of falling through its health wait. All three exist because on 2026-10-29 the unguarded chain's
|
||||
rig boot died of an OOM, the health loop expired without failing, and the pusher fed four rounds
|
||||
of 10M tokens into **restored production** — 86 minutes of live prefill that recorded nothing.
|
||||
- **A calibration step has an OPTIMUM, and it is not the widest step that boots.** The tap's cost is
|
||||
expert re-reads per document (a 7,168-token document cut into four 2,048-token steps re-streams
|
||||
each layer's routed expert union four times), which argues for the widest step — but the measured
|
||||
curve turns over: at the 32,768 calibration window one 2-way prefill of the same document ran
|
||||
**106 tok/s at an 8,192-token step and 76 tok/s at 16,384**, with the computed activation arena
|
||||
growing 3.39 → 4.50 GiB, so the wider step pays more compute than it saves in reads. The
|
||||
200,000-window `QSA score plane` refusal that made the wide step look impossible is an artifact of
|
||||
serving the production window: the same 16,384-token step needs a 512 MiB plane at 32,768.
|
||||
`Q38_CALIB_MNBT` is the only knob — the KL modes keep production's step, because their numbers are
|
||||
only comparable at one shape.
|
||||
- **A container that exists, a port that answers, and a `/health` 200 are three different claims —
|
||||
bind identity before believing any of them.** Measured 2026-10-30: `podman inspect` named the rig
|
||||
while the port still belonged to the *dying* production process (`health ok waited=0s`, window dead
|
||||
|
||||
Reference in new issue
Block a user