fix(quant): a tap is only a tap when its rig is running

Four gates, all bought with the 2026-10-29 incident: the rig's calibration boot died of an OOM,
the chain's health loop expired without failing, and the pusher fed four rounds of 10M tokens
(86 minutes of prefill) into RESTORED PRODUCTION, recording nothing.

- run-qwen38-bf16.sh: --ngram-placement becomes a knob whose calib default is `disk` (the 95.37
  GiB table pin is what starved that boot: 230.22 GiB pinned of 246, 8.06 GiB available) while
  kldref and score keep `ram`. calib writes .rig-boot into the tap dir and retires it on a
  non-zero exit, so a dead boot cannot leave the tap looking rig-authorized.
- calib-push.py: refuses before the first prompt unless the marker exists, names this container,
  and that container is running. With the driver's existing tap-progress gate, a push can no
  longer spend hours feeding an endpoint that is not the rig.
- qwen38-calib-window.sh: the calibration window as a gated chain — stop on a failed phase
  instead of falling through the health wait, bank the 336 GiB reference container into trash
  only when both phases exit 0 and the tap holds grams, and print the whole plan on --dry-run.
- kld-gate-qwen38.sh: two-report mode (candidate vs the served payload) for the disk where the
  172.8 GiB FP8 anchor cannot live, plus the laws the report always carried and the gate never
  read: a p99.9 tail ceiling and every by_source slice passing mean, tail and top-1.
- frank-archon.md: the pipeline laws (marker, n-gram tier, three-metric acceptance) and a Files
  section that names the live campaign instead of the retired swift15 era.

Tests: 587/0 — 13 new behavioral cases (dry-run argv, marker refusals, two-report gate math).
This commit is contained in:
Frank committed 2026-10-29 16:14:16 -07:00
1 parent 3e5a37a21f
commit c238e32d6f
6 files changed
+523 -56

No files matched your search

+33 -5
View File
@@ -562,6 +562,29 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.
### The quantization pipeline
- **A calibration tap is only a tap when the rig that opened it is running.** The runner writes
`.rig-boot` into the tap dir (container, image, pid) and retires it when its boot exits non-zero;
`calib-push.py` refuses without that marker, refuses when it names another container, and refuses
when the named container is not running; `qwen38-calib-window.sh` stops on a failed phase instead
of falling through its health wait. All three exist because on 2026-10-29 the unguarded chain's
rig boot died of an OOM, the health loop expired without failing, and the pusher fed four rounds
of 10M tokens into **restored production** — 86 minutes of live prefill that recorded nothing.
- **The n-gram table's pin is what starves a rig boot, and the tier is a knob.** On the BF16 payload
the table is 95.37 GiB, `--ngram-placement ram` pins it, and the calibration boot died at
`startup stopped: out of memory` with 230.22 GiB pinned of 246.07 GiB and 8.06 GiB available —
while the reference recording and every score boot succeeded at the same pool and slot count.
Calibration only needs the routed weights prefilled, so it reads the table from the container's
file tier (`--ngram-placement disk`); kldref and score keep production's placement, because their
numbers are only comparable at an identical shape.
- **Acceptance is three metrics and every slice, not a mean.** `scripts/kld-gate-qwen38.sh` grades
mean KL, the p99.9 tail, top-1 agreement and every `by_source` slice (code, prose, tools):
a mean can hide a few positions taking the whole hit, and an overall win can be bought by
regressing one distribution. The FP8-native anchor is an interval, not a requirement — without
the 172.8 GiB fetch the gate runs in two-report mode (candidate vs the served payload) and says
so on its first line.
## Contract (the wire)
- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
@@ -638,14 +661,19 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
`scripts/qwen38-froggeric-v22.5.jinja`.
- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
`scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
- The campaign (qwen38-w5nl64, live): `scripts/run-qwen38-bf16.sh` (kldref / calib / score — the
ONE measuring shape), `scripts/qwen38-calib-window.sh` (the gated calibration window),
`scripts/convert-qwen38-w5nl64.sh` with
`scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe` (the arm driver and its template),
`scripts/kld-gate-qwen38.sh` (acceptance), `scripts/calib-push.py`,
`scripts/make-calib-corpus.py`, `scripts/rechunk-kld-corpus.py`,
`scripts/qwen38_common.py` (the tokenizer, JSONL and refusal plumbing those three
corpus tools share — one `n_tokens`, so every tool counts the same way),
`scripts/derive-protected-experts.py`
`scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
`scripts/derive-protected-experts.py`, `scripts/apply-protected-cast.sh`.
- Retired swift15-era campaign scripts, superseded by the above and kept only as reference:
`scripts/convert-swift15-w4dn8.sh`, `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
`scripts/run-swift15-bf16.sh`, `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`,
`scripts/verify-swift15-ple.py`.
- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
`scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.