---
name: radiance-calibration-tap-latest
description: "Capture radiance calibration Grams for GPTQ (RADIANCE_CALIB_DIR) on the R9700 rig: the tap only arms on latest, the fitting shape, the 12M floor rule, and how to read a run that looks hung."
---

Friction-earned 2026-10-30 on `dank/serverAppConfigs` (qwen38-w5nl64 campaign, 2×R9700 TP=2).

## The tap arms only on the latest radiance image

A 17-hour calib run on the *production* pin (1.8.1) wrote **zero bytes** into `RADIANCE_CALIB_DIR`
while `/health`, `/stats` and the pusher all reported healthy. The tap's arming code is byte-identical
in both trees — the shipped 1.8.1 build drops it. The tell is the boot line:

```
I[0] recorded passes: off, the program accumulates calibration Grams (gram_accum)
```

Present on latest (v1.14.0 = `:latest` 2026-10-30), absent on 1.8.1. Verify with
`podman inspect IMG --format '{{json .Labels}}'` → `org.opencontainers.image.version` /
`revision`, and pin the **index digest**, not the tag: `:latest` moves under you, and a local stale
`:latest` can be a 3-week-old `dev` build (check `.Created`).

Run the tap from a report container of the *same* model, but the tap needs `--ngram-placement disk`
for a calibration boot: the n-gram table pinned to RAM (`ram`) OOMs the rig's boot
(`startup stopped: out of memory`) while kldref/score boots succeed at the same pool.

## The fitting shape (vendor recipe does not boot 2×32 GiB)

The vendor's calibration line (`--max-num-batched-tokens 65536 --max-num-seqs 64`, ~930 tok/s)
assumes a bigger box: the activation arena alone is 12.83 GiB of 27.95 GiB claimable and every
variant stops in the VRAM resolver (short 3.19 GiB → 279 MiB → 59 MiB → 86 MiB → cache starved).
Wider steps are refused by the arch itself, pre-allocation:

```
qwen4exp: the QSA score plane would be 3125 MiB -- 16384 rows (--max-num-batched-tokens 16384)
          by --max-model-len 200000 / compress ratio 4. Lower either.
```

What boots and seals: 4 slots, `--max-num-batched-tokens 2048` (the file-tier ceiling),
`--kv-cache-dtype fp8`, `--weights-disk-tier`, `--placement expert_tiered`,
`--ngram-placement disk`, `--max-model-len 200000`, `--num-speculative-tokens 0`, `--deterministic`.
Ledger at that shape: `elastic 22.8 GiB — experts first, kv cache 1.50 GiB … 1.0 full-length`
(pool ≈ 199,948 tokens). On 1.8.1 the same payload's pool was 7,436 tokens — the resolver, not the
window, is what changed.

## Floor

`RADIANCE_CALIB_TOKENS` is a **pooled** token floor. Engine default 65,536. Fleet rule (Frank,
2026-10-30): **12,000,000**, because the served quant this arm competes with was calibrated on a
10 M-token set. The vendor's "10 M beats 1 M by under 1 %" measures *their* plate, not ours.
Consequence: a ~9.3 M-token corpus round seals in round 1 plus part of round 2, and a round that
seals nothing is NORMAL (`round_verdict` → continue, never failure).

## Reading a run (the trap that cost two cycles)

- Throughput is **streaming-bound**, ~60–70 tok/s whatever the slot count; concurrency only lengthens
  each request (2-way ≈ 4 min per 7.5 K-token prefill, 4-way ≈ 8).
- The engine logs **one line per completed request**. A log quiet for 8 minutes is therefore normal;
  `W step: nothing has been logged for 21 s` is step-level silence. Liveness = **completed-request
  count**, never log mtime. A watchdog on mtime kills healthy cycles and every false positive throws
  away accumulated tap tokens.
- One seal writes **~43 GB** (24,768 `gram.r<rank>.blk.<l>.ffn_down_exps.<e>.bin`) and the engine
  then **exits by itself**; a mid-flight client gets `429 rate_limit_error: the server is shutting
  down` — success, not failure.
- A port that answers is not the rig: after the runner stops production the dying process answers
  `:7770` briefly. Gate on `podman ps` naming the rig container, and on the `.rig-boot` marker.

## Orchestration and space

Use the repo's orchestrator, never a hand-rolled driver:
`bash scripts/qwen38-calib-window.sh` (gates: reference present, `.rad` present, tap dir ≥ 60 GiB,
healthy rig, marker, then push; banks the reference container only on success). Knobs:
`FRANK_QWEN38_CALIB_TOKENS`, `Q38_WINDOW_CALIB_CORPUS`, `Q38_WINDOW_ROUNDS`,
`Q38_CALIB_MIN_FREE_MIB`, `Q38_WINDOW_KEEP_RAD`.

A hand-rolled driver skips the space preflight; on this host the tap dir sat on `/` with 35 G free
while one seal writes 43 GB — the orchestrator refuses that, which is the point. Frank's rule: keep
artifacts on `/`, never `/mnt/entropy`.

## Then

`gram_files → scripts/grams-to-imatrix.py DIR -o imatrix.gguf` (stdlib-only, skips non-gram names
like `.rig-boot`) → `scripts/convert-qwen38-w5nl64.sh --plan-only` (space bill, `--plan-only` first)
→ convert (card-less, CPU BLAS) → `score` each arm + the served payload at the **production shape** →
`scripts/kld-gate-qwen38.sh` (mean KL ≤ 0.03, p99.9, top-1 ≥ 95 %, every `by_source` slice).