---
name: radiance-calib-window-serve-gate
description: "Diagnose a radiance calibration window that dies at second zero or refuses its push: the health answer coming from the dying production socket, the listener-identity gate, the preflight trio (space, re-chunked corpus, KEEP_RAD), the 12M floor and the wide-step knob."
---

# radiance calibration window — the serve gate and its preflight

Friction-earned 2026-10-30 on `dank/serverAppConfigs` (`scripts/qwen38-calib-window.sh`,
`scripts/run-qwen38-bf16.sh`). A 12 M-token window died at second zero; the cause was the gate,
not the engine.

## The signature, and what it really means

```
phase=calib health ok waited=0s
the rig container qwen38-bf16 is not running        <- pusher refuses
phase=calib push end rc=2  ... FAIL: serve rc=1 push rc=2
```

`waited=0s` is the tell. The runner stops `frank-archon` **first**; the dying production process keeps
its listener for a moment, while the freshly created rig container is already `State.Running=true` but
has not bound the port yet. So `podman inspect` (container running) **and** `/health` 200 (production's
socket) both hold for the wrong process.

**The gate must prove the listener belongs to the rig:**

```bash
lpid=$(ss -H -ltnp 'sport = :7770' | grep -o 'pid=[0-9]*' | head -1 | cut -d= -f2)
[ -n "$lpid" ] || return 1
podman top "$RIG" pid | grep -qw "$lpid"
```

Landed as PR #90 with two red-first tests (foreign listener ⇒ no push, window fails; in-container
listener ⇒ proceeds and banks). If you see `waited=0s` on a checkout older than that PR, the window
there is pre-fix: do not run a long window on it.

## Preflight trio — the window refuses correctly, so do not "fix" it

1. **Tap dir ≥ 60 GiB free.** One seal writes **43 GB in 24,768 files**
   (`gram.r<rank>.blk.<l>.ffn_down_exps.<e>.bin`). Point it at a 35 GiB root and the seal fills the
   filesystem to zero. `Q38_CALIB_MIN_FREE_MIB` overrides — for *tests only* (fixtures on `/tmp` tmpfs).
2. **The re-chunked corpus.** `Q38_WINDOW_CALIB_CORPUS` defaults to `calib-7168.jsonl`; pushing
   `calib.jsonl` instead runs un-chunked documents through a window sized for chunks.
3. **`Q38_WINDOW_KEEP_RAD=1` when two windows run back to back.** The window banks the BF16 reference
   container into `~/.AGENTTRASH/` on success; without this, stage 2 has no reference to tap.

## Floor and the "add more tokens later" trap

`RADIANCE_CALIB_TOKENS` is the campaign's number, not the vendor's: the reference container this arm
competes with was calibrated on 10 M tokens, so the fleet floor is **12 M**. Engine default is 65,536.

**A sealed tap cannot be extended.** Hessians are VRAM accumulators; the seal writes and drops them.
So "run 1 M to verify, then add 11 M" is two windows — 1 M then a fresh 12 M — not an append. Budget
accordingly, and rotate the verification tap to `.AGENTTRASH/`.

One round of the re-chunked corpus is **9.31 M tokens**, so a 12 M floor seals in round 1 plus part
of round 2; a flat round is normal (the floor is pooled; `round_verdict` says "continue").

## The wide-step knob (the tap's real cost)

`Q38_CALIB_MNBT` / `Q38_CALIB_SEQS` set the calibration step and slots; everything else inherits
production. The tap is **read-bound**: a 7,168-token document cut into four 2,048-token steps
re-streams each layer's routed expert union four times, so a wider step is worth up to ~4× (the
runner's own header says this).

Wide steps are refused by the arch, not the budget, and the refusal scales with the window:

```
QSA score plane = rows x (--max-model-len / 4) x 4 bytes
16,384 rows @ 200,000 window -> 3,125 MiB  -> REFUSED at boot
16,384 rows @  32,768 window ->   512 MiB  -> legal
```

The calibration corpus is chunked to 7,168 tokens, so **32,768 is the window to calibrate at** —
copying production's 200,000 window is what disqualifies the wide step.

## Judging a run

- **A quiet log is not a hang.** The engine logs one line per *completed* request; at 2-way a 7.5 K
  prefill takes ~4 min. Watch `kv_tokens_used` (steps of the chunk size) and the `req cmpl` count.
  A watchdog on log mtime kills healthy cycles and throws away accumulated tokens.
- **Sealing is success.** The engine exits by itself; a mid-flight client gets
  `429 rate_limit_error: the server is shutting down`.
- **Kill by pid** (`sudo fuser /dev/kfd`), never `pkill -f`: an orphaned engine keeps the device and
  the next boot dies in ~20 s with `E startup stopped: device error`, which reads like a bad shape.