frank-primary: record measured battery numbers and the drafter-placement law

Battery 6/6: alias, reasoning_content, tool_calls, vision (probe image
described exactly), reasoning_effort high -> 200. Decode 11.8 t/s at
depth-0 prose (draft 109/accepted 72), 0.864 mean len 2.73 on tool
traffic. VRAM 29%/36% (output-head card carries the drafter). Also
records the ROCm1 drafter law in the runbook body (borrowed output.weight
+ n_copies==1 abort) and the two cutover traps: foreign-process health
false green, and llama-server SIGTERM cleanup hang.
This commit is contained in:
Frank
2026-09-26 20:55:02 -07:00
parent 5d632bbaea
commit bacabdcd49
@@ -72,7 +72,14 @@ Lucebox DS4.1 quadlet that held the port for hours earlier the same day
draft load with `invalid vector subscript`). `ngram-mod` is draftless,
so it chains legally behind the drafter. Depth 2: each longer
verification round re-batches the CPU expert reads, so depth buys less
than bandwidth here.
than bandwidth here. **The drafter must sit on ROCm1** — the head also
borrows the target's `output.weight`, which a `-ts 1,1` split places on
the last-layers card; with tensor overrides set the draft scheduler runs
`n_copies==1`, and a draft context elsewhere aborts at load
(`ggml-backend.cpp:941`, "pre-allocated tensor (output.weight) in a
buffer (ROCm1) that cannot run the operation"). The 2026-09-26 cutover
crash-looped 4× on `ROCm0` before the probe confirmed `ROCm1` loads —
the frank-medium precedent (drafter on the last-layers card) was right.
- **Vision, explicitly.** This build never autoloads projectors — without
`--mmproj` the server comes up healthy and text-only and answers image
requests with 500 (the frank-medium 2026-09-17 trap). The BF16 projector
@@ -118,7 +125,23 @@ truly free, relinks + enables + restarts the root unit, and waits up to
- VRAM/RAM headroom: `rocm-smi` per card + `free -g` after the first long
completion.
_Measured numbers land here after the deploy battery._
_**Measured (2026-09-26 battery, 6/6 pass).** Load: ~7 s warm (mmap from
page cache; the fetch's own verify pass pre-warms it — a cold first load
pages in 175 GiB). Decode: **11.8 t/s** `predicted_per_second` at
depth-0 prose, 128-token generation, draft_n 109 / accepted 72 (0.66 on
reasoning-heavy prose; **0.864 mean len 2.73** on the tool-call task).
Vision: the probe image (green square centered on a red field) described
exactly. VRAM: GPU0 29% / GPU1 36% of 32 GiB (~9.3 / ~11.5 GiB — the
output-head card carries the drafter, as designed; ~22 GiB free per
card). RAM: ~11 GiB used, ~226 GiB page cache (the payload itself),
available ~235 of 246 GiB. `reasoning_effort: high` → 200. Two cutover
traps for the record: a LAST-ACK tail from a LAN client held
`wait_port_free` until it cleared (a foreign process on the port gates
the deploy correctly — but the deployer's `/health` wait cannot tell WHO
answers; after any aborted deploy, confirm the responder is the unit's
own MainPID), and llama-server's SIGTERM cleanup hung after
spec-decode service (logged `cleaning up before exit` and never exited —
`kill -9` was needed; `systemctl stop` relies on TimeoutStopSec)._
## Rollback