frank-secondary: record the deployed boot, keep the unit description honest

Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.

Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.

The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.

Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
Frank
2026-10-07 17:33:21 -07:00
co-authored by omp
parent ec6184de9a
commit 5278f5be26
3 changed files with 42 additions and 10 deletions
+10 -7
View File
@@ -451,13 +451,16 @@ decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
property of the file, not of the flags. The publisher card advertises "MTP
support" for the base architecture; the quantized export dropped the head.
**Measured boots** (nvidia-smi, the referee): the pricing boots are the
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry — 2 × 200,704 f16 K +
f16 V: **21,276 used / 2,849 free** (ubatch 1024) and **21,846 / 2,279**
(ubatch 2048). This file is 415 MiB lighter in weights ⇒ projected ~21,430 /
~2,700. The deployed boot ledger: see the runbook. Decode is ubatch-invariant
(Swift-era 3090 law). The Q6_K rung costs ~10 % decode vs Q5_K_M (weight
bandwidth; MiMo-measured 94.4 vs 104.8 t/s).
**Measured boots** (nvidia-smi, the referee): the deployed boot is
**21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
bought back what the projection predicted. The engine ledger confirms the KV
law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
`V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
vs 104.8 t/s).
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
natively multimodal; the Q6_K file carries no vision tensors). Probe traps
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit]
Description=llama.cpp Swift-1.5-Qwen3.8-27B Uncensored RCO IQ3_XXS GGUF (qwen35 hybrid, 2x131072 slots, q8_0 K+V, draft-mtp depth 3, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
After=network-online.target
Wants=network-online.target
@@ -76,9 +76,30 @@ Ubatch 2048 is the measured CUDA optimum for this card and arch (+4.6 %
prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented
fallback rung if a future shape lands under the 2 GiB floor.
## Measured boot (2026-10-07)
## Measured boot (2026-10-07, deploy `ec6184d`)
TO BE FILLED BY THE DEPLOY VERIFICATION.
`nvidia-smi`: **21,436 MiB used / 2,689 MiB free** — the MiMo-priced
projection (~21,430 / ~2,700) was within 11 MiB. Engine ledger
(`--log-verbosity 4`, `journalctl -u frank-secondary`):
|Line|Value|
|---|---|
|`llama_context`|`n_ctx = 401408`, `n_ctx_seq = 200704`, `kv_unified = false`|
|`llama_kv_cache`|`CUDA0 KV buffer size = 12544.00 MiB` — `size = 12544.00 MiB (200704 cells, 8 layers, 2/2 seqs), K (f16): 6272.00 MiB, V (f16): 6272.00 MiB`|
|`sched_reserve`|`CUDA0 compute buffer size = 1136.06 MiB`, `CUDA_Host compute buffer size = 848.07 MiB`|
|`clip_model_loader` (mmproj)|`model size: 878.99 MiB` + `reserve_compute_meta: CUDA0 248.10 MiB, CPU 24.93 MiB`|
|`srv load_model`|`n_slots = 2, n_ctx_slot = 200704`|
|`print_info`|`n_ctx_train = 262144`, `n_head_kv = 4`|
The `32 KiB/token` law is confirmed exactly: 401,408 × 32 KiB = 12,544 MiB,
which is what the engine printed. The `n_ctx_seq (200704) < n_ctx_train
(262144)` line is informational (a context **below** the trained window is
safe; the Gemma4 era's warning was the opposite case, a drafter running 53 %
**past** its window). Weights load in ~6 s from page cache; `/health` answers
`{"status":"ok"}` and `/v1/models` serves the alias `frank-secondary`.
The deploy's Playwright gate passed (33 tests) and the fleet e2e set is
unchanged.
## Client contract
@@ -111,3 +132,11 @@ one slot, not the pool; queue behind two long generations ⇒ the slot ceiling
is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in
`journalctl -u frank-secondary` (KV MiB, compute buffers) before touching
flags — the engine prints the exact figures the shape needs.
**`status=203/EXEC` boot loop**: the rewrite of `serve-frank-secondary.sh`
shipped without its `#!/usr/bin/env bash` line, and systemd failed the
`ExecStart` before the script ran — 10 s restarts, `Mem peak: 1.9M`, and
`nvidia-smi` flat at 1 MiB used (the GPU is never touched). Journal signature:
`Failed at step EXEC spawning /home/dank/…/serve-frank-secondary.sh`. A
shebang is not decoration on this host: every LLM serve script is a direct
`ExecStart`, so `head -1` is part of the deploy check (commit `ec6184d`).