frank-secondary: record the deployed boot, keep the unit description honest

Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.

Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.

The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.

Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
Frankandomp committed 2026-10-07 17:33:21 -07:00
1 parent ec6184de9a
commit 5278f5be26
3 files changed
+42 -10

No files matched your search

+10 -7
View File
@@ -451,13 +451,16 @@ decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
property of the file, not of the flags. The publisher card advertises "MTP property of the file, not of the flags. The publisher card advertises "MTP
support" for the base architecture; the quantized export dropped the head. support" for the base architecture; the quantized export dropped the head.
**Measured boots** (nvidia-smi, the referee): the pricing boots are the **Measured boots** (nvidia-smi, the referee): the deployed boot is
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry — 2 × 200,704 f16 K + **21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
f16 V: **21,276 used / 2,849 free** (ubatch 1024) and **21,846 / 2,279** MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
(ubatch 2048). This file is 415 MiB lighter in weights ⇒ projected ~21,430 / ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
~2,700. The deployed boot ledger: see the runbook. Decode is ubatch-invariant bought back what the projection predicted. The engine ledger confirms the KV
(Swift-era 3090 law). The Q6_K rung costs ~10 % decode vs Q5_K_M (weight law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
bandwidth; MiMo-measured 94.4 vs 104.8 t/s). `V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
vs 104.8 t/s).
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is **Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
natively multimodal; the Q6_K file carries no vision tensors). Probe traps natively multimodal; the Q6_K file carries no vision tensors). Probe traps
+1 -1
View File
@@ -1,5 +1,5 @@
[Unit] [Unit]
Description=llama.cpp Swift-1.5-Qwen3.8-27B Uncensored RCO IQ3_XXS GGUF (qwen35 hybrid, 2x131072 slots, q8_0 K+V, draft-mtp depth 3, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary) Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
@@ -76,9 +76,30 @@ Ubatch 2048 is the measured CUDA optimum for this card and arch (+4.6 %
prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented
fallback rung if a future shape lands under the 2 GiB floor. fallback rung if a future shape lands under the 2 GiB floor.
## Measured boot (2026-10-07) ## Measured boot (2026-10-07, deploy `ec6184d`)
TO BE FILLED BY THE DEPLOY VERIFICATION. `nvidia-smi`: **21,436 MiB used / 2,689 MiB free** — the MiMo-priced
projection (~21,430 / ~2,700) was within 11 MiB. Engine ledger
(`--log-verbosity 4`, `journalctl -u frank-secondary`):
|Line|Value|
|---|---|
|`llama_context`|`n_ctx = 401408`, `n_ctx_seq = 200704`, `kv_unified = false`|
|`llama_kv_cache`|`CUDA0 KV buffer size = 12544.00 MiB` — `size = 12544.00 MiB (200704 cells, 8 layers, 2/2 seqs), K (f16): 6272.00 MiB, V (f16): 6272.00 MiB`|
|`sched_reserve`|`CUDA0 compute buffer size = 1136.06 MiB`, `CUDA_Host compute buffer size = 848.07 MiB`|
|`clip_model_loader` (mmproj)|`model size: 878.99 MiB` + `reserve_compute_meta: CUDA0 248.10 MiB, CPU 24.93 MiB`|
|`srv load_model`|`n_slots = 2, n_ctx_slot = 200704`|
|`print_info`|`n_ctx_train = 262144`, `n_head_kv = 4`|
The `32 KiB/token` law is confirmed exactly: 401,408 × 32 KiB = 12,544 MiB,
which is what the engine printed. The `n_ctx_seq (200704) < n_ctx_train
(262144)` line is informational (a context **below** the trained window is
safe; the Gemma4 era's warning was the opposite case, a drafter running 53 %
**past** its window). Weights load in ~6 s from page cache; `/health` answers
`{"status":"ok"}` and `/v1/models` serves the alias `frank-secondary`.
The deploy's Playwright gate passed (33 tests) and the fleet e2e set is
unchanged.
## Client contract ## Client contract
@@ -111,3 +132,11 @@ one slot, not the pool; queue behind two long generations ⇒ the slot ceiling
is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in
`journalctl -u frank-secondary` (KV MiB, compute buffers) before touching `journalctl -u frank-secondary` (KV MiB, compute buffers) before touching
flags — the engine prints the exact figures the shape needs. flags — the engine prints the exact figures the shape needs.
**`status=203/EXEC` boot loop**: the rewrite of `serve-frank-secondary.sh`
shipped without its `#!/usr/bin/env bash` line, and systemd failed the
`ExecStart` before the script ran — 10 s restarts, `Mem peak: 1.9M`, and
`nvidia-smi` flat at 1 MiB used (the GPU is never touched). Journal signature:
`Failed at step EXEC spawning /home/dank/…/serve-frank-secondary.sh`. A
shebang is not decoration on this host: every LLM serve script is a direct
`ExecStart`, so `head -1` is part of the deploy check (commit `ec6184d`).