dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-secondary: record the deployed boot, keep the unit description honest

Browse Source 
Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.

Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.

The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.

Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
Frank
2026-10-07 17:33:21 -07:00
co-authored by omp
parent ec6184de9a
commit 5278f5be26
3 changed files with 42 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+10 -7
View File
@@ -451,13 +451,16 @@ decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
property of the file, not of the flags. The publisher card advertises "MTP
support" for the base architecture; the quantized export dropped the head.
**Measured boots** (nvidia-smi, the referee): the pricing boots are the
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry — 2 × 200,704 f16 K +
f16 V: **21,276 used / 2,849 free** (ubatch 1024) and **21,846 / 2,279**
(ubatch 2048). This file is 415 MiB lighter in weights ⇒ projected ~21,430 /
~2,700. The deployed boot ledger: see the runbook. Decode is ubatch-invariant
(Swift-era 3090 law). The Q6_K rung costs ~10 % decode vs Q5_K_M (weight
bandwidth; MiMo-measured 94.4 vs 104.8 t/s).
**Measured boots** (nvidia-smi, the referee): the deployed boot is
**21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
bought back what the projection predicted. The engine ledger confirms the KV
law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
`V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
vs 104.8 t/s).
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
natively multimodal; the Q6_K file carries no vision tensors). Probe traps