docs(vault): the work log keeps incidents and laws; the numbers move to the page that owns them

log.md drops 466 lines of per-swap ledger prose (1,169 lines out, 703 back) and
keeps the causal stories: why the host went down, why a knob resets at boot, why
a stale drop-in served a different model for 24 minutes, why the RAM was the
fault. Date sections are unchanged (65 before, 65 after) -- this is compression
inside sections, not deletion of days.

Every number the compression would have orphaned was ported to its owner first,
and a census of the deleted lines against the whole vault is what found them:

- frank-zealot: the two boots that priced non-KV (20,190/KV 12,800 => 7,390
  without the projector; 22,972/KV 14,113 => 8,859 with it, so the vision tower
  costs ~1.4 GiB for a 918 MB file), the 1,153 MiB-free 3-slot boot that made
  the seam price non-KV at 9 GiB, and the f16 wall's own figures (fit 33,198 vs
  23,558 free on a 25,088.00 MiB KV request).
- frank-archon: the pool-cut-is-headroom-not-arena law (2.75 GiB/rank bought
  zero slots; d(reserve) = d(arena) = +248 slots; 14.87 GiB / 6,496 slots
  against a 398,658-token pool), the MNBT compile-RANGE vs SEQS+1 law, and the
  2 x 196,919 prefix-cache proof (393,837 tokens resident above the pool's
  386,904; 94.1/102.3/117.6 t/s, acceptance 0.7284/0.7525).
- frank-dragoon: the KL arm of the 2026-10-17 A/B (mean 0.155 / median 0.057,
  top-1 agreement 85.1 %) joins the PPL and NIAH figures already on the era row.
- Payload and Disk Reclaim: the 121 GB layer census lesson -- a huge image layer
  can be the engine image, not a payload.

Also fixes a wikilink the rewrite had hard-wrapped across two lines, which the
link checker reads as a target that does not exist.

New seam: 'one page per live LLM endpoint'. The live set is derived from
scripts/ (an endpoint has both an env and a unit), and any <endpoint>-*.md page
in the vault fails it -- the enforcement of the one-page-per-endpoint law, so a
re-created era page cannot pass tests.
This commit is contained in:
Frank committed 2026-10-19 01:05:32 -07:00
1 parent 6bdfd2853e
commit 846f81ddb8
6 files changed
+763 -1170

No files matched your search

@@ -118,6 +118,9 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
- **Addressable ≠ advertised.** The card is 24,576 MiB but leaves **24,123 MiB**: quote `nvidia-smi`'s `free` field, never `total − used` — the driver's reservation is the difference.
- **Exact KV and a 786,432 pool cannot coexist here**: 32,768 B/token ⇒ 24,576 MiB before a single weight loads (the retired "26,112 MiB" note doubled q8_0's scale overhead); the 27B analogue died at **25,088 MiB** with `cudaMalloc failed: out of memory`. Halving the pool funds exact KV; the per-slot window stops at the trained 262,144 (no YaRN here), so a third conversation is a KV-dtype decision, never more context.
The wall's own figures: `cudaMalloc` failed on a **25,088.00 MiB** KV request when the engine's fit projection
asked for **33,198 MiB** against **23,558 MiB** free. It booted f16 at all because the arm env's invented
`KV_CACHE_TYPE` was swallowed by the serve script's `:=f16` default — so a failed window became the measurement.
- **Exact KV reads 1.882× the bytes per generated token, not 2×** (32,768 vs 17,408 B/token, 2026-10-18): a decode drop of that order is EXPECTED and must be measured, while prefill should hold — and this service can afford the dtype precisely because its load is the compaction hub's ~170k-token prefill, which is weight-bandwidth bound and never re-reads the cache per generated token. The dtype is paid in decode and concurrency.
- Never `--fit` with `parallel > 1` — it maximizes the *total* pool and shrinks each slot: set `CTX_SIZE` explicitly and keep it divisible by `PARALLEL` (the seam enforces it).
- **KV rate law.** `full_layers × 4 KV heads × 256 dim` = **8,192 elements per token per cache** on this arch (the engine prints `n_embd_k_gqa = n_embd_v_gqa = 1024`), ×2 B (f16), ×1.0625 (q8_0 — 32 codes plus one 2-byte scale
@@ -133,6 +136,11 @@ re-run the same arms with the box quiet before a figure is carried anywhere else
`tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
projections drift.
The two boots that priced non-KV: 2 × 262,144 **without** the projector — 20,190 used, KV 12,800 ⇒ non-KV
**7,390 MiB**; 3 × 196,608 **with** it — 22,972 used, KV 14,113 ⇒ non-KV **8,859 MiB**. The difference prices the
vision tower at **~1.4 GiB for a 918 MB file**, because the encoder's activation and compute buffers ride with it.
The first 3-slot deploy (3 × 196,608) booted healthy and left only **1,153 MiB** free — that is how the 2–3 GiB
band got violated in practice, and why the seam prices non-KV at the measured 9 GiB and refuses that shape.
- **Ubatch is a prefill knob.** Decode never moves with it and the logical `--batch-size` is neutral; prefill climbs monotonically to **2048** on this CUDA card (+4.6 % at 8.9k depth; 710 → 1,188 t/s), a prompt shorter than the ubatch cannot see the knob at all (the control row), and a verdict never crosses backends — the 2× R9700 split peaks at 512. Each rung costs ~0.6–0.75 GiB of compute buffers; 2048 is the ceiling for a 27B-on-3090 shape.
- **A quant rung is paid in decode**: Q5_K_M → Q6_K is +875 MiB of weights and **−10 %** rate on this bandwidth-bound card, and the byte-bandwidth law predicts a rung's rate within 1 %.
- **Speculation pays only with room on the card.** The 2026-10-18 arm census (all q8_0 K+V at the 786,432 pool): 3 × 262,144 spec-OFF ubatch 2048 → 17,876 / 6,249 free, 167.2 t/s; spec-ON ubatch 2048 → 22,706 / 1,419, 109.2 t/s; spec-ON ubatch 1024 → 21,400 / 2,725, 213.5 t/s; 2 × 262,144 spec-ON → 213.2; 1 × 262,144 spec-ON → 214.2. That is **+28 % decode whenever the card has room**, and the doors back are one window per conversation, or q8_0 KV with ubatch 1024. The draft's own bill at that shape was 3,072.00 MiB of KV plus ~1,758 MiB of buffers.