Docs-only follow-up to the merged reslice (#25). No behavior change; deploy.sh treats a docs-only diff as a no-op.
Records the boot of record for ba69ea5 (verbosity-4 ledger + settled nvidia-smi): target KV 14,406.00 MiB (K f16 9,408.00 + V q8_0 4,998.00), draft KV 2,352.00 MiB over the same cells, recurrent state 452.25, compute 1,665.00 + 424.88, weights 2,283.29 + 257.66, projector 595.29 + 248.10 + 24.93 — settled 22,870 used / 1,255 free, n_slots = 3, n_ctx_slot = 200704.
Two carried numbers did not survive it and are corrected in the env header, the serve rationale and the page:
UBATCH_SIZE 2048 → 1024 was quoted at "−~1,306 MiB"; that belonged to the 786,432 / q8_0-pair arm. Here the target's compute buffers moved 2,496.13 → 2,089.88 — 406 MiB, CUDA0 up 257 while CUDA_Host down 663. The drafter is funded by CACHE_TYPE_V=q8_0 (4,410 MiB, not the 3,840 previously written), alone.
The seam's fit gate priced 22,428 against a real 22,870: conservative by 442 MiB, so it stays as written.
One finding stated plainly: this is the first boot on this card that misses llama.cpp's own free-memory target — the engine printed cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB and settled at 1,255 MiB free, under the ~2 GiB band every previous era held. Ordered shape kept; the cheapest rungs back into the band are CTX_SIZE=401408 (two conversations) or SPEC_TYPE=none, one line each.
Also records that the drafter is engaged: acceptance 0.706 / 0.763, mean accepted length 2.41 / 2.53, and the three swap probes (chat / tools / vision) with their exact evidence. ./tests/run.sh green (449).
Harness note (outside the repo): ~/.omp/agent/models.ymlcontextWindow moved 262144 → 200704 to match the per-slot window.
Docs-only follow-up to the merged reslice (#25). No behavior change; `deploy.sh` treats a docs-only diff as a no-op.
Records the boot of record for `ba69ea5` (verbosity-4 ledger + settled `nvidia-smi`): target KV **14,406.00 MiB** (K f16 9,408.00 + V q8_0 4,998.00), draft KV **2,352.00 MiB** over the same cells, recurrent state 452.25, compute 1,665.00 + 424.88, weights 2,283.29 + 257.66, projector 595.29 + 248.10 + 24.93 — settled **22,870 used / 1,255 free**, `n_slots = 3`, `n_ctx_slot = 200704`.
Two carried numbers did not survive it and are corrected in the env header, the serve rationale and the page:
- `UBATCH_SIZE` 2048 → 1024 was quoted at "−~1,306 MiB"; that belonged to the 786,432 / q8_0-pair arm. Here the target's compute buffers moved 2,496.13 → 2,089.88 — **406 MiB**, CUDA0 up 257 while CUDA_Host down 663. The drafter is funded by `CACHE_TYPE_V=q8_0` (**4,410 MiB**, not the 3,840 previously written), alone.
- The seam's fit gate priced 22,428 against a real 22,870: conservative by 442 MiB, so it stays as written.
One finding stated plainly: this is the first boot on this card that misses llama.cpp's own free-memory target — the engine printed `cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB` and settled at **1,255 MiB free**, under the ~2 GiB band every previous era held. Ordered shape kept; the cheapest rungs back into the band are `CTX_SIZE=401408` (two conversations) or `SPEC_TYPE=none`, one line each.
Also records that the drafter is engaged: acceptance 0.706 / 0.763, mean accepted length 2.41 / 2.53, and the three swap probes (chat / tools / vision) with their exact evidence. `./tests/run.sh` green (449).
Harness note (outside the repo): `~/.omp/agent/models.yml` `contextWindow` moved 262144 → **200704** to match the per-slot window.
---
cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 7f352f4d0f041b0564688e79b954b10d1a536a0f52254402
The boot of record for the new shape, from the verbosity-4 journal plus
settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00),
draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state
452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29
+ 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255
free with n_slots = 3 / n_ctx_slot = 200704.
Two inherited numbers did not survive it, and both are corrected in the env,
the serve rationale and the page:
- The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the
786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went
2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host
went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is
what funds the drafter, alone.
- The fit gate priced 22,428 against a real 22,870: conservative by 442
MiB, so it stays honest as written.
And one finding stated plainly: this is the first boot on this card that
misses llama.cpp's own free-memory target - the engine printed "cannot meet
free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
and settled at 1,255 MiB free, under the ~2 GiB band every previous era
held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one
line.
Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of
thinking - empty content at max_tokens 64 is the documented trap), tools
(finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red ->
"Red", 280 prompt tokens), and the drafter's own statistics (acceptance
0.706 / 0.763, mean accepted length 2.41 / 2.53).
dank
merged commit b9d21ae947 into main2026-10-19 03:38:41 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Docs-only follow-up to the merged reslice (#25). No behavior change;
deploy.shtreats a docs-only diff as a no-op.Records the boot of record for
ba69ea5(verbosity-4 ledger + settlednvidia-smi): target KV 14,406.00 MiB (K f16 9,408.00 + V q8_0 4,998.00), draft KV 2,352.00 MiB over the same cells, recurrent state 452.25, compute 1,665.00 + 424.88, weights 2,283.29 + 257.66, projector 595.29 + 248.10 + 24.93 — settled 22,870 used / 1,255 free,n_slots = 3,n_ctx_slot = 200704.Two carried numbers did not survive it and are corrected in the env header, the serve rationale and the page:
UBATCH_SIZE2048 → 1024 was quoted at "−~1,306 MiB"; that belonged to the 786,432 / q8_0-pair arm. Here the target's compute buffers moved 2,496.13 → 2,089.88 — 406 MiB, CUDA0 up 257 while CUDA_Host down 663. The drafter is funded byCACHE_TYPE_V=q8_0(4,410 MiB, not the 3,840 previously written), alone.One finding stated plainly: this is the first boot on this card that misses llama.cpp's own free-memory target — the engine printed
cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiBand settled at 1,255 MiB free, under the ~2 GiB band every previous era held. Ordered shape kept; the cheapest rungs back into the band areCTX_SIZE=401408(two conversations) orSPEC_TYPE=none, one line each.Also records that the drafter is engaged: acceptance 0.706 / 0.763, mean accepted length 2.41 / 2.53, and the three swap probes (chat / tools / vision) with their exact evidence.
./tests/run.shgreen (449).Harness note (outside the repo):
~/.omp/agent/models.ymlcontextWindowmoved 262144 → 200704 to match the per-slot window.cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 7f352f4d0f041b0564688e79b954b10d1a536a0f52254402
ba69ea5boot ledger and correct the carried credits 30e8ed55a0The boot of record for the new shape, from the verbosity-4 journal plus settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00), draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state 452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29 + 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255 free with n_slots = 3 / n_ctx_slot = 200704. Two inherited numbers did not survive it, and both are corrected in the env, the serve rationale and the page: - The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the 786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went 2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is what funds the drafter, alone. - The fit gate priced 22,428 against a real 22,870: conservative by 442 MiB, so it stays honest as written. And one finding stated plainly: this is the first boot on this card that misses llama.cpp's own free-memory target - the engine printed "cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB" and settled at 1,255 MiB free, under the ~2 GiB band every previous era held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one line. Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of thinking - empty content at max_tokens 64 is the documented trap), tools (finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red -> "Red", 280 prompt tokens), and the drafter's own statistics (acceptance 0.706 / 0.763, mean accepted length 2.41 / 2.53).