Files

T

Frank 8212d4451a docs: rename frank-default-large/small/sightsound to frank-large/small/npu across vault

Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.

2026-09-03 16:17:14 -07:00

6.3 KiB

Raw Blame History

SUPERSEDED 2026-08-17. The Frank redo retired this topology. See frank-redo-2026-08-17.md and deployment.md for the active configuration.

type: reference title: Dankserver VRAM and RAM Growth Under Load description: Investigation of the recurring cuBLAS crash on frank-small. What grows, what does not, and what to do next. tags:

dankserver
llamacpp
vram
ram
ctx-checkpoints
crash

Dankserver VRAM and RAM Growth Under Load

This page records the investigation of the recurring cuBLAS crash on frank-small (llama.cpp, RTX 3090 24 GiB). The service aborts with CUDA error: an unsupported value or parameter was passed to the function in cublasSgemm_v2 at ggml-cuda.cu:1541 during update_slots -> decode -> process_ubatch. It crashed on 2026-08-13 at 19:45 and 19:55 and on 2026-08-14 at 12:00:43.

What Was Ruled Out

ctx-checkpoints do not grow VRAM

--ctx-checkpoints 4 stores context checkpoints in host RAM. The log records every checkpoint at exactly 62.813 MiB. The size never grows. The reasoning:

common_prompt_checkpoint (common/common.h) stores the serialized KV state in std::vector<uint8_t> fields. These live on the host heap.
update_tgt() and update_dft() call llama_state_seq_get_data_ext with LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY. The serialized state moves from VRAM to RAM for the snapshot and back on restore.
With 4 slots and --ctx-checkpoints 4, the maximum checkpoint RAM is 16 x 62.813 MiB, about 1.0 GiB. That is fixed and small.
data_spec (spec-decode state) is empty for ngram-mod. The ngram-mod implementation does not override get_state().

ngram-mod does not grow VRAM

ngram-mod allocates one fixed 64 MiB CPU hash table (common_ngram_mod::entries, 16M int32 entries). It never resizes. The matcher keeps about 12 bytes per sequence. Draft generation runs on the CPU. The speculative path allocates zero GPU memory.

The CUDA graph cache has eviction in this build

ggml_backend_cuda_context::cuda_graph() sweeps every 5 seconds and evicts graphs unused for 10 seconds. Upstream issue #25082 describes an unbounded graph cache, but build b10108-13-g555881ebc includes the eviction sweep. This build does not exhibit that leak.

What Grows

RAM growth is suspected, not yet confirmed

systemd reported MemoryPeak 46.9 GiB for the crashed process and 45.9 GiB for the replacement. RSS is flat at 32.3 GiB, which matches the documented idle baseline of about 32 GiB. The difference is file-backed page cache from the --no-mmap model read; cgroup memory.current counts file pages attributed to the process. That gap is not proof of a leak by itself.

The upstream pattern to watch is issue #26216: the prompt cache and checkpoint accumulation exceed the --cache-ram cap and do not plateau. The environment file does not set --cache-ram, so the llama.cpp default of 8192 MiB applies. The monitor logs RSS, which separates anonymous memory from file cache. If RSS climbs under load while VRAM is flat, the RAM-side accumulation is real.

VRAM suspects under load

VRAM is flat at idle: 21,268 MiB across every monitor sample. The crash correlates with load density and slot context size (143K tokens on slot 2), not with wall-clock time. The two mechanisms worth testing:

Flash-attention F16 K conversion workspace. The prefill kernel on Ampere is MMA_F16. It converts the quantized K cache (q8_0) to F16 for the attention math. The workspace size scales with the attended context length (ggml_nelements(K) x 2). It is allocated into the graph buffer via ggml_cuda_flash_attn_ext_get_alloc_size.
The CUDA VMM pool grows monotonically. ggml_cuda_pool_vmm::alloc only reserves more physical memory; free only releases the reservation slot. The pool never shrinks during the process lifetime. Any new peak allocation permanently expands VRAM usage.

Both match the observed pattern: growth under prefill bursts at deep context, flat at idle, crash when a burst exceeds the ~2.8 GiB headroom.

Monitoring

The monitor runs on Dankserver:

/home/dank/vram-monitor.sh
/home/dank/vram-log.csv

It samples GPU VRAM and process RAM every 10 seconds and appends a CSV row. The columns are timestamp, uptime, GPU VRAM MiB, GPU utilization, process RSS, process VSZ, and systemd MemoryPeak.

Resolution: dual-f16 KV Cache (2026-08-14)

The fix chose the first mechanism over the second. Both K and V moved to f16 on 2026-08-14. When K is already f16, the flash-attention MMA_F16 prefill kernel needs no F16 conversion copy, so the context-proportional workspace disappears. The pool becomes static: 20,480 bytes per cell (10 full-attention layers x 2 kv heads x 256 dims x 2 bytes for K plus 2 bytes for V).

The pool at 800768 cells cost 15,640 MiB and did not fit. The load-time warmup graph (1472x1472 image) failed cudaMalloc by 248 MiB, and the service idled at 23,966 of 24,576 MiB. The context pool was refitted to 700416 cells (4 slots x 175104): KV pool 13,680 MiB, idle 22,110 MiB, about 2.0 GiB free after warmup. Parallel stays 4.

Verified 2026-08-14 after deploy: flash attention enabled, warmup clean, decode about 42-44 t/s, cached prompt tokens working. The vram monitor log shows the restart arc (23,966 -> 5,210 during load -> 22,110 idle).

Mitigation Options

The documentation for the benchmark (frank-small-benchmark) prescribes the first option for this crash class. None of these change the model.

Option	Effect	Cost
`GGML_CUDA_FORCE_MMQ=1`	Bypasses the cuBLAS path in the crashing function	Slight decode slowdown
`--cache-ram 2048`	Bounds the confirmed RAM growth	Cache misses on prompt reuse
`--ctx-checkpoints 2`	Cuts checkpoint RAM and save cost in half	More prefill work after eviction
Apply PR #24649	Clears slot checkpoints before the prompt-cache save	Needs a llama.cpp rebuild

Upstream references: issue #26216 (prompt cache growth), issue #25082 (graph cache, not applicable to this build), issue #23635 (quantized KV with speculative decoding), PR #24649 (checkpoint-to-cache duplication), PR #25649 (slot checkpoints freed on prompt clear, merged), issue #26119 (same cuBLAS signature, confirmed on V100 with speculative decoding).

6.3 KiB Raw Blame History