Files
serverAppConfigs/serverAppConfigs-docs/LLM
Frank de4d22785d frank-primary runbook: record the KV law and per-ctx VRAM costs
29,696 B/token ctx-scaled (12 of 48 layers carry KV - full_attention_interval=4;
indexer and draft KV included); weights/RS/compute are ctx-independent.
262k = 18.75 GiB total / 9.37 per card; 100k = 14.3 / 7.1. Parallel
slots split the pool rather than multiplying it.
2026-09-26 21:01:12 -07:00
..