Dankserver System Specs
Server Overview
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 24.04 with an RTX 3090 and 62 GB of system RAM.
Hardware Specifications
CPU
| Property |
Value |
| Architecture |
x86_64 |
| Vendor |
AuthenticAMD |
| Model |
AMD Ryzen 7 7700X |
| Cores |
8 (1 socket) |
| NUMA Nodes |
1 (node 0, 63503 MB) |
GPU
| Property |
Value |
| Model |
NVIDIA GeForce RTX 3090 |
| VRAM |
24576 MiB (24 GB) |
| Driver |
595.84 |
| CUDA Version |
13.2 |
| CUDA Toolkit |
12.0 (nvcc) |
Memory
| Property |
Value |
| System RAM |
62 GiB |
| Swap |
127 GiB |
| /dev/shm |
32 GiB |
Storage
| Filesystem |
Size |
Used |
Avail |
Use% |
| / (nvme0n1p2) |
915G |
729G |
140G |
84% |
| /mnt/entropy (dm-0) |
91T |
86T |
910G |
99% |
OS and Software
| Property |
Value |
| OS |
Ubuntu 24.04.4 LTS (Noble Numbat) |
| Kernel |
7.0.0-28-generic |
| Python |
3.12.3 |
Active Services
The active frank-default-small service (llama.cpp, b10108+) serves the HauhauCS Qwen3.6-35B-A3B uncensored Q6_K_P model on port 7770 with ngram-mod speculative decoding. It runs 4 parallel slots at 175104 context each (pool 700416, refitted 2026-08-14 for the dual-f16 KV cache). Of the 40 layers' MoE expert weights, 36 sit on CPU (--n-cpu-moe 36) while the remaining 4 layers' expert weights are resident on GPU along with all non-expert tensors (--gpu-layers all). KV stays on GPU with K=f16, V=f16 (13680 MiB pool at 700416 cells). Batch 8192 / ubatch 3072, --ctx-checkpoints 4, --no-mmap. At idle ~22.1 GB VRAM used, leaving ~2.0 GiB headroom. Idle RAM ~32 GB of 62 GB.
Hardware Relevance
- 24 GB VRAM is close to saturated with this config. At idle ~22.1 GB used, leaving ~2.0 GiB headroom at
--n-cpu-moe 36 (4 layers' experts on GPU) with ubatch 3072 and a dual-f16 KV pool of 13680 MiB at 700416 cells (2026-08-14 refit; the same pool at 800768 cells OOM'd the load warmup). A llama-bench sweep (frank-default-small-benchmark) guided this state: prefill is the priority, so ubatch 3072 stays (top prefill tier), and ncmoe 36 was raised from 35 to fix a cuBLAS OOM-class crash during big prefills by freeing ~0.75 GiB peak VRAM at only -2% prefill. If it still crashes under load, add GGML_CUDA_FORCE_MMQ=1. If decode speed matters more, use ncmoe 32 - ubatch 1024 (gen ~48.7 t/s). By editing scripts/frank-default-small.env.
- 62 GB system RAM holds the ~29 GB of MoE expert weights with room to spare (~57 GB available after load).
- 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
- Root disk has approximately 140 GB free. Store the GGUF models under
/home/dank/models (about 32 GB total).
See Also
