Dankserver System Specs
Server Overview
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.04 (Resolute Raccoon) with an RTX 3090 and 62 GB of system RAM.
Hardware Specifications
CPU
| Property |
Value |
| Architecture |
x86_64 |
| Vendor |
AuthenticAMD |
| Model |
AMD Ryzen 7 7700X |
| Cores |
8 (1 socket) |
| NUMA Nodes |
1 (node 0, 63503 MB) |
GPU
| Property |
Value |
| Model |
NVIDIA GeForce RTX 3090 |
| VRAM |
24576 MiB (24 GB) |
| Driver |
595.84 |
| CUDA Version |
13.2 |
| CUDA Toolkit |
12.0 (nvcc) |
Memory
| Property |
Value |
| System RAM |
62 GiB |
| Swap |
127 GiB |
| /dev/shm |
32 GiB |
Storage
| Filesystem |
Size |
Used |
Avail |
Use% |
| / (nvme0n1p2) |
915G |
729G |
140G |
84% |
| /mnt/entropy (dm-0) |
91T |
86T |
910G |
99% |
Network
| Interface |
Type |
Speed |
IP |
Notes |
| eno1 |
onboard |
2500 Mb/s (2.5 GbE) |
192.168.0.235 (DHCP) |
RPC worker endpoint (port 50052). Default route via 192.168.0.1. |
| enxe8ea6ab7ebca |
USB-C class adapter |
2500 Mb/s |
192.168.0.252 |
UNPLUGGED 2026-08-17. Was idle (unused by the RPC path). |
| wlp8s0 |
WiFi |
- |
- |
Unused (carrier down). |
| docker0 |
bridge |
- |
172.17.0.1 |
Down; internal docker traffic only. |
OS and Software
| Property |
Value |
| OS |
Ubuntu 26.04 LTS (Resolute Raccoon) |
| Kernel |
7.0.0-30-generic |
| Python |
3.14.4 |
Active Services
The active service on Dankserver is frank-pp-worker.service: the ggml-rpc worker (RTX 3090) that receives the PP head's tensor share (45 of 65 blocks, ~23.8 GiB of 24 GiB at steady state). The old llama.cpp Qwen3.6-35B-A3B service (frank-default-small) is retired (2026-08-16) and disabled (2026-08-17). See Qwen3.8 PP Deployment.
Hardware Relevance
- 24 GB VRAM is close to saturated with this config. At idle ~22.1 GB used, leaving ~2.0 GiB headroom at
--n-cpu-moe 36 (4 layers' experts on GPU) with ubatch 3072 and a dual-f16 KV pool of 13680 MiB at 700416 cells (2026-08-14 refit; the same pool at 800768 cells OOM'd the load warmup). A llama-bench sweep (frank-default-small-benchmark) guided this state: prefill is the priority, so ubatch 3072 stays (top prefill tier), and ncmoe 36 was raised from 35 to fix a cuBLAS OOM-class crash during big prefills by freeing ~0.75 GiB peak VRAM at only -2% prefill. If it still crashes under load, add GGML_CUDA_FORCE_MMQ=1. If decode speed matters more, use ncmoe 32 - ubatch 1024 (gen ~48.7 t/s). The tuning guidance applied to the retired frank-default-small service; the PP worker's allocations are fixed at load.
- 62 GB system RAM holds the ~29 GB of MoE expert weights with room to spare (~57 GB available after load).
- 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
- Root disk has approximately 140 GB free. Store the GGUF models under
/home/dank/models (about 32 GB total).
See Also
