# Dankserver System Specs

## Server Overview

Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 24.04 with an RTX 3090 and 62 GB of system RAM.

## Hardware Specifications

### CPU

| Property | Value |
|----------|-------|
| Architecture | x86_64 |
| Vendor | AuthenticAMD |
| Model | AMD Ryzen 7 7700X |
| Cores | 8 (1 socket) |
| NUMA Nodes | 1 (node 0, 63503 MB) |

### GPU

| Property | Value |
|----------|-------|
| Model | NVIDIA GeForce RTX 3090 |
| VRAM | 24576 MiB (24 GB) |
| Driver | 595.84 |
| CUDA Version | 13.2 |
| CUDA Toolkit | 12.0 (nvcc) |

### Memory

| Property | Value |
|----------|-------|
| System RAM | 62 GiB |
| Swap | 127 GiB |
| /dev/shm | 32 GiB |

### Storage

| Filesystem | Size | Used | Avail | Use% |
|------------|------|------|-------|------|
| / (nvme0n1p2) | 915G | 729G | 140G | 84% |
| /mnt/entropy (dm-0) | 91T | 86T | 910G | 99% |

### OS and Software

| Property | Value |
|----------|-------|
| OS | Ubuntu 24.04.4 LTS (Noble Numbat) |
| Kernel | 7.0.0-28-generic |
| Python | 3.12.3 |

### Active Services

The active `frank-default-small` service (llama.cpp, `b10108+`) serves the HauhauCS Qwen3.6-35B-A3B uncensored `Q6_K_P` model on port 7770 with **ngram-mod** speculative decoding. It runs **4 parallel slots** at 200000 context each (pool 800000). Of the 40 layers' MoE expert weights, **35 sit on CPU** (`--n-cpu-moe 35`) while the remaining **5 layers' expert weights** are resident on GPU along with all non-expert tensors (`--gpu-layers all`). KV stays on GPU with K=`q8_0`, V=`f16`. Batch 8192 / ubatch 3072, `--ctx-checkpoints 4`, `--no-mmap`. At idle ~22.8 GB VRAM used, leaving an estimated **~1.2 GiB** headroom (chosen by benchmark for prefill speed, see [[frank-default-small-benchmark]]). Idle RAM ~32 GB of 62 GB.

## Hardware Relevance

- 24 GB VRAM is close to saturated with this config. At idle ~22.8 GB used, leaving an estimated **~1.2 GiB** headroom at `--n-cpu-moe 35` (5 layers' experts on GPU) with `ubatch 3072`. A llama-bench sweep ([[frank-default-small-benchmark]]) chose this state because prefill is the priority: `ubatch 3072` reaches ~1458 t/s prefill, the top tier, and generation is independent of ubatch. If decode speed matters more, use `ncmoe 32 - ubatch 1024` (gen ~48.7 t/s). By editing `scripts/frank-default-small.env`.
- 62 GB system RAM holds the ~29 GB of MoE expert weights with room to spare (~57 GB available after load).
- 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
- Root disk has approximately 140 GB free. Store the GGUF models under `/home/dank/models` (about 32 GB total).

## See Also

- [[VLLM Overview]]
- [[Hybrid CPU GPU Processing Dankserver]]
- [[RAM CPU Offloading in VLLM]]
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]