Frank
90c4ac69a7
Fanned out 5 read-only doc-auditor subagents (one per service group) that fact-checked the knowledge base against the live system + repo config, researched official setup docs online, and verified progress. Applied: - All 34 broken links fixed (real pages / external URLs / correct relative paths); 0 broken links remain. No .AGENTTRASH links (the 4 mentions are accurate descriptive text, not links). - Fact-check fixes: Readarr retired (May 2026), Radarr v4->v6, Calibre 9.2.1 live, Calibre-Web 0.6.25 live, Copyparty v1.20.21, LLM index ncmoe 38 + 786432 ctx, frank-default-small active (not frank-pp-worker), Hermes/SearXNG/Grafana hostnames, Loki/Alloy local images, Observability diagram (Promtail->Alloy), VLLM retired note, DFlash status, benchmark live-drift notes, Homepage test count 116.
77 lines
3.1 KiB
Markdown
77 lines
3.1 KiB
Markdown
# Dankserver System Specs
|
|
|
|
## Server Overview
|
|
|
|
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.04 (Resolute Raccoon) with an RTX 3090 and 62 GB of system RAM.
|
|
|
|
## Hardware Specifications
|
|
|
|
### CPU
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Architecture | x86_64 |
|
|
| Vendor | AuthenticAMD |
|
|
| Model | AMD Ryzen 7 7700X |
|
|
| Cores | 8 (1 socket) |
|
|
| NUMA Nodes | 1 (node 0, 63503 MB) |
|
|
|
|
### GPU
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Model | NVIDIA GeForce RTX 3090 |
|
|
| VRAM | 24576 MiB (24 GB) |
|
|
| Driver | 595.84 |
|
|
| CUDA Version | 13.2 |
|
|
| CUDA Toolkit | 12.0 (nvcc) |
|
|
|
|
### Memory
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| System RAM | 62 GiB |
|
|
| Swap | 127 GiB |
|
|
| /dev/shm | 32 GiB |
|
|
|
|
### Storage
|
|
|
|
| Filesystem | Size | Used | Avail | Use% |
|
|
|------------|------|------|-------|------|
|
|
| / (nvme0n1p2) | 915G | 729G | 140G | 84% |
|
|
| /mnt/entropy (dm-0) | 91T | 86T | 910G | 99% |
|
|
|
|
### Network
|
|
|
|
| Interface | Type | Speed | IP | Notes |
|
|
|-----------|------|-------|----|-------|
|
|
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.235 (DHCP) | RPC worker endpoint (port 50052). Default route via 192.168.0.1. |
|
|
| enxe8ea6ab7ebca | USB-C class adapter | 2500 Mb/s | 192.168.0.252 | UNPLUGGED 2026-08-17. Was idle (unused by the RPC path). |
|
|
| wlp8s0 | WiFi | - | - | Unused (carrier down). |
|
|
| docker0 | bridge | - | 172.17.0.1 | Down; internal docker traffic only. |
|
|
|
|
### OS and Software
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
|
| Kernel | 7.0.0-30-generic |
|
|
| Python | 3.14.4 |
|
|
|
|
### Active Services
|
|
|
|
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Qwen3.6-35B-A3B (Q6_K_P + ngram-mod) on port 7770. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
|
|
|
## Hardware Relevance
|
|
|
|
- 24 GB VRAM is close to saturated with this config. At idle ~22.1 GB used, leaving ~2.0 GiB headroom at `--n-cpu-moe 36` (4 layers' experts on GPU) with `ubatch 3072` and a dual-f16 KV pool of 13680 MiB at 700416 cells (2026-08-14 refit; the same pool at 800768 cells OOM'd the load warmup). A llama-bench sweep ([[frank-default-small-benchmark]]) guided this state: prefill is the priority, so `ubatch 3072` stays (top prefill tier), and `ncmoe 36` was raised from 35 to fix a cuBLAS OOM-class crash during big prefills by freeing ~0.75 GiB peak VRAM at only -2% prefill. If it still crashes under load, add `GGML_CUDA_FORCE_MMQ=1`. If decode speed matters more, use `ncmoe 32 - ubatch 1024` (gen ~48.7 t/s). The tuning guidance applied to the retired `frank-default-small` service; the PP worker's allocations are fixed at load.
|
|
- 62 GB system RAM holds the ~29 GB of MoE expert weights with room to spare (~57 GB available after load).
|
|
- 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
|
|
- Root disk has approximately 140 GB free. Store the GGUF models under `/home/dank/models` (about 32 GB total).
|
|
|
|
## See Also
|
|
|
|
- [[VLLM Overview]]
|
|
- [[Hybrid CPU GPU Processing Dankserver]]
|
|
- [[RAM CPU Offloading in VLLM]]
|
|
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] |