docs: frank-medium runbook, fleet topology and client cutover
New runbook page serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md records the hardware swap, the self-contained gfx1201 build, the measured layer-vs- tensor decision (llama-bench multi-device syntax gotcha included), the tensor-mode draft abort, served-path numbers, and the post-cleanup host ROCm posture. AGENTS.md, README, docs indexes, Dankserver System Specs, deployment/Hermes/OpenWebUI/Plex/Stack pages follow the rename; freetoken-frank-small is marked RETIRED and kingjones-q8agent SUPERSEDED. Work log entry added.
This commit is contained in:
Frank committed
1 parent
d53962fec5
commit
d4024a9180
15 files changed
+426
-80
No files matched your search
@@ -1,8 +1,24 @@
|
||||
---
|
||||
type: configuration
|
||||
title: Dankserver System Specs
|
||||
description: Dankserver hardware and OS state since the 2026-09-15 GPU swap (2x AMD Radeon AI PRO R9700, gfx1201) serving frank-medium.
|
||||
tags:
|
||||
- dankserver
|
||||
- hardware
|
||||
- r9700
|
||||
- gfx1201
|
||||
- rocm
|
||||
timestamp: 2026-09-15
|
||||
---
|
||||
|
||||
# Dankserver System Specs
|
||||
|
||||
## Server Overview
|
||||
|
||||
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.04 (Resolute Raccoon) with an RTX 3090 and 62 GB of system RAM.
|
||||
Dankserver is a media/library server and the `frank-medium` LLM host. The
|
||||
host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that served the
|
||||
FreeToken/CUDA `frank-small` era was removed on 2026-09-15 and replaced with
|
||||
two AMD Radeon AI PRO R9700 cards.
|
||||
|
||||
## Hardware Specifications
|
||||
|
||||
@@ -13,42 +29,49 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
|
||||
| Architecture | x86_64 |
|
||||
| Vendor | AuthenticAMD |
|
||||
| Model | AMD Ryzen 7 7700X |
|
||||
| Cores | 8 (1 socket) |
|
||||
| NUMA Nodes | 1 (node 0, 63503 MB) |
|
||||
| Cores | 8 cores / 8 threads (1 socket, SMT off) |
|
||||
| NUMA Nodes | 1 (node 0) |
|
||||
|
||||
### GPU
|
||||
### GPUs
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Model | NVIDIA GeForce RTX 3090 |
|
||||
| VRAM | 24576 MiB (24 GB) |
|
||||
| Driver | 595.84 |
|
||||
| CUDA Version | 13.2 |
|
||||
| CUDA Toolkit | 12.0 (nvcc) |
|
||||
| Model | 2x AMD Radeon AI PRO R9700 (Navi 48, DID 0x7551, rev c0) |
|
||||
| Architecture | gfx1201 (RDNA4) |
|
||||
| VRAM | 32,624 MiB each |
|
||||
| Link | PCIe, no XGMI; different IOMMU groups |
|
||||
| PCI | 03:00.0 and 0b:00.0 |
|
||||
| iGPU | Raphael (gfx1036) at 10:00.0 — enumerates as ROCm device 2, hidden from HIP |
|
||||
|
||||
The RTX 3090 is gone; stale `nvidia-smi` binaries and the disabled NVIDIA
|
||||
apt lists remain on disk but no NVIDIA PCI device exists.
|
||||
|
||||
### Memory
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| System RAM | 62 GiB |
|
||||
| Swap | 127 GiB |
|
||||
| System RAM | 60 GiB |
|
||||
| Swap | 63 GiB |
|
||||
| /dev/shm | 32 GiB |
|
||||
|
||||
### Storage
|
||||
|
||||
| Filesystem | Size | Used | Avail | Use% |
|
||||
|------------|------|------|-------|------|
|
||||
| / (nvme0n1p2) | 915G | 729G | 140G | 84% |
|
||||
| / (nvme0n1p2) | 915G | ~640G | ~280G | ~70% |
|
||||
| /mnt/entropy (dm-0) | 91T | 86T | 910G | 99% |
|
||||
|
||||
Models live under `/home/dank/models`; the ROCmFPX build tree under
|
||||
`/home/dank/GitPackages/charlie-ROCmFPX` with its self-contained ROCm 7.14
|
||||
toolchain under `~/.local/share/rocm-7.14.0a20260624`.
|
||||
|
||||
### Network
|
||||
|
||||
| Interface | Type | Speed | IP | Notes |
|
||||
|-----------|------|-------|----|-------|
|
||||
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.235 (DHCP) | RPC worker endpoint (port 50052). Default route via 192.168.0.1. |
|
||||
| enxe8ea6ab7ebca | USB-C class adapter | 2500 Mb/s | 192.168.0.252 | UNPLUGGED 2026-08-17. Was idle (unused by the RPC path). |
|
||||
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.235 (DHCP) | LLM endpoint (port 7770). Default route via 192.168.0.1. |
|
||||
| enxe8ea6ab7ebca | USB-C class adapter | 2500 Mb/s | 192.168.0.252 | Unplugged since 2026-08-17. |
|
||||
| wlp8s0 | WiFi | - | - | Unused (carrier down). |
|
||||
| docker0 | bridge | - | 172.17.0.1 | Down; internal docker traffic only. |
|
||||
|
||||
### OS and Software
|
||||
|
||||
@@ -57,21 +80,35 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
|
||||
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
||||
| Kernel | 7.0.0-30-generic |
|
||||
| Python | 3.14.4 |
|
||||
| ROCm userspace | Ubuntu 7.1 packages in /usr (hipcc 7.1.1, rocminfo, rocm-smi 7.1.1) |
|
||||
| Serving stack | charlie12345/ROCmFPX llama.cpp with its own ROCm 7.14 toolchain (self-contained, RPATH-bundled) |
|
||||
|
||||
The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
|
||||
hijack `libhsa-runtime64` were purged on 2026-09-15 — see
|
||||
[[frank-medium-rocmfpx]].
|
||||
|
||||
### Active Services
|
||||
|
||||
The active service on Dankserver is `frank-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q6_K) on port 7770, with expert tensors selected by `--override-tensor`; every full-attention block plus blocks 35–40 remain on CUDA. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
||||
The active inference service is `frank-medium.service`: charlie12345/ROCmFPX
|
||||
llama-server on port 7770, layer-split across both R9700s with the MTP draft
|
||||
head on ROCm1. See [[frank-medium-rocmfpx]]. The former
|
||||
`frank-small.service` (FreeToken/CUDA) was retired with the GPU swap; its
|
||||
unit is archived in `~/.AGENTTRASH/20260915-frank-small-retire/host-units/`.
|
||||
The earlier PP worker (`frank-pp-worker.service`) was retired with the
|
||||
2026-08-17 redo. See [[deployment]].
|
||||
|
||||
## Hardware Relevance
|
||||
|
||||
- 24 GB VRAM hosts the Q6_K non-expert tensors and the full-attention/final-layer expert placement. The context pool contains 491520 f16 cells across three 163840-token slots. The service uses ngram-mod only and `ubatch=1024`. Verify actual headroom after each model restart with `nvidia-smi` and the service journal.
|
||||
- 62 GB system RAM holds the remaining CPU-resident expert tensors with room to spare.
|
||||
- 127 GB swap is available for CPU KV cache paging if GPU KV offload is disabled.
|
||||
- Root disk has approximately 140 GB free. Store the GGUF models under `/home/dank/models` (about 32 GB total).
|
||||
- 2x 32 GB VRAM: the 26.28 GiB Q8_0_ROCMFPX_AGENT checkpoint plus MTP draft
|
||||
and a 131,072-token f16 KV cache fit comfortably; layer split gives each
|
||||
card about half the layers and its own KV share.
|
||||
- 60 GB system RAM is mostly headroom; nothing in the serving path is
|
||||
CPU-resident.
|
||||
- The iGPU is deliberately unused: `HIP_VISIBLE_DEVICES=0,1` keeps work off
|
||||
the shared-memory Raphael device.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[VLLM Overview]]
|
||||
- [[Hybrid CPU GPU Processing Dankserver]]
|
||||
- [[RAM CPU Offloading in VLLM]]
|
||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
|
||||
- [[frank-medium-rocmfpx]] — the active serving runbook
|
||||
- [[deployment]] — the deploy flow
|
||||
- [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory
|
||||
Reference in new issue
Block a user