dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: 2026-08-21 incident (distro upgrade dropped CUDA 12 runtime) + Dankserver now Ubuntu 26.04

Browse Source
This commit is contained in:
Frank
2026-08-21 19:31:05 -07:00
parent 583efd8cf2
commit 577003bed1
3 changed files with 22 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Dankserver System Specs.md
+4 -4
View File
@@ -2,7 +2,7 @@
## Server Overview
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 24.04 with an RTX 3090 and 62 GB of system RAM.
Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.04 (Resolute Raccoon) with an RTX 3090 and 62 GB of system RAM.
## Hardware Specifications
@@ -54,9 +54,9 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 24.0
| Property | Value |
|----------|-------|
| OS | Ubuntu 24.04.4 LTS (Noble Numbat) |
| Kernel | 7.0.0-28-generic |
| Python | 3.12.3 |
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
| Kernel | 7.0.0-30-generic |
| Python | 3.14.4 |
### Active Services
docs/deployment.md
+16
View File
@@ -160,6 +160,22 @@ If a service segfaults at load with no obvious cause, compare the exported env a
Historical: on the retired topology, parallel jobs on the Dankserver 35B service died with a cuBLAS crash under researcher load. The affected service is retired. A client abort logs `W srv stop: cancel task, id_task = N`; a natural stop logs only `slot release` with `truncated = 0`. The server never preempts a running generation.
## Incident: frank-default-small Crash Loop After Distro Upgrade (2026-08-21)
On 2026-08-21 a release upgrade moved Dankserver from Ubuntu 24.04 to Ubuntu 26.04 (Resolute Raccoon). The upgrade removed the CUDA 12 runtime packages: `libcudart12`, `libcublas12`, `libcublaslt12`, and `nvidia-cuda-toolkit`. The `llama-server` binary links `libcudart.so.12` and `libcublas.so.12`. On every start the binary exited with code 127. systemd restart-looped the service continuously. The health endpoint on port 7770 stayed down.
Fix: reinstall the runtime packages. The same versions remain in the 26.04 repository.
sudo apt-get install -y libcudart12 libcublas12 libcublaslt12
Verify that the binary resolves all libraries:
ldd /home/dank/llama.cpp/build/bin/llama-server | grep "not found"
Start the service and check the health endpoint on port 7770.
After any release upgrade on Dankserver, run the `ldd` check above. A missing CUDA 12 library means the packages need a reinstall.
## See Also
- [AGENTS.md](../AGENTS.md) — the git workflow rules
docs/index.md
+2 -2
View File
@@ -8,7 +8,7 @@ CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large).
```mermaid
graph LR
subgraph Dankserver[NVIDIA Dankserver - Ubuntu 24.04]
subgraph Dankserver[NVIDIA Dankserver - Ubuntu 26.04]
A[RTX 3090 24GB + 62GB RAM]
B[upstream llama.cpp b10121
frank-default-small, port 7770]
@@ -34,7 +34,7 @@ frank-default-large, port 52625]
| Server | GPU | VRAM / Memory | OS |
| --------------------------------------- | ----------------------------- | -------------------- | ------------ |
| [[Dankserver System Specs\|Dankserver]] | NVIDIA RTX 3090 | 24 GB VRAM, 62 GB RAM | Ubuntu 24.04 |
| [[Dankserver System Specs\|Dankserver]] | NVIDIA RTX 3090 | 24 GB VRAM, 62 GB RAM | Ubuntu 26.04 |
| [[Dankclaw System Specs\|Dankclaw]] | AMD Radeon 8060S (Strix Halo) | gfx1151 unified memory | Ubuntu 26.04 |
## Active Deployment