dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: 2026-08-21 incident (distro upgrade dropped CUDA 12 runtime) + Dankserver now Ubuntu 26.04

Browse Source
This commit is contained in:
Frank
2026-08-21 19:31:05 -07:00
parent 583efd8cf2
commit 577003bed1
3 changed files with 22 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment.md
+16
View File
@@ -160,6 +160,22 @@ If a service segfaults at load with no obvious cause, compare the exported env a
Historical: on the retired topology, parallel jobs on the Dankserver 35B service died with a cuBLAS crash under researcher load. The affected service is retired. A client abort logs `W srv stop: cancel task, id_task = N`; a natural stop logs only `slot release` with `truncated = 0`. The server never preempts a running generation.
## Incident: frank-default-small Crash Loop After Distro Upgrade (2026-08-21)
On 2026-08-21 a release upgrade moved Dankserver from Ubuntu 24.04 to Ubuntu 26.04 (Resolute Raccoon). The upgrade removed the CUDA 12 runtime packages: `libcudart12`, `libcublas12`, `libcublaslt12`, and `nvidia-cuda-toolkit`. The `llama-server` binary links `libcudart.so.12` and `libcublas.so.12`. On every start the binary exited with code 127. systemd restart-looped the service continuously. The health endpoint on port 7770 stayed down.
Fix: reinstall the runtime packages. The same versions remain in the 26.04 repository.
sudo apt-get install -y libcudart12 libcublas12 libcublaslt12
Verify that the binary resolves all libraries:
ldd /home/dank/llama.cpp/build/bin/llama-server | grep "not found"
Start the service and check the health endpoint on port 7770.
After any release upgrade on Dankserver, run the `ldd` check above. A missing CUDA 12 library means the packages need a reinstall.
## See Also
- [AGENTS.md](../AGENTS.md) — the git workflow rules