Frank
b312042706
The operator's revised spec: batch x3 (6144/1536), parallel 2, a 1.5 GiB per-card floor, and the context maximised inside it. Measured at 262144 with the old 38:27 split, 3x batch was brutally asymmetric: ROCm1 (output layers + the drafter's graphs) paid ~3.3 GiB where ROCm0 paid ~1.7, leaving the draft-carrying card at 0.08 GiB free against ROCm0's 3.78. One -ts step is worth ~1.5 GiB at this batch, so the balance is a measurement: 41:24 overshot (1.47 / 2.38), 40:25 landed at 1.92 / 1.96 GiB free. That balance is also what maximises the context -- an unbalanced split would have to buy its headroom out of CTX_SIZE, and CTX_SIZE is already at the model's trained window (262144), now split over two slots of 131072. Verified under two concurrent 1832-token prompts: 200/200, 1.87 / 1.86 GiB free, n_slots = 2, n_ctx_slot = 131072, tool calls intact, and DSpark drafting on BOTH slots (acceptance 0.38/0.22 and 0.52/0.43). 8192/2048 does not fit at this context at all: its 2020 MiB pp buffer fails to allocate on ROCm1, and 4096/1024 fits but leaves 1.11 GiB -- both recorded in the env's sizing table. The unit seam now models both cards (share + draft + ubatch-scaled buffers, calibrated to the rocm-smi figures to within 0.002%), asserts the 1.5 GiB floor on each, and asserts the two cards stay within 0.5 GiB of each other. Both assertions were confirmed to FAIL on a deliberately bad config (ubatch 8192, and a 1:1 split) rather than pass by construction.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.