Frank
e5893dc30d
BATCH_SIZE 2048 -> 4096, UBATCH_SIZE 1024 -> 2048. Why the ubatch move is right on THIS card: the prefill curve is card-specific. On the 2x R9700 layer-split sibling it peaks at ubatch 512 and every step up costs GiB for no speed; on this CUDA 3090 it climbs monotonically to 2048 (710 -> 1188 t/s on the 35B MoE sweep, 1944 -> 2423 t/s on the Ornith 8k sweep) with decode flat to under 1 t/s on every rung. The logical batch is free either way: two sweeps agree that at a fixed ubatch, 2048/4096/8192 give identical prefill speed and identical peak VRAM, because the compute buffers are sized by the ubatch alone. The risk is all in the ubatch, and it is documented rather than assumed: the step cost ~1,244 MiB of peak graph VRAM on the 35B MoE, the shape this replaces is 2,815 MiB free, and the Q5_K_M predecessor recorded "ubatch 2048 failed the scratch boot" with MORE free than that. Compute buffers are allocated at graph capture during boot, so a failure is loud and fast; the step-down rungs (4096/1536, then 4096/1024) are named in the env. The seam stops hard-coding a rung: serve_argv now carries BATCH_SIZE and UBATCH_SIZE the way it already carried CTX_SIZE et al., and the pair test asserts the argv follows the shipped env plus the llama.cpp invariant ubatch <= batch. Verified both ways: retuning the env to 8192/4096 keeps the seam green, inverting the pair to 2048/4096 turns it red.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.