Frank
803398051a
Asked whether Vulkan on both devices unlocks the split. It does not: the per-tensor design aborts byte-identically, because the gate is not the driver mix but `--override-tensor` — `has_tensor_overrides()` is a required term of the scheduler's `pipeline_parallel` flag (llama-context.cpp:433, the only place in the tree that reads it), and that flag sets `n_copies` in `ggml_backend_sched_new`. With `n_copies == 1` a pre-allocated weight on a non-primary device has no backend and the graph aborts. Chasing it exposed a harness artifact that invalidated the earlier conclusion: llama-bench splits a device LIST with `/` and SWEEP values with `,`, so `-dev CUDA0,Vulkan1` was two single-device runs all along (and `-ts 0.85,0.15` two sweep values, i.e. device 0 takes everything — which is where the bogus OOM came from). With the correct syntax two-device splits work in both flavours: pure Vulkan -dev Vulkan1/Vulkan0 -ts 0.85/0.15 -> 27.55 tok/s @64k (12.8 GiB on the 3090) cross-vendor -dev CUDA0/Vulkan1 -ts 0.15/0.85 -> 27.23 tok/s @64k, pp512 268.5 (15.0 GiB) iGPU alone 25.77 tok/s @64k, pp512 231.8 +7 % decode, +16 % prefill — real but not a regime change, and it costs the 3090 ~15 GiB, which this fleet spends on its only vision endpoint. The design's premise still fails: whole layers carry their experts (1.62 GiB each), the 12 attention layers are spread every 4th, and the 32.8 GiB PLE must stay on the main device, so KV cannot be consolidated on the 3090. Also records the evening's incident (a dead rootless podman pause process took all container management down; `podman system migrate` panics in 5.7.0; reboot repaired it) and the netavark host-address lesson from the hermes 502.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.