Frank
a11fae07d8
The batch/ubatch sweep (scripts/bench-batch-ubatch.sh, 23 cells of a 28-cell grid, 3 reps each) says the ubatch knob has an OPTIMUM, not a plateau: ubatch 256 384 512 768 1024 1280 1536 1792 2048 tok/s 1046 1068 1193 1135 1152 1151 1165 1148 1078 The curve is remarkably reproducible -- 512 came out 1192.88 / 1193.23 / 1193.28 / 1193.88 across batch 2048, 4096, 6144 and 8192, and 2048 came out 1077.38 / 1077.54 / 1077.59 -- so this is a deterministic property of the shape, not noise. The logical batch is throughput-neutral (2048..8192 all within 2 % at a fixed ubatch) and VRAM-neutral (the compute buffers are sized by ubatch alone). Decode is 15.88 tok/s at depth 65536 on every single cell. So the deployed 6144x1536 was both 2.4 % slower than the optimum and 3.71 GiB more expensive. Measured at the deployed shape (262144, 2 slots, DFlash2): 36,29 ubatch 1536 -> 4.71 / 4.91 GiB free (what was deployed) 36,29 ubatch 512 -> 6.01 / 7.32 GiB free (ubatch alone) 35,30 ubatch 512 -> 6.44 / 6.90 GiB free <- deployed The ubatch move freed 1.30 GiB on ROCm0 but 2.41 GiB on ROCm1, because ROCm1 holds the output layers and the drafter's graphs -- so it shifted the balance toward ROCm1 by 1.31 GiB and the split had to follow it one layer. 0.46 GiB is one DeltaNet layer's worth of asymmetry, i.e. the granularity floor: no split can be tuned finer than a layer, which is why 34,31 is not better. Re-fitted the seam with both ubatch coefficients measured at this shape and re-anchored it on the new deployed point (it reproduces 6.44 / 6.90 to 0.01 GiB). Every neighbouring split still fails it: 36,29 -> 1.85, 34,31 -> 0.94, 37,28 -> 3.24, 1,1 -> 3.03, 39,26 -> 6.03 GiB. Also replaced the unit test's literal `TENSOR_SPLIT=36,29` assertion with the shape it actually cares about -- two shares that add up to the model's 65 blocks, with ROCm1 taking the fewer -- so re-measuring the balance no longer breaks a test about parallelism. The numeric contract lives in the seam test.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.