Frank
65d1be06ac
Deployed HALOGEN_MAX_TOK=32768 by operator call (upstream default, full prefill speed) and confirmed the prediction from the ledger: kv pool: 524288 positions need ~35.0 GiB ... LOWERING THE POOL TO 262144 (~27.8 GiB) memory: 68.0 weights + 7.2 KV pool + 21.1 working = 96.3 GiB in all so the effective pool is 262144 again and the doubling this task set out to do is not in force. Healthy, completion verified. Left the pool requesting 524288 deliberately: with KV_POOL_FIT on that is a grow-if-RAM-frees-up declaration, not a mistake. Two corrections to what I wrote earlier today: 1. The engine's sizing block, and its "WARNING even 262144 ... host RAM cannot spare it" line, print ONLY when the fit check had to clamp something. The 09-11 baseline boot prints no kv pool: sizing lines at all. So the absence of a warning in old logs was never evidence of headroom. 2. The budget needs a vision term. Actual need is pool + arena + 1.97 GiB of vision tower, not pool + a 1.5 GiB nominal margin. That makes the ceiling for a doubled pool max_tok <= 18,130 rather than 19,115, and it means the 32768/262144 corner this box has run in since 09-11 is ~0.3 GiB OVER its own stated budget -- and has been the whole time. It boots and serves with 12-14 GiB genuinely free because the 24 GiB reserve is generous. A soft floor, not a wall. The arena model also reproduces upstream's three published "on the device" figures exactly (35 / 42 / 41 GiB), confirming they are pool+arena totals at the default max_tok and not pool-only -- which is the misreading that made the doubling look affordable in my first estimate.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-small (llama.cpp, Qwen3.6-35B-A3B) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-small | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.