Frank
68a525740d
Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B) + the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/. Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the 24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch 1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk, not loaded: its draft KV is built against the target's full window, measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise). Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off = 21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319. Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5 full-attention layers) so KV scales with 5 layers only, and attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the measured q8_0 pool = 2.05 GiB = the K-only figure). Serve script gained VERBOSITY (+ --log-verbosity in the argv), the SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4. deploy-remote.sh payload gates follow the env. The frank-secondary bats seam is rewritten for the new contract (argv-based) and fixes a pre-existing vacuity: serve_argv regenerates the test env, so mutations made before it were silently undone - tests that mutate now exec through run_stage. frank-secondary is the fleet compaction hub (200,704 is the only window that fits a worst-case compaction input).
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.