Frank
68afc6dbaf
Frank: "shift frank-zealot to 2 parallels of unquantized kv". Shape: CTX_SIZE 786432 -> 524288, PARALLEL 3 -> 2, CACHE_TYPE_K/V q8_0 -> f16. The per-slot window is UNCHANGED at 262,144 (the native trained window, and models.yml contextWindow), so what the exact cache buys is fidelity and what it pays is one concurrent conversation. The POOL is the term that had to move: exact KV bills 32,768 B/token at this geometry (8 full-attention layers x 4 KV heads x 256 dim, 2 B, K and V), so the old 786,432 pool asks 24,576 MiB against the 24,123 MiB the driver leaves addressable. 524,288 positions = 16,384 MiB, projected ~21,200 used / ~2,900 free from the 13:19 boot ledger (only the KV pool and one recurrent-state cell move). Also corrects a repo-wide arithmetic error: "f16 would bill 26,112 MiB" doubled the q8_0 ledger figure INCLUDING its 1/32 scale overhead. q8_0 is 1.0625 B/element, f16 exactly 2.0, so the true ratio is 1.882x. The wall stands; the number was wrong. And it corrects a false claim about the drafter: this build DOES expose --spec-draft-type-k / --spec-draft-type-v (verified against llama-server --help), so the draft cache is quantizable -- the exact-KV-plus-MTP exclusion is a measurement gap, not a law, and the 1,088 MiB quantized-draft arm is recorded as unmeasured. Seam: tests/frank_zealot_unit.bats re-pins the shape contract (2 slots, 524,288 pool, 262,144 per slot) and adds a VRAM fit gate that computes dtype x pool + the measured non-KV ledger terms against 24,123 MiB with >=1 GiB of slack -- and proves the gate REFUSES exact KV at the old pool, so passing means something. 18/18 green.
Server App Configs
AI Disclosures:
- Slop rating: 2/5 - Actively using, actively de-slopping, repository mostly human owned, lacks formal process.
- Completeness: Unfinished, planning planned, no plan yet
- Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
- Local model: Qwen 3.8 Flash Next
slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.