Frank
30e8ed55a0
The boot of record for the new shape, from the verbosity-4 journal plus
settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00),
draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state
452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29
+ 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255
free with n_slots = 3 / n_ctx_slot = 200704.
Two inherited numbers did not survive it, and both are corrected in the env,
the serve rationale and the page:
- The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the
786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went
2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host
went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is
what funds the drafter, alone.
- The fit gate priced 22,428 against a real 22,870: conservative by 442
MiB, so it stays honest as written.
And one finding stated plainly: this is the first boot on this card that
misses llama.cpp's own free-memory target - the engine printed "cannot meet
free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
and settled at 1,255 MiB free, under the ~2 GiB band every previous era
held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one
line.
Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of
thinking - empty content at max_tokens 64 is the documented trap), tools
(finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red ->
"Red", 280 prompt tokens), and the drafter's own statistics (acceptance
0.706 / 0.763, mean accepted length 2.41 / 2.53).
Server App Configs
AI Disclosures:
- Slop rating: 2/5 - Actively using, actively de-slopping, repository mostly human owned, lacks formal process.
- Completeness: Unfinished, planning planned, no plan yet
- Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
- Local model: Qwen 3.8 Flash Next
slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-dragoon (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-dragoon | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.