Frank
c2116658f8
The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:
arm I llama.cpp layer split 32:33 + DFlash2, Q8_0 geo decode 45.7 t/s, TTFT 0.72 s, burst-4 71.7 t/s
arm L llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4 54.3 t/s
arm R radiance vLLM TP=2 + DFlash2-FP8, MXFP4 geo decode 95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s
Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.
- un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
The replica pair and its affinity proxy stay archived; one process owns
both cards now.
- retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
stay: they are the tooling that rollback needs.
- deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
the drafter, the pinned patch clone and the patched libr4d; retires the
llama root unit into the home trash instead of relinking it; then
install_quadlet + wait_container_healthy 1800s (first start compiles).
retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
a/b/proxy trio only, since frank-medium is installed again.
- planner + test-changed map dankserver/frank-medium/** to frank-medium
and its seam; the scripts-trio mapping is gone with the files.
- tests/frank_medium_unit.bats rewritten for the container contract: boot
persistence of a GENERATED unit (no enable on this host's v1 generator),
TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
contract, and the archive invariants.
- homepage card + docs updated, and they were stale twice over: they still
described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.
Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-medium | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.