Frank
3c2a5d2fe7
Model: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF (87.06 GiB, Q4_0_ROCMFP4_FAST imatrix-calibrated, per-head PLE table inside the GGUF — fully VRAM-resident). Draft: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST (2.28 GiB, matched ROCmFP4-FAST quant, ~66% acceptance at n-max 3). Vision tower (f16) loaded via mmproj/. Runtime: charlie12345/ROCmFPX build-qwen38 (HIP, ROCm 7.2.4, gfx1151) — the only binary with the ROCmFP4 FAST types. Replaces the jcbtc CIRU v1.1 deployment (ciru-ai runtime). Profile: card sampling table (temp 0.7, top_p 0.8, top_k 20, min_p 0.0, presence 1.5) mixed with the validated build-qwen38 serving profile: 262144 ctx x 1 slot, b 4096 / ub 4096, t 16 / tb 16, f16 KV, fa on, cont-batching, prompt cache + 8 GiB cache-ram + idle slots, 16 ctx checkpoints (no --checkpoint-min-step — unsupported on this binary), --fit off (auto-fit segfaults on qwen4exp hyper-connection config), --spec-type draft-mtp,ngram-mod with n-max 3 (adaptive p-split measured worse per author card), p-min 0.75. Qwen-Sharp chat template kept; reasoning stays in DeepSeek format. The 2026-08-28 failure of this draft head on the kingjones STRIX_LEAN model was a cross-vendor pairing problem: the head carries blk.48.hc_attn_norm.weight (verified in-GGUF, same shape as the working CIRU draft). Docs: new runbook agentionai-fast-imatrix, index updates.
Server App Configs
The single repository for the two homelab servers: the Podman container stacks and the LLM inference fleet. One repo, one checkout per host, one deploy script.
2026-08-22: the former
vllmScriptsrepository (LLM fleet) is merged into this repo with its full git history. See AGENTS.md for the merge note.
Targets
| Host | User | IP | Podman stack | LLM service | LLM port |
|---|---|---|---|---|---|
| dankserver | dank | 192.168.0.235 | media and library | frank-default-small (llama.cpp, Qwen3.6-35B-A3B) | 7770 |
| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-default-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
Active Services
| Service | Host | Hostname | Port |
|---|---|---|---|
| RomM | dankserver | romm.dankserver.local | 8085 |
| Copyparty | dankserver | files.dankserver.local | 3923 |
| Calibre-Web | dankserver | books.dankserver.local | 8083 |
| frank-default-small | dankserver | — (bare endpoint) | 7770 |
| SearXNG | dankclaw | search.dankclaw.local | 8081 |
| ComfyUI | dankclaw | comfyui.dankclaw.local | 8188 |
| OpenWebUI | dankclaw | chat.dankclaw.local | 3000 |
| Hermes | dankclaw | hermes.dankclaw.local | 8090 |
| Gitea | dankclaw | gitea.dankclaw.local | 3001 |
| Grafana | dankclaw | grafana.dankclaw.local | 3002 |
| Prometheus | dankclaw | prometheus.dankclaw.local | 9090 |
| frank-default-large | dankclaw | — (bare endpoint) | 7770 |
Structure
dankserver/— compose.yaml + Caddyfile + .env.example for the Dankserver stackdankclaw/— compose.yaml + Caddyfile + .env.example for the Dankclaw stackshared/— homepage, prometheus config, deploy helpersscripts/—deploy.shanddeploy-remote.sh(service-scoped deployment), the planner, LLM serve scripts, systemd units, env tunables, and bench harnessesold/— retired PP/NPU-era LLM scripts (reference only)serverAppConfigs-docs/— project knowledge base (Obsidian vault, wikilinks).serverAppConfigs-docs/LLM/is the LLM fleettests/— bats test suite (see Testing Policy)
Deployment
See serverAppConfigs-docs/Architecture/Deployment Flow.md
and serverAppConfigs-docs/LLM/deployment.md. Run
./scripts/deploy.sh <dankclaw|dankserver> after committing changes. It
plans and deploys only the changed services. Use --service NAME for an
explicit single-service retry; inference is restarted only when its own files
(or the shared chat template) changed.
Knowledge Base
The serverAppConfigs-docs/ directory is an Obsidian vault. Start at
serverAppConfigs-docs/index.md. The LLM fleet knowledge base lives at
serverAppConfigs-docs/LLM/index.md.
Testing
Run ./tests/run.sh before every push. See
serverAppConfigs-docs/Architecture/Testing Policy.md.