dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: rename frank-default-large/small/sightsound to frank-large/small/npu across vault

Browse Source 
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
This commit is contained in:
Frank committed 2026-09-03 16:17:14 -07:00
1 parent 5a083eee6c
commit 8212d4451a
68 files changed
+1053 -1207
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+20 -20
View File
@@ -8,10 +8,10 @@ one checkout per host, one deploy script.
- **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
Calibre-Web, the *arr stack, Jellyfin) as bare containers, plus the
`frank-default-small` LLM service.
`frank-small` LLM service.
- **Dankclaw** (192.168.0.143) — AI/search/development/observability stack
(SearXNG via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus,
Loki, Alloy, Pi-hole) plus the `frank-default-large` LLM service. Caddy on
Loki, Alloy, Pi-hole) plus the `frank-large` LLM service. Caddy on
Dankclaw reverse-proxies both hosts.
**2026-08-22 merge:** the former `vllmScripts` repository (LLM fleet scripts,
@@ -26,9 +26,9 @@ host.
| Target | Runtime | Service | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | FreeToken v0.1.2 | frank-default-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
| Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-default-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 |
| Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-default-sightsound | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 7771 |
| Dankserver | FreeToken v0.1.2 | frank-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
| Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 |
| Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-npu | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 7771 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -47,7 +47,7 @@ Qwen3.6-35B-A3B-FP8. On 2026-08-30 both services swapped again:
Dankclaw to the kingjones STRIX model on the card's pinned
kingjones30/ROCmFPX runtime, Dankserver to the Huihui abliterated
FP8 checkpoint on FreeToken. On 2026-08-30 the NPU endpoint returned as
`frank-default-sightsound` — FLM v1.0.1 on XDNA2 serving sight+sound (see the
`frank-npu` — FLM v1.0.1 on XDNA2 serving sight+sound (see the
Runtime & Troubleshooting section and its runbook). Runbooks:
[serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
@@ -117,13 +117,13 @@ selected service and records its successful revision outside the checkout.
./scripts/deploy.sh dankclaw
./scripts/deploy.sh dankserver
./scripts/deploy.sh dankclaw --service searxng
./scripts/deploy.sh dankserver --service frank-default-small
./scripts/deploy.sh dankserver --service frank-small
./scripts/deploy.sh dankclaw --dry-run
```
A documentation-only change is a no-op. The LLM services are selected only by
their own unit/env/serve files or the shared chat template, so unrelated
container changes cannot interrupt inference. `frank-default-npu` is relink-only
container changes cannot interrupt inference. `frank-npu` is relink-only
when explicitly selected.
Set `FRANK_SUDO_PW` for headless runs on a host without passwordless sudo
@@ -263,10 +263,10 @@ ALWAYS bench using llama-bench for llama.cpp or vllm bench for vllm runtimes.
DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE OFFICIAL BENCHES. ASK THE USER BEFORE DEVIATING FROM THIS DIRECTIVE.
1. Stop the service before any benchmark:
`sudo systemctl stop frank-default-large` (Dankclaw) or
`sudo systemctl stop frank-default-small` (Dankserver).
`sudo systemctl stop frank-large` (Dankclaw) or
`sudo systemctl stop frank-small` (Dankserver).
Then mask it to prevent the self-healing unit from restarting:
`sudo systemctl mask frank-default-large` (Dankclaw).
`sudo systemctl mask frank-large` (Dankclaw).
**CRITICAL WARNING:** Do NOT use `systemctl disable` on these service units.
On these hosts, `systemctl disable` doesn't just remove symlinks — it **deletes
the unit file itself** from `/etc/systemd/system/`. You'd have to manually
@@ -279,7 +279,7 @@ DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE
(the model + KV fill the card), so changes are validated directly against
the service itself.
3. Restart the service when the bench completes: unmask, then start:
`sudo systemctl unmask frank-default-large && sudo systemctl start frank-default-large` (Dankclaw).
`sudo systemctl unmask frank-large && sudo systemctl start frank-large` (Dankclaw).
4. Verify health before reporting:
`curl http://192.168.0.235:7770/health` or
`curl http://192.168.0.143:7770/health`.
@@ -289,7 +289,7 @@ DO NOT ROLL YOUR OWN BENCH UNLESS YOU CANNOT TEST THE DESIRED PARAMETER WITH THE
The Pi agent reaches the fleet through `/home/dank/.pi/agent/models.json`.
This file lives outside the repository. Each provider is one host endpoint.
Each model `id` is the served alias. The two aliases are
`frank-default-small` (Dankserver `:7770`) and `frank-default-large`
`frank-small` (Dankserver `:7770`) and `frank-large`
(Dankclaw `:7770`).
The model `id` values are stable. The file
@@ -310,7 +310,7 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting
### frank-default-sightsound — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE)
### frank-npu — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE)
The third inference endpoint on Dankclaw. The `flm serve` runtime (dpkg
package `fastflowlm`, v1.0.1, binary `/usr/bin/flm`) serves the **whole local
@@ -333,19 +333,19 @@ Hard requirements verified at first deploy (2026-08-30):
bind needs PASID/SVA, which requires enabled IOMMU hardware. Fix applied:
`iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps the
iGPU default domain identity-mapped — zero DMA-translation overhead for
frank-default-large — while enabling IOMMU + PASID; PASID domains are
frank-large — while enabling IOMMU + PASID; PASID domains are
paged regardless of the default-domain type). Never set `iommu=off` on
this host again; it silently kills the NPU.
- Health probe = `GET /v1/version` (FLM has no `/health` route).
- Unit: `User=dank`, `LimitMEMLOCK=infinity` (FLM requires unlimited
memlock), `Restart=on-failure`, `RestartSec=10`. Deploy via
`scripts/deploy.sh dankclaw --service frank-default-sightsound`.
`scripts/deploy.sh dankclaw --service frank-npu`.
- Verified live at first deploy: image probe (red PNG as `image_url`
data-URI → correct color answer, vision tower consumed the pixels) and an
audio control-pair probe (tone WAV via `input_audio` consumed ~22
audio-tower tokens vs a text-only control).
Runbook: [serverAppConfigs-docs/LLM/frank-default-sightsound.md](serverAppConfigs-docs/LLM/frank-default-sightsound.md).
Runbook: [serverAppConfigs-docs/LLM/frank-npu.md](serverAppConfigs-docs/LLM/frank-npu.md).
### kingjones Qwen3.8-Flash-Next STRIX on kingjones30/ROCmFPX (Dankclaw, ACTIVE)
The large service serves
@@ -375,7 +375,7 @@ correct here — mmap pages ARE page cache) and **never pin the PLE
tensor with `-ot`** (23 → 13.4 t/s). The 51.2B-param PLE table never
enters GPU memory (~20 bytes/token-file per 500 decoded). On-host file
sha256 verified == HF LFS oid `c6770d74...49fa`. Env
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`.
`scripts/frank-large.env`, serve `scripts/serve-frank-large.sh`.
Runbook:
[serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md](serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md).
@@ -416,9 +416,9 @@ validated `--memory-num-gpu-blocks-experts 2048
(never grow the expert pool). 262144 context, reasoning on (the
checkpoint ships the froggeric fixed chat template — no override).
FreeToken's API REQUIRES an explicit `"model"` field in chat requests
(422 without it). Env `scripts/frank-default-small.env`, serve
(422 without it). Env `scripts/frank-small.env`, serve
`scripts/serve-frank-small.sh`. Runbook:
[serverAppConfigs-docs/LLM/freetoken-frank-default-small.md](serverAppConfigs-docs/LLM/freetoken-frank-default-small.md).
[serverAppConfigs-docs/LLM/freetoken-frank-small.md](serverAppConfigs-docs/LLM/freetoken-frank-small.md).
**Serving status (2026-08-30):** swapped the model within the FreeToken
runtime (official FP8 → Huihui abliterated FP8). ~40 tok/s verified on