Compare commits

..
Author SHA1 Message Date
Frank 2693259227 configure anonymous SearXNG provider defaults 2026-08-25 17:58:22 -07:00
Frank e6670f914f docs: record live SearXNG provider baseline 2026-08-25 17:58:22 -07:00
Frank a05584b155 tune SearXNG VPN request timeouts and retries 2026-08-25 17:58:22 -07:00
Frank e1f05bd3c2 Configure Hermes access to small LLM 2026-08-25 17:55:52 -07:00
Frank 96f5ae6fe1 Raise ubatch benchmark context to 32K 2026-08-25 17:44:45 -07:00
Frank 4cb67d7b33 Set ubatch sweep context and batch to 16K 2026-08-25 17:32:12 -07:00
Frank babc823f11 Avoid shell variable collision in bench parser 2026-08-25 17:26:32 -07:00
Frank 56ee7cca45 Pass run number into ubatch benchmark parser 2026-08-25 17:12:51 -07:00
Frank c072c75fa9 Make ubatch benchmark cleanup failure safe 2026-08-25 17:11:42 -07:00
Frank 119ca2493f feat: add tmux session snapshot and restore utility 2026-08-25 17:04:15 -07:00
Frank a8e0b88d99 Parse current llama timing output in ubatch bench 2026-08-25 16:57:02 -07:00
Frank 835996a67b Keep deployment moving when Caddy restart is blocked 2026-08-25 16:55:10 -07:00
Frank eae5bc2565 Continue LLM deployment when compose restart fails 2026-08-25 16:54:18 -07:00
Frank c7a72b465e Fix privileged shared config deployment 2026-08-25 16:53:13 -07:00
Frank d6ea71b9a4 Configure DeepSeek reasoning and large-model ubatch sweep 2026-08-25 16:52:45 -07:00
Frank 8c8c3f7cd0 Enable focused tests as repository pre-commit hook 2026-08-25 16:18:01 -07:00
Frank 63ff1b6b43 Add focused changed-path test wrapper 2026-08-25 16:05:59 -07:00
Frank 7a0feb39aa Add CLI helper for entropy volume 2026-08-25 15:38:57 -07:00
Frank 6ee195dd8f Restore historical Q6 large model runtime 2026-08-25 14:23:47 -07:00
Frank ddfc64013f Fix Q6 AGENT projector path 2026-08-25 13:58:50 -07:00
Frank 86d5f38142 Restore Q6 AGENT large model 2026-08-25 13:36:07 -07:00
Frank 205b11aae4 Clarify historical ROCmFPX payload sizes 2026-08-25 13:28:38 -07:00
Frank 6c9e71efd2 Document Q8 AGENT set_rows serving blocker 2026-08-25 12:16:47 -07:00
27 changed files with 1154 additions and 148 deletions

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# Repository pre-commit hook: run only tests selected by staged paths.
set -euo pipefail
REPO_ROOT="$(git rev-parse --show-toplevel)"
exec "$REPO_ROOT/scripts/test-changed.sh"
+16 -12
View File
@@ -27,7 +27,7 @@ host.
| Target | Runtime | Service | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT Q8_0_ROCMFPX (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8 Uncensored Q6_0_ROCmFPX_AGENT (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -35,8 +35,9 @@ Dankclaw. On 2026-08-21 a DFlash2 Vulkan deployment briefly replaced CIRU,
then was reverted the same day for speed. On 2026-08-22 the service briefly
swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day
(operator decision). On 2026-08-25 the service switched to the kingjones
Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model (standalone MTP draft, 262144 context,
reasoning on); the previous Kairic Edge model was retired the same day. Runbooks:
Qwen3.8-27B-Uncensored-Q6_0-ROCmFPX_AGENT model (standalone MTP draft,
262144 context, reasoning on); the previous Q8_0 AGENT model was retired after
its `set_rows` crash. Runbooks:
[docs/LLM/index.md](docs/LLM/index.md).
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
@@ -182,19 +183,22 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting
### kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT llama.cpp (Dankclaw, ACTIVE)
### kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT llama.cpp (Dankclaw, ACTIVE)
The large service serves `kingjones/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT`
(public) from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/`
with a charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
The large service serves `kingjones/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP`
(public) from `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` with a
charlie12345/ROCmFPX `llama-server` (build `build-promptforge`). It runs a
standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full
262144 context / 1 slot, and reasoning on (Sharp template, budget -1,
`--reasoning-format none`; chat kwargs keep `preserve_thinking` +
`preserve_reasoning`). The payload (~28 GB main + 1.7 GB draft + 0.9 GB
mmproj, ~30 GB total) plus the 262k KV cache fit in the lifted 114 GiB TTM/GTT
budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
`scripts/serve-frank-large.sh`, runbook
[docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
`preserve_reasoning`). The Q6_0 main model is 24.86 GB. Env
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`, and
runbook `docs/LLM/kingjones-q8agent-rocmfpx.md` define the deployment.
**Serving status (2026-09-01):** restored the previously deployed Q6_0 AGENT
model. The Q8_0 AGENT experiment is documented in
`docs/LLM/kingjones-q8agent-rocmfpx.md`; it was retired because it crashed in
`ggml_compute_forward_set_rows` and did not provide a working service.
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
+55
View File
@@ -0,0 +1,55 @@
# Hermes configuration backup / non-secret baseline.
#
# Source on Dankclaw: /home/dank/hermes-data/config.yaml
# This file is a sanitized Git backup, not a complete replacement for the
# live Hermes state. Do not add dashboard password hashes, phone numbers,
# OAuth credentials, API keys, sessions, memories, or generated state here.
# The live directory is bind-mounted into the containers as /data.
model:
provider: custom
base_url: http://192.168.0.143:52625/v1
api_key: not-needed
default: frank-default-large
providers:
custom:
stale_timeout_seconds: 1800
request_timeout_seconds: 1800
dankserver-small:
api: http://192.168.0.235:7770/v1
discover_models: false
models:
frank-default-small:
context_length: 163840
# Auxiliary routing from the live configuration. These use the existing
# custom endpoint shape and are retained here as a settings backup.
auxiliary:
vision: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
web_extract: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
compression: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
skills_hub: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
approval: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
mcp: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
title_generation: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
memory_query_rewrite: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
tts_audio_tags: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
triage_specifier: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
kanban_decomposer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
profile_describer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
curator: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
dashboard:
# basic_auth intentionally omitted; its password_hash remains live-only.
font: atkinson-hyperlegible
skills:
disabled: []
onboarding:
seen:
busy_input_prompt: true
_config_version: 34
platforms:
signal:
enabled: true
# extra.http_url and account intentionally omitted from the backup.
+22
View File
@@ -13,6 +13,28 @@ This repository tests its configuration at defined seams. The tests run with `ba
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
| Deployment flow | `tests/deployment_flow.bats` | deploy.sh exists, targets known, SSH reachable (skips offline) |
## Focused pre-commit testing
For normal commits, use the focused wrapper:
```bash
./scripts/test-changed.sh
```
It reads staged paths with `git diff --cached` and runs only the mapped Bats
files. The tracked `.githooks/pre-commit` hook invokes this wrapper. Enable the
repository hook path once per checkout:
```bash
git config core.hooksPath .githooks
```
Use `--list` to inspect the selection without running tests, or `--paths FILE
...` to preview a mapping. Shared test infrastructure and unmapped paths fail
closed rather than silently skipping coverage. Run the full suite with
`./tests/run.sh` when changing shared test helpers, the test runner, or when
focused coverage is not sufficient.
## Red-Green
Tests are written first. A failing test means the seam is not satisfied; the config change that makes it pass is the only acceptable fix. No `--no-verify`, no deleting tests without approval.
+1 -1
View File
@@ -22,7 +22,7 @@ The gateway container runs `hermes gateway run` (the Signal platform). That entr
The dashboard refuses to bind to a non-loopback address without an auth provider. There is no unauthenticated public bind. This repo uses basic auth:
- `dashboard.basic_auth.username` and `dashboard.basic_auth.password_hash` live in the git-ignored `/home/dank/hermes-data/config.yaml`.
- `dashboard.basic_auth.username` and `dashboard.basic_auth.password_hash` live in the live `/home/dank/hermes-data/config.yaml`; the tracked backup at `dankclaw/hermes/config.yaml.example` intentionally omits the password hash.
- The stored value is a one-way scrypt hash. The plaintext password is never written to a file or the repo.
- The hash is generated with `plugins.dashboard_auth.basic.hash_password`.
+11 -2
View File
@@ -4,6 +4,10 @@ Hermes is the self-improving AI agent from Nous Research. It runs in a terminal,
**Host:** Dankclaw — **Hostname:** `hermes.dankclaw.local` — **Docs:** https://hermes-agent.nousresearch.com/docs
**Persistent state:** `/home/dank/hermes-data` on Dankclaw, mounted as `/data` in
both containers. The tracked, sanitized configuration baseline is
[`dankclaw/hermes/config.yaml.example`](../../dankclaw/hermes/config.yaml.example).
## Facts (from research, primary source)
| Item | Value |
@@ -17,15 +21,20 @@ Hermes is the self-improving AI agent from Nous Research. It runs in a terminal,
## Configuration
- Point Hermes at the local model endpoint (`http://localhost:7770/v1`) or an external provider through its provider config.
- Point Hermes at the local model endpoint or an external provider through its provider config.
- The gateway process exposes one port for chat integration; the exact port is set in the Hermes config (`8090` convention in this repo).
- The data volume persists skills, memory, and session history.
- The configured `dankserver-small` provider reaches `http://192.168.0.235:7770/v1`
and exposes the `frank-default-small` alias. It is a named provider, so it can
coexist with the existing `custom` provider for the large model.
- The live `config.yaml` contains secrets and user state and is not committed;
the tracked example is the sanitized backup of its non-secret settings.
## Volumes
| Container path | Host path (convention) |
|----------------|------------------------|
| `/data` | `/opt/hermes/data` |
| `/data` | `/home/dank/hermes-data` |
## Gotchas
+91 -21
View File
@@ -15,6 +15,26 @@ engine for an hour to 15 days and silently drops it from results. That is the
This page is the fixbook. Read [[SearXNG]] for the stack; the sub-pages below
are the detail if you need it.
## Provider policy
The default aggregation should maximize **independent, anonymous coverage**
without turning one user query into a request storm. Disabled engines remain
installed and selectable on demand; `inactive` is reserved for engines that
should disappear completely.
| Category | Default baseline | On-demand / deferred |
|----------|------------------|----------------------|
| General/web | Bing, DuckDuckGo, Wikipedia | Google CSE, Brave, Startpage, Qwant, Mojeek, Wiby, MWMBL after retesting |
| Technical | GitHub, MDN, Stack Overflow, Ask Ubuntu, Superuser, Docker Hub, PyPI | Other technical indexes as needed |
| Science | arXiv, PubMed, Semantic Scholar, OpenAlex-compatible sources | Google Scholar until independently tested |
| News | Reuters, Bing News, DuckDuckGo News, Wikinews | Google News, Brave News, Startpage News |
| Images | Openverse, Wikimedia Commons, Bing Images, Unsplash | Google/Brave/Startpage image engines |
| Videos | YouTube, Vimeo, Dailymotion, Bing Videos | Brave video engines |
This policy uses no API credentials and does not identify the operator to a
search provider. It is deliberately conservative: a bundled engine is not
necessarily healthy, anonymous, or suitable for every default query.
## Root cause (one line)
All SearXNG engine traffic shares **one ProtonVPN exit IP** with no rotation, so
@@ -51,6 +71,12 @@ Most "Google is broken on a VPN host" cases are a **mix of both**.
## Fix A — Engine config (settings.yml `engines:`)
The current tracked policy enables Bing and DuckDuckGo for default general
search, gives each weight 2, disables pagination, and allows 5 seconds for the
VPN path. CAPTCHA/429-prone Google, Brave, and Startpage families are retained
but disabled from default aggregation. This limits fan-out while preserving
explicit/on-demand selection.
Important: SearXNG's engines are configured in the **`engines:` list inside
`settings.yml`**, not a separate `engines.yml` file. With
`use_default_settings: true` (our setting) your `engines:` list is *merged* over
@@ -137,12 +163,12 @@ Tune the `outgoing` block and the suspend model to survive load:
```yaml
outgoing:
request_timeout: 6.0 # raise from 2.0 so slow engines aren't dropped as unresponsive
max_request_timeout: 15.0
pool_connections: 200 # bigger pool = more concurrent connections per engine host
pool_maxsize: 40 # keep-alive connections per host
enable_http2: true # multiplexing = fewer distinct "new connection" events per engine
retries: 3 # on 429/5xx, retry against a different proxy + source IP
request_timeout: 4.0 # current Dankclaw value; modest VPN-latency allowance
max_request_timeout: 12.0
pool_connections: 100
pool_maxsize: 20
enable_http2: true
retries: 1 # avoid amplifying 429/CAPTCHA responses from one exit IP
useragent_suffix: "ops@yourhomelab.example" # so a per-engine blocker can reach you
search:
@@ -159,8 +185,12 @@ Notes:
that is a historical-searx concept. The functional equivalent is the suspend
schedule above (widen to back off before blocking, narrow to recover faster).
- `request_timeout` interacts with the suspend model: too tight → engines look
"unresponsive"; too loose → slow pages. Aim per-engine `timeout` at
*proxy-add-on time + engine latency*, capped by `max_request_timeout`.
"unresponsive"; too loose → slow pages. Dankclaw uses 4 seconds globally and
12 seconds as the cap; raise it only when measurements show real upstream
latency rather than 429/CAPTCHA responses.
- Keep retries low: retrying a 429/CAPTCHA through the same VPN exit can make the
provider's rate limit worse. A retry count of 1 is a bounded compromise for a
transient network failure.
- `safe_search: 0` when diagnosing — a too-strict global safe search can make an
engine return zero rows and *look* blocked.
@@ -226,6 +256,39 @@ gateway.
> containers/systemd/<unit>.container` `Environment` to inject names into the
> tunnel's `/etc/hosts`.
## Observed Dankclaw baseline (2026-08-25)
The read-only audit found:
- SearXNG image: `ghcr.io/searxng/searxng:latest`; embedded version `1.0.0`.
- Gluetun uses `VPN_SERVICE_PROVIDER=custom` with a static WireGuard profile;
no `SERVER_COUNTRIES`, `SERVER_CITIES`, `SERVER_REGIONS`, or
`SERVER_HOSTNAMES` filters are configured.
- The observed exit was `149.22.88.19` (Seattle, Washington, US).
- Bing and DuckDuckGo returned results. Mojeek returned no results without an
upstream error.
- `google cse` and Brave were suspended for too many requests; Startpage was
suspended for CAPTCHA; Qwant returned CAPTCHA. SearXNG logs also recorded
Google's "unusual traffic from your network" response.
- `/healthz` returned `200 OK`; the SearXNG service and Gluetun service were
healthy. This isolates the primary failures to upstream provider handling,
not application availability.
This deployment returns the legacy array form for `unresponsive_engines`:
```json
"unresponsive_engines": [
["brave", "Suspended: too many requests"],
["startpage", "Suspended: CAPTCHA"]
]
```
Do not assume the newer object form (`engine`, `error_type`, `suspended`)
until the live response has been checked. The direct published port is the
useful diagnostic path because it exercises the ProtonVPN exit. The Caddy
hostname resolved to the WAN address during the audit and timed out from the
audit host, so it was not used as the baseline path.
## Fix D — Verify an engine is unblocked
Three layers, cheapest first. All queries should hit the path production uses
@@ -242,11 +305,11 @@ curl -s "$BASE/search?q=test&engines=google&format=json" \
Inspect:
- `content[].engine` — which engine produced each result.
- `unresponsive_engines[]` — `{engine, error_type, suspended}`. `error_type`
tells you *why* (`SearxEngineTooManyRequests`, `SearxEngineCaptcha`,
`timeout`, `SearxEngineAccessDenied`); `suspended: true` = auto-banned right
now.
- `timings[].engine` — per-engine latency.
- `unresponsive_engines[]` — inspect the live shape first. On Dankclaw it is
`[engine, message]`, for example `["google cse", "Suspended: too many requests"]`.
Newer builds may return objects containing `engine`, `error_type`, and
`suspended`.
- `timings[].engine` — per-engine latency when the running build emits timings.
Full status check (bash + jq):
@@ -285,16 +348,23 @@ misleading).
## Recommended path for Dankclaw (ranked)
1. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
1. **Measure individual engines** — use `/json` + `?engines=<name>` and several
query classes before enabling a bundled engine in default aggregation.
2. **Keep default fan-out small** — prefer independent anonymous providers and
disable pagination for rate-sensitive engines.
3. **Promote by evidence** — raise `weight` only for engines that return useful
results consistently; weight is not a health check.
4. **Avoid credentials** — do not add API-backed engines unless the provider is
genuinely free, anonymous, and compatible with our privacy requirements.
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
exactly which engines are down and why. Confirm the running SearXNG version
(check whether `parsers`/`rate_limit` exist in the pinned build).
2. **Tighten Fix B** — raise `outgoing.request_timeout`, add `retry_on_http_error:
[403, 429]` on the Google engines, widen `suspended_times`. Cheapest, no new
infra.
3. **Duplicate the Google engines** (Fix A) so a CAPTCHA ban on one leaves the
other returning results.
4. **If Google still re-blocks** — Fix C Option 1: point the Google engines at a
dedicated-IP proxy (Option 2 dedicated second Gluetun if reputation
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds per
upstream request, caps it at 15 seconds, and retries once. This addresses
modest VPN latency without multiplying 429/CAPTCHA traffic. Adjust only
after measuring the resulting error and latency profile.
7. **If Google still re-blocks** — Fix C Option 1: point the Google engines at
a dedicated-IP proxy (Option 2 dedicated second Gluetun if reputation
isolation is needed). Avoid the same-netns-proxy trap.
---
+33 -7
View File
@@ -61,9 +61,29 @@ valkey:
url: "valkey://searxng-valkey:6379/0"
outgoing:
request_timeout: 3.0
max_request_timeout: 10.0
retries: 3
request_timeout: 5.0
max_request_timeout: 15.0
retries: 1 # avoid amplifying upstream 429/CAPTCHA responses
# Anonymous, bounded default aggregation. Disabled engines remain configured
# for explicit/on-demand selection; see SearXNG Providers for the policy.
engines:
- name: bing
disabled: false
weight: 2
paging: false
timeout: 5.0
- name: duckduckgo
disabled: false
weight: 2
paging: false
timeout: 5.0
- name: google cse
disabled: true
- name: brave
disabled: true
- name: startpage
disabled: true
```
## Volumes
@@ -96,10 +116,16 @@ Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local s
## Unblocking Google & Search Providers
SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP. The
rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429,
and SearXNG **suspends** the offending engine for hours to days, which is what
shows up as "Google went unavailable". This is upstream, not local.
The current request policy allows a modest amount of VPN latency (`4.0s` per
request, `12.0s` maximum) while limiting retries to one. This can recover slow
providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared
VPN exit IP. SearXNG queries all ~50 upstream engines from one Gluetun VPN exit
IP. The rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429.
The default provider policy therefore favors anonymous Bing and DuckDuckGo,
disables observed CAPTCHA/429 providers from normal aggregation, and disables
pagination to reduce upstream request volume. SearXNG **suspends** offending
engines for hours to days, which is what shows up as "Google went unavailable".
This is upstream, not local.
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
+43
View File
@@ -0,0 +1,43 @@
# Entropy Volume
The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at
`/mnt/entropy`. The command-line helper reuses the host's desktop-integrated
`udisks2`, `crypttab`, and `fstab` configuration; it does not store or handle
the passphrase itself.
## Command-line mount
```bash
/home/dank/serverAppConfigs/scripts/mount-entropy.sh
```
Run it as the normal desktop user (`dank`), not from a root shell. `udisksctl`
will securely prompt for the LUKS passphrase. `sudo mount /mnt/entropy` then
uses the existing `/etc/fstab` entry, preserving its options:
```text
nosuid,nodev
```
The existing mountpoint is owned by `dank:dankserver` with mode `0755`; the
helper deliberately does not run `chmod` or `chown`, so it cannot silently
change those permissions.
## Device path
The current storage chain is:
```text
/dev/md0 -> LUKS UUID cd86c728-459e-4f3a-bcc2-41ecf84685ec
-> ext4 label entropy
-> /mnt/entropy
```
The helper checks that `/dev/md0` exists and exits successfully if the volume
is already mounted. It is intentionally specific to this host and volume:
that avoids accidentally unlocking or mounting a different disk.
## Related
- [[Storage Policy]]
- [[Dankserver Stack]]
+10 -10
View File
@@ -18,7 +18,7 @@ repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT, standalone MTP, 262K ctx / 1 slot) | frank-default-large | 7770 |
## Workflow
@@ -138,21 +138,21 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
### Dankclaw — frank-default-large
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
[kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
under the alias `frank-default-large` on port 7770. The main model is
`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the Q8_0 vision projector is loaded alongside
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
service keeps the existing 65536 context / 1-slot profile. The unit file is
service uses the 262144 context / 1-slot profile. The unit file is
`scripts/frank-default-large.service`; it launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
> See [[dflash2-large-vulkan]] for the archived runbook.
> The Q8_0 AGENT experiment was retired on 2026-09-01 after repeated
> `set_rows` crashes and device-local allocation failures. The former caf-eemin
> ROCmI4 embedded-MTP model and the v3 PromptForge payload are historical
> deployments. The intermediate DFlash2 Vulkan deployment was retired on
> 2026-08-21. See [[kingjones-q8agent-rocmfpx]] and [[dflash2-large-vulkan]].
## Validation
+9 -7
View File
@@ -3,7 +3,7 @@
Documentation for the LLM inference fleet on two local machines
(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
(frank-default-large, kingjones Qwen3.8 Uncensored Q6 AGENT). Both
services run the Qwen-Sharp chat template.
> 2026-08-22: the former `vllmScripts` repository is merged into the
@@ -26,9 +26,9 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
D[gfx1151 iGPU 64 GiB unified memory]
E[CIRU ROCmFPX llama.cpp PromptForge v3
E[charlie12345/ROCmFPX build-kairic (v81)
frank-default-large, port 7770]
F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
F[Qwen3.8 Uncensored Q6 ROCmFPX AGENT
MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
D --> E --> F
end
@@ -48,14 +48,16 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [kingjones Qwen3.8 Uncensored ROCmFP4 STRIX MTP](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF) — active frank-default-large model on Dankclaw; Q6 AGENT main model with standalone MTP
- [[kingjones-strix-mtp]] — the active Q6 AGENT runbook: serving profile, standalone MTP, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
on Dankclaw (retired)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, active 2026-08-24)
- [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
model on Dankclaw (superseded 2026-08-25 by the ROCmI4 embedded-MTP model)
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, active 2026-08-24)
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime
+65 -5
View File
@@ -1,6 +1,9 @@
# frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
**Status:** ACTIVE on Dankclaw (since 2026-08-25).
**Status:** Deployed on Dankclaw (since 2026-08-25). Serving is **BLOCKED by
the `set_rows` build bug** — the model loads but crashes in
`ggml_compute_forward_set_rows` before it can serve. All config goals are met;
a build fix is required for the model to run.
**Alias:** `frank-default-large`, port **7770**.
@@ -26,6 +29,53 @@ fit in the lifted **114 GiB TTM/GTT** budget (see the 114 GiB change in
| Tunables | `scripts/frank-default-large.env` |
| Serve script | `scripts/serve-frank-large.sh` |
## Serving status: the `set_rows` build bug (blocks serving)
As of 2026-08-25 the Q8_0 AGENT model will not serve on Dankclaw. It loads,
reaches `initializing slots, n_slots = 1`, but never prints `start serving`.
Health stays `503` until the process exits. The crash is in
`ggml_compute_forward_set_rows` in `libggml-cpu.so.0`, a NULL function-pointer
jump (address `0x00000000`).
The backtrace (gdb). The main thread and one OpenMP worker crash in the same
path:
```mermaid
graph TD
A[llama_context::graph_compute] --> B[ggml_backend_cpu_graph_compute]
B --> C[ggml_graph_compute]
C --> D[GOMP_parallel]
D --> E[ggml_compute_forward_set_rows]
E --> F[NULL dispatch -> address 0x0]
D --> G[Thread 23 worker: same path]
```
Test matrix. Five ROCmFPX builds and the all-CPU path were tested on a scratch
port (`/tmp/g80bench/run.sh`), `gl=45`, `ctx=131072`. All crash in `set_rows`.
| Build | Binary | Version | Result |
| ----- | ------ | ------- | ------ |
| build-kairic | `ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server` | v81 (205a3e5f4) | segfault in set_rows |
| build-promptforge | `ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` | PromptForge/ActiveFPX lineage | segfault in set_rows |
| build-rocmi4-w4a4 | `rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` | v257 (c49ebdb) | segfault in set_rows |
| build-strix-rocmfp4 | `GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-server` | v209 (00d5452) | segfault in set_rows |
| all-CPU | build-kairic, `-ngl 0` | v81 | segfault in set_rows |
The crash is not GPU-specific (the all-CPU path fails too), not MTP-specific
(`--spec-type none` fails), and not fixed by `LLAMA_SET_ROWS=0`. So it is a
build bug: the Q8_0 AGENT GGUF graph needs a `set_rows` dispatch the ROCmFPX
backends do not register. The older caf-eemin ROCmI4 and Kairic Edge models
served fine on related ROCmFPX builds. The caf-eemin payload was about 14.4
GiB; the separate
PromptForge/ActiveFPX deployment used a much larger multi-file payload of about
36.7 GiB. These are different deployments. The Q8_0 AGENT graph is the trigger
for the `set_rows` failure.
**Fix path:** rebuild charlie12345/ROCmFPX from source with a working
`set_rows` for the ROCmFPX backend, or find a build where it is registered.
This unblocks the model. See the context budget reality below once `set_rows`
is fixed.
## Model files
Three files, from the HF repo above:
@@ -93,9 +143,14 @@ Key choices:
- **f16 K + TurboQuant V** cache. f16 K is the safe long-ctx default; TurboQuant
halves the V cache at 262k while keeping quality (see the ubatch/threads-batch
notes in [[qwen3.8-strix-optimizations]]).
- **Full 262144 context, 1 slot.** The whole context KV cache plus the ~30 GB
payload (main + draft + mmproj) fit in the 114 GiB budget. Concurrent
requests queue; the server never preempts a running generation.
- **262144 context, 1 slot — the target.** The whole context KV cache plus the
~30 GB payload (main + draft + mmproj) are meant to fit in the 114 GiB
budget. Measured reality: `gl=999` (all layers on ROCm0) at 262144 **OOMs**
on device-local (`cudaMalloc failed`, the KV cache needs ~10-18 GiB that does
not fit once the payload is on the card). `gl=45` (partial offload) fits a
131072 context. The env ships `CTX_SIZE=262144`; the proven serving context
today is **131072 at `gl=45`** (before the `set_rows` fix). `deploy.sh` waits
up to 20 minutes (see `LLM_WAIT`).
- **Reasoning on.** The Sharp template drives thinking (medium effort);
`--reasoning-format none` lets the template project the reasoning content into
`reasoning_content`. `--reasoning-budget -1` = no budget cap.
@@ -160,6 +215,11 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
- **2026-08-25 (today):** switched the large service to the kingjones
Q8_0-AGENT model with 262144 context and reasoning on. Downloaded the three
payload files from the HF repo, raised the TTM/GTT to 114 GiB for the full
context, and persisted the env/serve script in this repo.
context, and persisted the env/serve script in this repo. Measured that the
model loads but crashes in `ggml_compute_forward_set_rows` on all five
available ROCmFPX builds; serving is BLOCKED pending a rebuild. The config
goals (GTT 114 GiB, reasoning on, preserve_thinking/preserve_reasoning on,
262144 env context) are in place. Proven serving context today is 131072 at
`gl=45`. See the `set_rows` build bug section above.
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
retired the same day for the AGENT model). See [[kairic-edge]].
+6 -7
View File
@@ -15,13 +15,12 @@ timestamp: 2026-08-24
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
> **Retired 2026-08-25.** The kingjones ROCmFP4 STRIX MTP payload has been
> replaced by the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on the same
> port and iGPU, under a new runtime (charlie12345/ROCmFPX ROCmI4 W4A4). See
> [[eemin-rocmi4-mtp]]. The old kingjones GGUF + standalone MTP Q4_0 draft are
> retired on this slot. The CIRU ROCmFPX runtime/binary is also retired here.
> **Active again 2026-09-01.** The kingjones ROCmFP4 STRIX MTP payload is
> restored as `frank-default-large` after the Q8_0 AGENT experiment. The Q8_0
> model was retired because it crashed before serving. The Q6 AGENT model,
> standalone MTP draft, Q8 projector, and PromptForge runtime are now active.
> _This page documents the retired model and its runbook for reference._
> This page documents the active model and its runbook.
`frank-default-large` (port 7770) serves
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
@@ -34,7 +33,7 @@ here; the head is loaded with `--model-draft`.
The card ships six text files plus the vision projector. The service uses
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
and the Q8 vision projector.
and the Q8 vision projector (`mmproj-Qwen3.8-27B-Q8_0.gguf`).
| File | Size |
|------|------|
+60
View File
@@ -0,0 +1,60 @@
---
type: benchmark plan
title: frank-default-large ubatch sweep
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
tags:
- llama.cpp
- rocm
- benchmark
- ubatch
timestamp: 2026-09-01
---
# frank-default-large ubatch sweep
This benchmark sweeps the active `frank-default-large` model on Dankclaw using
scratch ports and a 32K-context workload. The production service is stopped
before the sweep and restarted by the benchmark's exit handler.
## Sweep
The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
5120, 6144`. This implements the requested 256-step range through 1536 and
1024-step range from 2048 through 6144. Each point runs three 256-token decode
requests by default.
Run on Dankclaw from the repository checkout:
```bash
STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
```
The script uses the active Q6 AGENT model, standalone MTP draft, a fixed
32K batch, and the tracked Sharp
template, and `--reasoning-format deepseek`. It reports server timing lines and
per-run generation throughput. Run only when the production service can be
interrupted; the GPU is shared with live workloads.
## Results
Record results below after the sweep, including failures or OOM points.
| ubatch | prefill | generation | notes |
| ---: | ---: | ---: | --- |
| 256 | — | — | pending |
| 512 | — | — | pending |
| 768 | — | — | pending |
| 1024 | — | — | pending |
| 1280 | — | — | pending |
| 1536 | — | — | pending |
| 2048 | — | — | pending |
| 3072 | — | — | pending |
| 4096 | — | — | pending |
| 5120 | — | — | pending |
| 6144 | — | — | pending |
## See also
- [LLM fleet index](index.md)
- [Deployment](deployment.md)
- [Active kingjones runbook](kingjones-strix-mtp.md)
+8 -3
View File
@@ -30,6 +30,10 @@ covers both workloads).
- [[Deployment Flow]] — git-based deploy, target map, health checks
- [[Testing Policy]] — the test seams and how to run them
## Dankserver Storage
- [[Entropy Volume]] — command-line unlock and mount helper
## Dankserver Services
- [[RomM]] — retro game manager
@@ -97,9 +101,9 @@ The LLM inference fleet on the two hosts. Full index:
- [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
- [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (restored 2026-09-01)
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
@@ -112,6 +116,7 @@ The LLM inference fleet on the two hosts. Full index:
### Benchmarks and Sizing
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
- [[frank-default-small-benchmark]] — the small-model benchmark suite
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
+57 -28
View File
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
# bench-ubatch-rocmi4.sh — sweep ubatch size at 8k context on the
# frank-default-large ROCmI4 embedded-MTP stack (dankclaw, gfx1151).
# frank-default-large ROCmFPX standalone-MTP stack (dankclaw, gfx1151).
#
# Boots one llama-server instance per ubatch on a scratch port with a fixed
# -b and -c 8192, runs a standardized ~75%-of-ctx prose workload, and reads
# -b and -c 32768, runs a standardized ~75%-of-ctx prose workload, and reads
# authoritative prefill/generation tok/s from the server's own print_timing
# lines. Steps from UBATCH_MIN to UBATCH_MAX inclusive by UBATCH_STEP.
#
@@ -26,21 +26,23 @@ if [[ -f "$ENV_FILE" ]]; then
fi
# --- sweep parameters -------------------------------------------------------
: "${UBATCH_MIN:=512}"
: "${UBATCH_MAX:=4096}"
: "${UBATCH_STEP:=512}"
: "${CTX:=8192}" # 8k contexts
: "${BENCH_BATCH:=$CTX}" # fixed batch = ctx, so the full 8k prompt fits
: "${UBATCH_MIN:=256}"
: "${UBATCH_MAX:=6144}"
: "${UBATCH_STEP:=256}"
: "${UBATCH_VALUES:=256 512 768 1024 1280 1536 2048 3072 4096 5120 6144}"
: "${CTX:=32768}" # 32k contexts
: "${BENCH_BATCH:=32768}" # batch is fixed to the 32k context size
: "${OUT_TOKENS:=256}"
: "${RUNS:=3}" # decode runs per ubatch
: "${PORT_BASE:=7894}" # scratch ports 7894..
: "${STOPSVC:=0}" # stop/restart the live unit for a clean GPU
: "${STOPSVC:=1}" # stop/restart the live unit for a clean GPU
# --- resolved paths (fall back to serve-script defaults) --------------------
: "${LLAMA_BIN:=/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server}"
: "${MODEL_DIR:=/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP}"
: "${MODEL_FILE:=Qwen3.8-27B-Q4_0_ROCMI4.gguf}"
: "${MMPROJ_FILE:=mmproj-BF16.gguf}"
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server}"
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP}"
: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-Q8_0.gguf}"
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
: "${ALIAS:=frank-default-large}"
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
@@ -68,7 +70,14 @@ if [[ -f "$MMPROJ" ]]; then
fi
STopped=0
SCRATCH_PID=""
cleanup() {
if [[ -n "$SCRATCH_PID" ]] && kill -0 "$SCRATCH_PID" 2>/dev/null; then
echo "=== stopping scratch llama-server (pid $SCRATCH_PID) ==="
kill "$SCRATCH_PID" 2>/dev/null || true
wait "$SCRATCH_PID" 2>/dev/null || true
SCRATCH_PID=""
fi
if [ "$STopped" = 1 ]; then
echo "=== restarting frank-default-large ==="
sudo systemctl restart frank-default-large || true
@@ -109,12 +118,12 @@ run_one() {
local log="/tmp/ubatch-${ubatch}-p${port}.log"
: > "$log"
echo "=== ubatch=${ubatch} ctx=${CTX} port=${port} runs=${runs} ==="
echo "=== ubatch=${ubatch} ctx=${CTX} batch=${BENCH_BATCH} port=${port} runs=${runs} ==="
"$LLAMA_BIN" \
--model "$MODEL" \
"${MMPROJ_ARGS[@]}" \
--chat-template-file "$CHAT_TEMPLATE_FILE" \
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
--alias bench-ubatch \
--host 127.0.0.1 --port "$port" \
-dev ROCm0 -ngl 999 -np 1 \
@@ -122,25 +131,32 @@ run_one() {
-c "$CTX" -b "$BENCH_BATCH" -ub "$ubatch" \
-fa on -ctk f16 -ctv f16 -ctxcp 16 \
--cache-ram 8192 --cache-prompt --metrics --no-mmap \
--spec-type draft-mtp --spec-mtp-strict-qwen \
--spec-draft-device ROCm0 --spec-draft-ngl all \
--spec-type draft-mtp --model-draft "$MTP_DRAFT_FILE" \
--spec-draft-device ROCm0 --spec-draft-ngl 999 \
--spec-draft-type-k f16 --spec-draft-type-v f16 \
--spec-draft-n-max "$MTP_N_MAX" --spec-draft-n-min 0 \
--spec-draft-p-min "$MTP_P_MIN" \
--spec-draft-backend-sampling \
--temp 0.7 --top-p 0.95 \
--reasoning on --reasoning-format deepseek --reasoning-budget -1 \
--reasoning "$REASONING" --reasoning-format "$REASONING_FORMAT" --reasoning-budget -1 \
-lv 4 >>"$log" 2>&1 &
local pid=$!
wait_health "$pid" "$port" || { tail -30 "$log"; return 2; }
SCRATCH_PID="$pid"
if ! wait_health "$pid" "$port"; then
tail -30 "$log"
kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
SCRATCH_PID=""
return 2
fi
# one python pass per run; reuse the same boot across runs
local r
for r in $(seq 1 "$runs"); do
echo "--- run ${r}/${runs} ---"
python3 - "$ubatch" "$port" "$out_tokens" "$CTX" "$log" <<'PY'
if ! python3 - "$ubatch" "$port" "$out_tokens" "$CTX" "$log" "$r" <<'PY'
import sys, time, json, urllib.request
ubatch, port, out_tokens, ctx, log = sys.argv[1], sys.argv[2], int(sys.argv[3]), int(sys.argv[4]), sys.argv[5]
ubatch, port, out_tokens, ctx, log, run_number = sys.argv[1], sys.argv[2], int(sys.argv[3]), int(sys.argv[4]), sys.argv[5], sys.argv[6]
# Flowing prose at ~75% of ctx so MTP acceptance is realistic.
VAR = (
@@ -175,8 +191,12 @@ body = json.dumps({
req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/completions",
data=body, headers={"Content-Type": "application/json"})
t0 = time.time()
with urllib.request.urlopen(req, timeout=7200) as resp:
out = json.load(resp)
try:
with urllib.request.urlopen(req, timeout=7200) as resp:
out = json.load(resp)
except Exception:
print("request failed; preserving benchmark cleanup", file=sys.stderr)
raise
dt = time.time() - t0
pt = out.get("usage", {}).get("prompt_tokens", 0)
ct = out.get("usage", {}).get("completion_tokens", 0)
@@ -190,24 +210,33 @@ def grab(arr):
g_tps = ""
if evals and ct:
# eval time line: "eval time = .. ms for N tokens"
ev = evals[-1].split("= ")[-1]
ms = float(ev.split(" for")[0].strip())
g_tps = f"{ct / (ms / 1000):.2f}"
print(f"== RESULT ubatch={ubatch} run={r} "
# eval timing line: "eval time = .. ms / N tokens (... tokens per second)"
import re
match = re.search(r"eval time =\s*([0-9.]+) ms", evals[-1])
if match:
ms = float(match.group(1))
g_tps = f"{ct / (ms / 1000):.2f}"
print(f"== RESULT ubatch={ubatch} run={run_number} "
f"PREFILL={grab(prefill)} GEN_TPS={g_tps} "
f"pt={pt} ct={ct} wall={dt:.1f}s")
PY
then
kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
SCRATCH_PID=""
return 2
fi
done
kill "$pid" 2>/dev/null || true
wait "$pid" 2>/dev/null || true
SCRATCH_PID=""
}
main() {
stop_live_service
local idx=0 ubatch port
printf "%-10s %-14s %-10s\n" "UBATCH" "PREFILL" "GEN_TPS"
for ubatch in $(seq "$UBATCH_MIN" "$UBATCH_STEP" "$UBATCH_MAX"); do
for ubatch in $UBATCH_VALUES; do
port=$((PORT_BASE + idx))
run_one "$ubatch" "$port" "$OUT_TOKENS" "$RUNS"
idx=$((idx + 1))
+9 -7
View File
@@ -154,11 +154,11 @@ fi
if [ '${COPY_SHARED[$TARGET]}' = '1' ]; then
echo '[3/5] Copying shared configs into place...'
mkdir -p /opt/searxng/config /opt/prometheus/config /opt/loki/config /opt/alloy/config
cp shared/searxng/settings.yml /opt/searxng/config/settings.yml
cp shared/searxng/limiter.toml /opt/searxng/config/limiter.toml
cp shared/prometheus/prometheus.yml /opt/prometheus/config/prometheus.yml
cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
${SUDO} cp shared/searxng/settings.yml /opt/searxng/config/settings.yml
${SUDO} cp shared/searxng/limiter.toml /opt/searxng/config/limiter.toml
${SUDO} cp shared/prometheus/prometheus.yml /opt/prometheus/config/prometheus.yml
${SUDO} cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
${SUDO} cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
echo ' Rendering Caddyfile (basicauth hash from .env)...'
# Extract the hash with grep/cut rather than sourcing .env: sourcing a
@@ -213,7 +213,9 @@ if [ '${COMPOSE_UP[$TARGET]}' = '1' ]; then
podman build --no-cache --label cfhash=\$cfhash -t \$img:latest -f dankclaw/\$dir/Containerfile .
fi
done
podman-compose -f '${COMPOSE[$TARGET]}' up -d
if ! podman-compose -f '${COMPOSE[$TARGET]}' up -d; then
echo ' WARN: compose restart failed; continuing with LLM deployment.' >&2
fi
# A Caddyfile render is a bind-mount content change, not a container
# config change: the caddy container often survives 'up -d' and keeps
@@ -233,7 +235,7 @@ if [ '${COMPOSE_UP[$TARGET]}' = '1' ]; then
done
if [ \"\$caddy_reloaded\" -ne 1 ]; then
echo \" Caddy reload failed; restarting the container to load the file.\"
podman restart caddy
podman restart caddy || true
fi
else
echo '[4/5] Skipping compose restart (stack runs as bare containers on this host)'
+20 -16
View File
@@ -1,4 +1,4 @@
# Tunables for frank-default-large (kingjones Qwen3.8-27B ROCmFPX Q8 AGENT,
# Tunables for frank-default-large (kingjones Qwen3.8 Uncensored Q6 AGENT,
# dankclaw, Strix Halo gfx1151).
#
# Runtime: ROCmFPX llama.cpp with TurboQuant support. Serves the kingjones
@@ -9,18 +9,22 @@
# HIP runtime reads those bare env vars (see the 2026-08-13 TEMP incident).
# llama.cpp reads --host/--port/--temp as CLI flags, never as env vars.
# ROCmFPX build with TurboQuant KV-cache support.
LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server
# PromptForge ROCmFPX build and its historical ROCm 7.15 SDK environment.
LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server
ROCM_DEVEL=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_devel
ROCM_LIB=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_libraries
ROCM_CORE=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_core
# Model files from https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF.
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
MODEL_FILE=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
# Model files from https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF.
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP
MODEL_FILE=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf
MTP_DRAFT_FILE=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf
MMPROJ_FILE=mmproj-Qwen3.8-27B-BF16.gguf
MMPROJ_FILE=mmproj-Qwen3.8-27B-Q8_0.gguf
# Qwen-Sharp chat template (tracked in the repo, staged on the host). The Sharp
# template drives thinking on by default (medium effort) and produces the
# peg-native + deepseek chat/reasoning shape the Pi agent expects.
# template drives thinking on by default (medium effort); the server emits
# reasoning in DeepSeek format so clients receive reasoning_content separately
# instead of literal <think> tags in content.
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
@@ -37,13 +41,13 @@ LISTEN_PORT=7770
CTX_SIZE=262144
PARALLEL=1
# Batch/ubatch follow the existing qualified Dankclaw profile.
BATCH_SIZE=512
UBATCH_SIZE=512
BATCH_SIZE=4096
UBATCH_SIZE=4096
NUM_THREADS=16
THREAD_BATCH=32
CACHE_TYPE_K=f16
CACHE_TYPE_V=turbo4
CTX_CHECKPOINTS=32
CACHE_TYPE_V=f16
CTX_CHECKPOINTS=16
CACHE_RAM=8192
# Standalone MTP draft head, loaded from MTP_DRAFT_FILE by the serve script.
@@ -60,10 +64,10 @@ MIN_P=0.0
PRESENCE_PENALTY=1.5
# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
# keep the reasoning trail in the conversation). reasoning-format none lets the
# template project the reasoning content into reasoning_content.
# keep the reasoning trail in the conversation). DeepSeek format projects the
# reasoning into the API's reasoning_content field instead of content tags.
REASONING=on
REASONING_FORMAT=none
REASONING_FORMAT=deepseek
# Strix Halo unified memory + native gfx1151 detection for this ROCmFPX build.
HSA_OVERRIDE_GFX_VERSION=11.5.1
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env bash
# Unlock and mount Dankserver's encrypted entropy volume.
#
# The desktop flow uses the system's crypttab and fstab entries. Reuse those
# entries here instead of embedding a passphrase, mount options, or ownership
# changes in this helper. udisksctl reads the passphrase through its secure
# desktop/terminal authentication path.
set -euo pipefail
MOUNTPOINT='/mnt/entropy'
MAPPER='/dev/mapper/luks-cd86c728-459e-4f3a-bcc2-41ecf84685ec'
if mountpoint -q "$MOUNTPOINT"; then
printf '%s is already mounted.\n' "$MOUNTPOINT"
exit 0
fi
if [ ! -e /dev/md0 ]; then
printf 'ERROR: RAID device /dev/md0 is not available.\n' >&2
exit 1
fi
if [ ! -e "$MAPPER" ]; then
udisksctl unlock --block-device /dev/md0
fi
# udisks2 may mount automatically as part of unlocking. Do not attempt a
# second mount in that case.
if ! mountpoint -q "$MOUNTPOINT"; then
sudo mount "$MOUNTPOINT"
fi
printf 'Mounted %s.\n' "$MOUNTPOINT"
+17 -13
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Serve the kingjones Qwen3.8-27B ROCmFPX Q8 AGENT model as frank-default-large.
# The deployment uses a standalone MTP draft head and Turbo4 V-side KV cache.
# Serve the kingjones Qwen3.8 Uncensored Q6 AGENT model as frank-default-large.
# The deployment uses a standalone MTP draft head and f16 KV cache.
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
@@ -12,23 +12,23 @@ if [[ -f "$ENV_FILE" ]]; then
set +a
fi
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server}"
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT}"
: "${MODEL_FILE:=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server}"
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP}"
: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-BF16.gguf}"
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-Q8_0.gguf}"
: "${ALIAS:=frank-default-large}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=7770}"
: "${CTX_SIZE:=262144}"
: "${PARALLEL:=1}"
: "${BATCH_SIZE:=2048}"
: "${UBATCH_SIZE:=512}"
: "${BATCH_SIZE:=4096}"
: "${UBATCH_SIZE:=4096}"
: "${NUM_THREADS:=16}"
: "${THREAD_BATCH:=32}"
: "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=turbo4}"
: "${CTX_CHECKPOINTS:=32}"
: "${CACHE_TYPE_V:=f16}"
: "${CTX_CHECKPOINTS:=16}"
: "${CACHE_RAM:=8192}"
: "${MTP_N_MAX:=4}"
: "${MTP_N_MIN:=0}"
@@ -55,14 +55,18 @@ if [[ ! -x "$LLAMA_BIN" ]]; then
exit 1
fi
SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib:$ROCM_PATH/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
ROCM_PATH="${ROCM_DEVEL:-${ROCM_PATH:-/opt/rocm}}"
if [[ -n "${ROCM_LIB:-}" && -n "${ROCM_CORE:-}" ]]; then
LIBRARY_PATH="$ROCM_LIB/lib:$ROCM_CORE/lib:$ROCM_DEVEL/lib/llvm/lib:$SERVER_DIR:${LD_LIBRARY_PATH:-}"
else
LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib:$ROCM_PATH/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
fi
exec /usr/bin/env \
ROCM_PATH="$ROCM_PATH" \
HIP_VISIBLE_DEVICES=0 \
HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
LD_LIBRARY_PATH="$LIBRARY_PATH" \
HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
GGML_CUDA_GRAPH_OPT=0 \
LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 \
LLAMA_MTP_CPU_ARGMAX_FASTPATH=1 \
+148
View File
@@ -0,0 +1,148 @@
#!/usr/bin/env bash
# Run only the Bats tests affected by the staged paths.
#
# Usage:
# scripts/test-changed.sh # inspect the staged commit
# scripts/test-changed.sh --list # print selected test files
# scripts/test-changed.sh --paths FILE ...
#
# --paths is intended for tests and local experimentation. The normal mode is
# deliberately staged-only so a pre-commit hook cannot accidentally validate
# an unrelated dirty working-tree file.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
LIST_ONLY=0
USE_PATHS=0
declare -a CHANGED=()
die() {
printf 'ERROR: %s\n' "$*" >&2
exit 2
}
while [ "$#" -gt 0 ]; do
case "$1" in
--list)
LIST_ONLY=1
shift
;;
--paths)
USE_PATHS=1
shift
while [ "$#" -gt 0 ] && [[ "$1" != --* ]]; do
CHANGED+=("$1")
shift
done
;;
--)
shift
CHANGED+=("$@")
break
;;
*)
die "unknown option '$1' (use --list or --paths FILE ...)"
;;
esac
done
cd "$REPO_ROOT"
if [ "$USE_PATHS" -eq 0 ]; then
while IFS= read -r -d '' path; do
CHANGED+=("$path")
done < <(git diff --cached --name-only -z --diff-filter=ACMRTUXB)
fi
[ "${#CHANGED[@]}" -gt 0 ] || {
printf 'No staged paths; no focused tests to run.\n'
exit 0
}
declare -a TESTS=()
add_test() {
local test="$1"
local existing
for existing in "${TESTS[@]:-}"; do
[ "$existing" != "$test" ] || return 0
done
TESTS+=("$test")
}
for path in "${CHANGED[@]}"; do
case "$path" in
tests/*.bats)
add_test "$path"
;;
tests/helpers.bash|tests/run.sh)
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
;;
.githooks/*)
add_test tests/changed_tests.bats
;;
dankserver/compose.yaml|dankclaw/compose.yaml)
add_test tests/compose_validity.bats
add_test tests/env_consistency.bats
;;
dankserver/Caddyfile|dankclaw/Caddyfile)
add_test tests/caddyfile.bats
;;
dankserver/.env.example|dankclaw/.env.example)
add_test tests/env_consistency.bats
;;
shared/homepage/*)
add_test tests/homepage_caddy.bats
add_test tests/homepage_config.bats
add_test tests/homepage_deploy.bats
add_test tests/homepage_pages.bats
add_test tests/homepage_checker.bats
;;
shared/searxng/*)
add_test tests/searxng_limiter.bats
add_test tests/searxng_quadlet.bats
;;
dankclaw/*.container|dankclaw/*.network)
add_test tests/open_webui_quadlet.bats
add_test tests/pihole.bats
add_test tests/searxng_quadlet.bats
;;
scripts/mount-entropy.sh)
add_test tests/mount_entropy.bats
;;
scripts/frank-default-large.env|scripts/serve-frank-large.sh)
add_test tests/frank_large_config.bats
add_test tests/llm_secrets.bats
add_test tests/scripts_syntax.bats
;;
scripts/frank-default-small.env|scripts/serve-frank-small.sh)
add_test tests/frank_small_config.bats
add_test tests/llm_secrets.bats
add_test tests/scripts_syntax.bats
;;
scripts/*.service)
add_test tests/llm_units.bats
add_test tests/scripts_syntax.bats
;;
scripts/*)
add_test tests/scripts_syntax.bats
;;
docs/*)
add_test tests/docs_completeness.bats
;;
dankclaw/hermes/*)
add_test tests/docs_completeness.bats
;;
*)
die "no focused-test mapping for '$path'; add one before committing"
;;
esac
done
if [ "$LIST_ONLY" -eq 1 ]; then
printf '%s\n' "${TESTS[@]}"
exit 0
fi
command -v bats >/dev/null 2>&1 || die 'bats is not installed'
printf 'Running focused tests for %d changed path(s):\n' "${#CHANGED[@]}"
printf ' %s\n' "${TESTS[@]}"
bats --print-output-on-failure "${TESTS[@]}"
+234
View File
@@ -0,0 +1,234 @@
#!/usr/bin/env bash
#
# tmux-resurrect.sh — snapshot and restore the exact layout of a tmux session
# on Dankclaw, including each pane's start directory and foreground command.
#
# Use it after a reboot (or when the display session is gone) to reopen the
# same windows/panes on the Dankclaw console:
#
# scripts/tmux-resurrect.sh snapshot # record the live session to JSON
# scripts/tmux-resurrect.sh restore # rebuild it, then attach
#
# Data file: ~/.tmux-resurrect/<session>.json (set TMUX_RES_DIR to relocate).
# Re-run snapshot as the session changes so the restore stays faithful.
#
# How each pane's command is captured: a tmux pane runs one main shell
# (#{pane_pid}). The foreground command is that shell's direct child — chosen
# as the child that launched its own child (ssh, sudo ...) or the first child
# when none did (btop sitting at the top of a pane). An idle shell yields no
# command and is left at its prompt on restore.
set -euo pipefail
TMUX_RES_DIR="${TMUX_RES_DIR:-$HOME/.tmux-resurrect}"
DEFAULT_SESSION="0"
die() { printf 'tmux-resurrect: %s\n' "$*" >&2; exit 1; }
b64() { base64 -w0; }
b64d() { base64 -d; }
# fg_command <shell_pid>: print the foreground command of a tmux pane, or the
# empty string when the shell is idle.
fg_command() {
local pid="$1" kids=() kid cmd_pid=""
mapfile -t kids < <(pgrep -P "$pid" 2>/dev/null || true)
if [ "${#kids[@]}" -eq 0 ]; then return 0; fi
# Prefer a child that launched its own child (a launcher such as ssh or
# sudo); fall back to the first direct child (e.g. btop at the top).
for kid in "${kids[@]}"; do
if pgrep -P "$kid" >/dev/null 2>&1; then
cmd_pid="$kid"; break
fi
done
if [ -z "$cmd_pid" ]; then cmd_pid="${kids[0]}"; fi
tr '\0' ' ' < "/proc/$cmd_pid/cmdline" 2>/dev/null | sed -E 's/[[:space:]]+$//'
}
# snapshot <session> [dest.json]
snapshot() {
local session="${1:-$DEFAULT_SESSION}" dest="${2:-}"
local exists
exists=$(tmux list-sessions -F '#{session_name}' 2>/dev/null | grep -Fxq "$session" && echo 1 || echo 0)
[ "$exists" = 1 ] || die "no tmux session named '$session'"
mkdir -p "$TMUX_RES_DIR"
[ -z "$dest" ] && dest="$TMUX_RES_DIR/$session.json"
python3 - "$dest" "$session" 3< <(
{
while IFS=$'\t' read -r widx wname wlayout; do
printf 'W\t%s\t%s\t%s\n' "$widx" "$(printf '%s' "$wname" | b64)" "$(printf '%s' "$wlayout" | b64)"
while IFS=$'\t' read -r pidx ppid pdir; do
pcmd=$(fg_command "$ppid")
printf 'P\t%s\t%s\t%s\t%s\n' "$widx" "$pidx" \
"$(printf '%s' "$pdir" | b64)" "$(printf '%s' "$pcmd" | b64)"
done < <(tmux list-panes -t "$session:$widx" \
-F $'#{pane_index}\t#{pane_pid}\t#{pane_current_path}' | sort -n -k1,1)
done < <(tmux list-windows -t "$session" \
-F $'#{window_index}\t#{window_name}\t#{window_layout}' | sort -n -k1,1)
}
) > /dev/null <<'PY'
import sys, json, base64, os
dest, session = sys.argv[1], sys.argv[2]
wmap, panes = {}, {}
for line in os.fdopen(3).read().splitlines():
if not line:
continue
p = line.split('\t')
if p[0] == 'W':
wmap[int(p[1])] = {"name": base64.b64decode(p[2]).decode(),
"layout": base64.b64decode(p[3]).decode()}
elif p[0] == 'P':
widx = int(p[1])
panes.setdefault(widx, []).append({
"index": int(p[2]),
"dir": base64.b64decode(p[3]).decode(),
"cmd": base64.b64decode(p[4]).decode(),
})
windows = []
for widx in sorted(wmap):
windows.append({
"index": widx,
"name": wmap[widx]["name"],
"layout": wmap[widx]["layout"],
"panes": sorted(panes.get(widx, []), key=lambda x: x["index"]),
})
with open(dest, "w") as f:
json.dump({"session": session, "windows": windows}, f, indent=2, ensure_ascii=False)
f.write("\n")
PY
printf 'snapshot: wrote %s (session %s)\n' "$dest" "$session"
}
# emit_plan <src.json> [session]: python reads the JSON and prints a base64
# TSV plan (W/P records) for the bash orchestrator to consume.
emit_plan() {
python3 - "$1" "$2" <<'PY'
import sys, json, base64
src, session = sys.argv[1], sys.argv[2]
with open(src) as f:
data = json.load(f)
wins = data.get("windows", [])
if session:
wins = [w for w in wins if str(w.get("index")) == "0"]
def b64(s): return base64.b64encode(s.encode()).decode()
for w in wins:
print('W\t%s\t%s\t%s' % (w["index"], b64(w["name"]), b64(w["layout"])))
for pan in w["panes"]:
print('P\t%s\t%s\t%s\t%s' % (w["index"], pan["index"],
b64(pan["dir"]), b64(pan["cmd"])))
PY
}
# restore [session] [--no-attach] [--dry-run]
restore() {
local session="${1:-$DEFAULT_SESSION}"
shift || true
local keep=0 noattach=0 dryrun=0 a
for a in "$@"; do
case "$a" in
--keep) keep=1 ;;
--no-attach) noattach=1 ;;
--dry-run) dryrun=1 ;;
*) die "unknown restore option: $a" ;;
esac
done
local dest="$TMUX_RES_DIR/$session.json"
[ -f "$dest" ] || die "no snapshot at $dest (run snapshot first)"
# Restore wrapper: run tmux, or print the command under --dry-run.
tmuxr() {
if [ "$dryrun" = 1 ]; then printf ' tmux %s\n' "$*"; else tmux "$@"; fi
}
local target_exists
target_exists=$(tmux list-sessions -F '#{session_name}' 2>/dev/null | grep -Fxq "$session" && echo 1 || echo 0)
if [ "$target_exists" = 1 ]; then
if [ "$keep" = 1 ]; then
die "session '$session' already exists; rebuild without --keep or save a copy"
fi
if [ "$dryrun" = 1 ]; then printf ' # kill existing session %s\n' "$session"
else tmux kill-session --force "$session"; fi
fi
# Parse the plan into flat, window-major lists.
local -a WNAME=() WLAY=() WSTART=() PD=() PC=()
local tag idx nameb layoutb dirb cmdb
while IFS=$'\t' read -r tag idx nameb layoutb dirb cmdb; do
case "$tag" in
W) WNAME+=("$(printf '%s' "$nameb" | b64d)"); WLAY+=("$(printf '%s' "$layoutb" | b64d)"); WSTART+=("${#PD[@]}") ;;
P) PD+=("$(printf '%s' "$dirb" | b64d)"); PC+=("$(printf '%s' "$cmdb" | b64d)") ;;
esac
done < <(emit_plan "$dest" "")
local n=${#WNAME[@]} w first last npanes i curwin wname wlayout dir cmd
for ((w = 0; w < n; w++)); do
first=${WSTART[w]}
if (( w + 1 < n )); then last=${WSTART[w + 1]}; else last=${#PD[@]}; fi
npanes=$(( last - first ))
wname="${WNAME[w]}"; wlayout="${WLAY[w]}"
if [ "$dryrun" = 1 ]; then
if (( w == 0 )); then
tmuxr new-session -d -s "$session" -n "$wname" -c "${PD[first]}"
else
tmuxr new-window -d -t "$session" -n "$wname" -c "${PD[first]}"
fi
curwin="$session:$wname"
elif (( w == 0 )); then
curwin=$(tmux new-session -d -P -F '#{window_id}' \
-s "$session" -n "$wname" -c "${PD[first]}")
else
curwin=$(tmux new-window -d -P -F '#{window_id}' \
-t "$session" -n "$wname" -c "${PD[first]}")
fi
# Create the remaining panes in recorded order (index == creation order).
for ((i = first + 1; i < last; i++)); do
tmuxr split-window -t "$curwin" -c "${PD[i]}"
done
# Restore the exact geometry for this window.
tmuxr select-layout -t "$curwin" "$wlayout"
# Launch each pane's recorded command, matched to panes by ascending
# pane index (so this is correct regardless of base-index).
local -a ordered=()
mapfile -t ordered < <(tmux list-panes -t "$curwin" \
-F $'#{pane_id}\t#{pane_target}' | sort -n -k1,1)
local k=0 tgt
for tgt in "${ordered[@]}"; do
cmd="${PC[first + k]}"
if [ -n "$cmd" ]; then tmuxr send-keys -t "$tgt" "$cmd" Enter; fi
k=$((k + 1))
done
done
if [ "$noattach" = 1 ]; then
return 0
elif [ -t 0 ]; then
tmuxr attach -t "$session"
elif [ "$dryrun" = 0 ]; then
echo "session '$session' rebuilt; attach with: tmux attach -t $session"
fi
}
usage() {
cat <<'USAGE'
tmux-resurrect.sh — snapshot & restore a tmux session on Dankclaw.
Usage:
tmux-resurrect.sh snapshot [session] [dest.json] record the live session
tmux-resurrect.sh restore [session] [options] rebuild the session
options:
--keep skip if the session already exists (default: rebuild)
--no-attach rebuild without attaching to this terminal
--dry-run print the commands instead of running them
USAGE
}
main() {
local cmd="${1:-snapshot}"
shift || true
case "$cmd" in
snapshot) snapshot "$@" ;;
restore) restore "$@" ;;
-h|--help|help) usage ;;
*) die "unknown command: $cmd (use snapshot|restore)" ;;
esac
}
main "$@"
+83 -3
View File
@@ -21,13 +21,93 @@ valkey:
url: "valkey://searxng-valkey:6379/0"
outgoing:
request_timeout: 3.0
max_request_timeout: 10.0
request_timeout: 5.0
max_request_timeout: 15.0
pool_connections: 100
pool_maxsize: 20
enable_http2: true
keepalive_expiry: 5.0
retries: 3
retries: 1
# Keep default aggregation anonymous and bounded. Disabled engines remain
# available for explicit selection/on-demand use; inactive engines are removed.
# Provider-specific overrides below also reduce pagination fan-out.
engines:
- name: bing
disabled: false
weight: 2
paging: false
timeout: 5.0
- name: bing images
paging: false
timeout: 5.0
- name: bing news
paging: false
timeout: 5.0
- name: bing videos
paging: false
timeout: 5.0
- name: duckduckgo
disabled: false
weight: 2
paging: false
timeout: 5.0
- name: duckduckgo images
paging: false
timeout: 5.0
- name: duckduckgo news
paging: false
timeout: 5.0
- name: duckduckgo videos
paging: false
timeout: 5.0
- name: wikipedia
weight: 1
timeout: 5.0
- name: google cse
disabled: true
paging: false
timeout: 5.0
- name: google cse images
disabled: true
paging: false
timeout: 5.0
- name: google news
disabled: true
paging: false
timeout: 5.0
- name: google scholar
disabled: true
paging: false
timeout: 5.0
- name: brave
disabled: true
paging: false
timeout: 5.0
- name: brave.images
disabled: true
paging: false
timeout: 5.0
- name: brave.news
disabled: true
paging: false
timeout: 5.0
- name: brave.videos
disabled: true
paging: false
timeout: 5.0
- name: startpage
disabled: true
paging: false
timeout: 5.0
- name: startpage images
disabled: true
paging: false
timeout: 5.0
- name: startpage news
disabled: true
paging: false
timeout: 5.0
plugins:
searx.plugins.unit_converter.SXNGPlugin:
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bats
# Seam: the changed-test wrapper selects only the tests associated with
# staged paths and refuses to silently run the entire suite.
load helpers
@test "changed-test wrapper is executable and uses staged diff by default" {
[ -x "$REPO_ROOT/scripts/test-changed.sh" ]
[ -x "$REPO_ROOT/.githooks/pre-commit" ]
run grep -F 'scripts/test-changed.sh' "$REPO_ROOT/.githooks/pre-commit"
[ "$status" -eq 0 ]
run grep -F 'git diff --cached' "$REPO_ROOT/scripts/test-changed.sh"
[ "$status" -eq 0 ]
}
@test "changed-test wrapper maps a compose change to compose and environment tests" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankserver/compose.yaml
[ "$status" -eq 0 ]
[[ "$output" == *"tests/compose_validity.bats"* ]]
[[ "$output" == *"tests/env_consistency.bats"* ]]
[[ "$output" != *"tests/homepage_pages.bats"* ]]
}
@test "changed-test wrapper maps a service script to its focused config tests" {
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/serve-frank-small.sh
[ "$status" -eq 0 ]
[[ "$output" == *"tests/scripts_syntax.bats"* ]]
[[ "$output" == *"tests/frank_small_config.bats"* ]]
[[ "$output" != *"tests/frank_large_config.bats"* ]]
}
+31 -6
View File
@@ -11,14 +11,15 @@ UNIT_FILE="$REPO_ROOT/scripts/frank-default-large.service"
TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
@test "frank-default-large serves the kingjones AGENT model" {
grep -q '^MODEL_FILE=Qwen3\.8-27B-Q8_0_ROCMFPX_AGENT\.gguf$' "$ENV_FILE"
grep -q '^MODEL_FILE=Qwen3\.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT\.gguf$' "$ENV_FILE"
grep -q '^MTP_DRAFT_FILE=' "$ENV_FILE"
grep -q '^MMPROJ_FILE=mmproj-Qwen3\.8-27B-Q8_0\.gguf$' "$ENV_FILE"
! grep -q -- '--kairic-edge' "$SERVE_FILE"
}
@test "frank-default-large uses the ROCmFPX runtime" {
grep -q 'ROCmFPX/.*/bin/llama-server' "$ENV_FILE"
grep -q 'build-kairic/bin/llama-server' "$ENV_FILE"
grep -q 'build-promptforge/bin/llama-server' "$ENV_FILE"
}
@test "frank-default-large runs standalone MTP" {
@@ -38,6 +39,11 @@ TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
grep -q ': "${CTX_SIZE:=262144}"' "$SERVE_FILE"
}
@test "frank-default-large uses DeepSeek reasoning output" {
grep -q '^REASONING_FORMAT=deepseek$' "$ENV_FILE"
grep -q -- '--reasoning-format "$REASONING_FORMAT"' "$SERVE_FILE"
}
@test "frank-default-large has reasoning and preserved reasoning/thinking enabled" {
grep -q '^REASONING=on$' "$ENV_FILE"
grep -q 'enable_thinking":true' "$ENV_FILE"
@@ -45,18 +51,37 @@ TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
grep -q 'preserve_reasoning":true' "$ENV_FILE"
}
@test "frank-default-large uses Turbo4 only for the V cache" {
@test "frank-default-large uses f16 for both KV cache sides" {
grep -q '^CACHE_TYPE_K=f16$' "$ENV_FILE"
grep -q '^CACHE_TYPE_V=turbo4$' "$ENV_FILE"
grep -q '^CACHE_TYPE_V=f16$' "$ENV_FILE"
grep -qF -- '-ctk "$CACHE_TYPE_K" -ctv "$CACHE_TYPE_V"' "$SERVE_FILE"
}
@test "frank-default-large exposes a swappable ubatch" {
grep -q '^UBATCH_SIZE=512$' "$ENV_FILE"
grep -q ': "${UBATCH_SIZE:=512}"' "$SERVE_FILE"
grep -q '^UBATCH_SIZE=4096$' "$ENV_FILE"
grep -q ': "${UBATCH_SIZE:=4096}"' "$SERVE_FILE"
grep -q -- '-ub "$UBATCH_SIZE"' "$SERVE_FILE"
}
@test "large ubatch bench defaults to the active model and requested sweep" {
local bench="$REPO_ROOT/scripts/bench-ubatch-rocmi4.sh"
grep -q ': "${UBATCH_MIN:=256}"' "$bench"
grep -q ': "${UBATCH_MAX:=6144}"' "$bench"
grep -q 'UBATCH_VALUES.*256 512 768 1024 1280 1536 2048 3072 4096 5120 6144' "$bench"
grep -q ': "${STOPSVC:=1}"' "$bench"
grep -q ': "${CTX:=32768}"' "$bench"
grep -q ': "${BENCH_BATCH:=32768}"' "$bench"
grep -q 'MODEL_FILE' "$bench"
grep -q 'REASONING_FORMAT' "$bench"
}
@test "large ubatch bench cleans up scratch server on request failure" {
local bench="$REPO_ROOT/scripts/bench-ubatch-rocmi4.sh"
grep -q 'SCRATCH_PID=' "$bench"
grep -q 'kill.*SCRATCH_PID' "$bench"
grep -qF 'eval time =\s' "$bench"
}
@test "frank-default-large uses the tracked Qwen-Sharp template on :7770" {
grep -q '^CHAT_TEMPLATE_FILE=' "$ENV_FILE"
grep -q -- '--jinja' "$SERVE_FILE"
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bats
# Seam: the entropy helper unlocks the known LUKS-on-md device and mounts via
# the host's fstab entry, preserving the existing mount options and directory
# ownership rather than inventing a second mount configuration.
load helpers
@test "entropy helper is executable and has the expected safe shell shape" {
run bash -c "test -x '$REPO_ROOT/scripts/mount-entropy.sh'"
[ "$status" -eq 0 ]
run grep -E '^set -euo pipefail$' "$REPO_ROOT/scripts/mount-entropy.sh"
[ "$status" -eq 0 ]
run grep -F 'udisksctl unlock --block-device /dev/md0' "$REPO_ROOT/scripts/mount-entropy.sh"
[ "$status" -eq 0 ]
run grep -F 'sudo mount "$MOUNTPOINT"' "$REPO_ROOT/scripts/mount-entropy.sh"
[ "$status" -eq 0 ]
}
@test "entropy helper does not embed a secret or alter mount ownership" {
run grep -Eiq 'password|secret|chmod|chown' "$REPO_ROOT/scripts/mount-entropy.sh"
[ "$status" -eq 1 ]
}
+32
View File
@@ -57,6 +57,38 @@ assert not missing, f"pass_ip missing {missing}; got {pass_ip}"
PYEOF
}
@test "SearXNG outgoing policy allows modest VPN latency without retry amplification" {
python3 - "$REPO_ROOT/shared/searxng/settings.yml" <<'PYEOF'
import sys, yaml
with open(sys.argv[1]) as fh:
data = yaml.safe_load(fh)
outgoing = data.get("outgoing", {})
assert outgoing.get("request_timeout") == 5.0, outgoing
assert outgoing.get("max_request_timeout") == 15.0, outgoing
assert outgoing.get("retries") == 1, outgoing
assert outgoing["request_timeout"] < outgoing["max_request_timeout"], outgoing
PYEOF
}
@test "SearXNG default provider policy favors anonymous healthy engines" {
python3 - "$REPO_ROOT/shared/searxng/settings.yml" <<'PYEOF'
import sys, yaml
with open(sys.argv[1]) as fh:
data = yaml.safe_load(fh)
engines = {entry["name"]: entry for entry in data.get("engines", [])}
assert engines["bing"]["disabled"] is False, engines.get("bing")
assert engines["bing"]["weight"] > 1, engines["bing"]
assert engines["bing"]["paging"] is False, engines["bing"]
assert engines["duckduckgo"]["disabled"] is False, engines["duckduckgo"]
assert engines["duckduckgo"]["paging"] is False, engines["duckduckgo"]
for name in ("google cse", "startpage", "brave"):
assert engines[name]["disabled"] is True, engines.get(name)
assert engines[name].get("inactive", False) is False, engines[name]
for name in ("bing", "duckduckgo", "wikipedia"):
assert engines[name]["timeout"] == 5.0, engines[name]
PYEOF
}
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }