Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2693259227 | ||
|
|
e6670f914f | ||
|
|
a05584b155 | ||
|
|
e1f05bd3c2 | ||
|
|
96f5ae6fe1 | ||
|
|
4cb67d7b33 | ||
|
|
babc823f11 | ||
|
|
56ee7cca45 | ||
|
|
c072c75fa9 | ||
|
|
119ca2493f | ||
|
|
a8e0b88d99 | ||
|
|
835996a67b | ||
|
|
eae5bc2565 | ||
|
|
c7a72b465e | ||
|
|
d6ea71b9a4 | ||
|
|
8c8c3f7cd0 | ||
|
|
63ff1b6b43 | ||
|
|
7a0feb39aa | ||
|
|
6ee195dd8f | ||
|
|
ddfc64013f | ||
|
|
86d5f38142 | ||
|
|
205b11aae4 | ||
|
|
6c9e71efd2 |
No files matched your search
Executable
+6
@@ -0,0 +1,6 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Repository pre-commit hook: run only tests selected by staged paths.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||||
|
exec "$REPO_ROOT/scripts/test-changed.sh"
|
||||||
@@ -27,7 +27,7 @@ host.
|
|||||||
| Target | Runtime | Service | Model | Port |
|
| Target | Runtime | Service | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
|
| Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
|
||||||
| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT Q8_0_ROCMFPX (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
|
| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8 Uncensored Q6_0_ROCmFPX_AGENT (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
|
||||||
|
|
||||||
History: the 2026-08-17 redo retired the pipeline-parallel split and the
|
History: the 2026-08-17 redo retired the pipeline-parallel split and the
|
||||||
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
|
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
|
||||||
@@ -35,8 +35,9 @@ Dankclaw. On 2026-08-21 a DFlash2 Vulkan deployment briefly replaced CIRU,
|
|||||||
then was reverted the same day for speed. On 2026-08-22 the service briefly
|
then was reverted the same day for speed. On 2026-08-22 the service briefly
|
||||||
swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day
|
swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day
|
||||||
(operator decision). On 2026-08-25 the service switched to the kingjones
|
(operator decision). On 2026-08-25 the service switched to the kingjones
|
||||||
Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model (standalone MTP draft, 262144 context,
|
Qwen3.8-27B-Uncensored-Q6_0-ROCmFPX_AGENT model (standalone MTP draft,
|
||||||
reasoning on); the previous Kairic Edge model was retired the same day. Runbooks:
|
262144 context, reasoning on); the previous Q8_0 AGENT model was retired after
|
||||||
|
its `set_rows` crash. Runbooks:
|
||||||
[docs/LLM/index.md](docs/LLM/index.md).
|
[docs/LLM/index.md](docs/LLM/index.md).
|
||||||
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
|
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
|
||||||
|
|
||||||
@@ -182,19 +183,22 @@ When you rename a service or move a model to a new host, update
|
|||||||
|
|
||||||
## Runtime & Troubleshooting
|
## Runtime & Troubleshooting
|
||||||
|
|
||||||
### kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT llama.cpp (Dankclaw, ACTIVE)
|
### kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT llama.cpp (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
The large service serves `kingjones/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT`
|
The large service serves `kingjones/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP`
|
||||||
(public) from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/`
|
(public) from `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` with a
|
||||||
with a charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
|
charlie12345/ROCmFPX `llama-server` (build `build-promptforge`). It runs a
|
||||||
standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full
|
standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full
|
||||||
262144 context / 1 slot, and reasoning on (Sharp template, budget -1,
|
262144 context / 1 slot, and reasoning on (Sharp template, budget -1,
|
||||||
`--reasoning-format none`; chat kwargs keep `preserve_thinking` +
|
`--reasoning-format none`; chat kwargs keep `preserve_thinking` +
|
||||||
`preserve_reasoning`). The payload (~28 GB main + 1.7 GB draft + 0.9 GB
|
`preserve_reasoning`). The Q6_0 main model is 24.86 GB. Env
|
||||||
mmproj, ~30 GB total) plus the 262k KV cache fit in the lifted 114 GiB TTM/GTT
|
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`, and
|
||||||
budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
|
runbook `docs/LLM/kingjones-q8agent-rocmfpx.md` define the deployment.
|
||||||
`scripts/serve-frank-large.sh`, runbook
|
|
||||||
[docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
|
**Serving status (2026-09-01):** restored the previously deployed Q6_0 AGENT
|
||||||
|
model. The Q8_0 AGENT experiment is documented in
|
||||||
|
`docs/LLM/kingjones-q8agent-rocmfpx.md`; it was retired because it crashed in
|
||||||
|
`ggml_compute_forward_set_rows` and did not provide a working service.
|
||||||
|
|
||||||
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
|
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Hermes configuration backup / non-secret baseline.
|
||||||
|
#
|
||||||
|
# Source on Dankclaw: /home/dank/hermes-data/config.yaml
|
||||||
|
# This file is a sanitized Git backup, not a complete replacement for the
|
||||||
|
# live Hermes state. Do not add dashboard password hashes, phone numbers,
|
||||||
|
# OAuth credentials, API keys, sessions, memories, or generated state here.
|
||||||
|
# The live directory is bind-mounted into the containers as /data.
|
||||||
|
|
||||||
|
model:
|
||||||
|
provider: custom
|
||||||
|
base_url: http://192.168.0.143:52625/v1
|
||||||
|
api_key: not-needed
|
||||||
|
default: frank-default-large
|
||||||
|
|
||||||
|
providers:
|
||||||
|
custom:
|
||||||
|
stale_timeout_seconds: 1800
|
||||||
|
request_timeout_seconds: 1800
|
||||||
|
dankserver-small:
|
||||||
|
api: http://192.168.0.235:7770/v1
|
||||||
|
discover_models: false
|
||||||
|
models:
|
||||||
|
frank-default-small:
|
||||||
|
context_length: 163840
|
||||||
|
|
||||||
|
# Auxiliary routing from the live configuration. These use the existing
|
||||||
|
# custom endpoint shape and are retained here as a settings backup.
|
||||||
|
auxiliary:
|
||||||
|
vision: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
web_extract: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
compression: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
skills_hub: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
approval: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
mcp: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
title_generation: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
memory_query_rewrite: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
tts_audio_tags: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
triage_specifier: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
kanban_decomposer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
profile_describer: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
curator: {provider: custom, model: frank-default-small, base_url: http://192.168.0.235:7770/v1}
|
||||||
|
|
||||||
|
dashboard:
|
||||||
|
# basic_auth intentionally omitted; its password_hash remains live-only.
|
||||||
|
font: atkinson-hyperlegible
|
||||||
|
skills:
|
||||||
|
disabled: []
|
||||||
|
onboarding:
|
||||||
|
seen:
|
||||||
|
busy_input_prompt: true
|
||||||
|
_config_version: 34
|
||||||
|
platforms:
|
||||||
|
signal:
|
||||||
|
enabled: true
|
||||||
|
# extra.http_url and account intentionally omitted from the backup.
|
||||||
@@ -13,6 +13,28 @@ This repository tests its configuration at defined seams. The tests run with `ba
|
|||||||
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
||||||
| Deployment flow | `tests/deployment_flow.bats` | deploy.sh exists, targets known, SSH reachable (skips offline) |
|
| Deployment flow | `tests/deployment_flow.bats` | deploy.sh exists, targets known, SSH reachable (skips offline) |
|
||||||
|
|
||||||
|
## Focused pre-commit testing
|
||||||
|
|
||||||
|
For normal commits, use the focused wrapper:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/test-changed.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
It reads staged paths with `git diff --cached` and runs only the mapped Bats
|
||||||
|
files. The tracked `.githooks/pre-commit` hook invokes this wrapper. Enable the
|
||||||
|
repository hook path once per checkout:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git config core.hooksPath .githooks
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `--list` to inspect the selection without running tests, or `--paths FILE
|
||||||
|
...` to preview a mapping. Shared test infrastructure and unmapped paths fail
|
||||||
|
closed rather than silently skipping coverage. Run the full suite with
|
||||||
|
`./tests/run.sh` when changing shared test helpers, the test runner, or when
|
||||||
|
focused coverage is not sufficient.
|
||||||
|
|
||||||
## Red-Green
|
## Red-Green
|
||||||
|
|
||||||
Tests are written first. A failing test means the seam is not satisfied; the config change that makes it pass is the only acceptable fix. No `--no-verify`, no deleting tests without approval.
|
Tests are written first. A failing test means the seam is not satisfied; the config change that makes it pass is the only acceptable fix. No `--no-verify`, no deleting tests without approval.
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ The gateway container runs `hermes gateway run` (the Signal platform). That entr
|
|||||||
|
|
||||||
The dashboard refuses to bind to a non-loopback address without an auth provider. There is no unauthenticated public bind. This repo uses basic auth:
|
The dashboard refuses to bind to a non-loopback address without an auth provider. There is no unauthenticated public bind. This repo uses basic auth:
|
||||||
|
|
||||||
- `dashboard.basic_auth.username` and `dashboard.basic_auth.password_hash` live in the git-ignored `/home/dank/hermes-data/config.yaml`.
|
- `dashboard.basic_auth.username` and `dashboard.basic_auth.password_hash` live in the live `/home/dank/hermes-data/config.yaml`; the tracked backup at `dankclaw/hermes/config.yaml.example` intentionally omits the password hash.
|
||||||
- The stored value is a one-way scrypt hash. The plaintext password is never written to a file or the repo.
|
- The stored value is a one-way scrypt hash. The plaintext password is never written to a file or the repo.
|
||||||
- The hash is generated with `plugins.dashboard_auth.basic.hash_password`.
|
- The hash is generated with `plugins.dashboard_auth.basic.hash_password`.
|
||||||
|
|
||||||
|
|||||||
+11
-2
@@ -4,6 +4,10 @@ Hermes is the self-improving AI agent from Nous Research. It runs in a terminal,
|
|||||||
|
|
||||||
**Host:** Dankclaw — **Hostname:** `hermes.dankclaw.local` — **Docs:** https://hermes-agent.nousresearch.com/docs
|
**Host:** Dankclaw — **Hostname:** `hermes.dankclaw.local` — **Docs:** https://hermes-agent.nousresearch.com/docs
|
||||||
|
|
||||||
|
**Persistent state:** `/home/dank/hermes-data` on Dankclaw, mounted as `/data` in
|
||||||
|
both containers. The tracked, sanitized configuration baseline is
|
||||||
|
[`dankclaw/hermes/config.yaml.example`](../../dankclaw/hermes/config.yaml.example).
|
||||||
|
|
||||||
## Facts (from research, primary source)
|
## Facts (from research, primary source)
|
||||||
|
|
||||||
| Item | Value |
|
| Item | Value |
|
||||||
@@ -17,15 +21,20 @@ Hermes is the self-improving AI agent from Nous Research. It runs in a terminal,
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
- Point Hermes at the local model endpoint (`http://localhost:7770/v1`) or an external provider through its provider config.
|
- Point Hermes at the local model endpoint or an external provider through its provider config.
|
||||||
- The gateway process exposes one port for chat integration; the exact port is set in the Hermes config (`8090` convention in this repo).
|
- The gateway process exposes one port for chat integration; the exact port is set in the Hermes config (`8090` convention in this repo).
|
||||||
- The data volume persists skills, memory, and session history.
|
- The data volume persists skills, memory, and session history.
|
||||||
|
- The configured `dankserver-small` provider reaches `http://192.168.0.235:7770/v1`
|
||||||
|
and exposes the `frank-default-small` alias. It is a named provider, so it can
|
||||||
|
coexist with the existing `custom` provider for the large model.
|
||||||
|
- The live `config.yaml` contains secrets and user state and is not committed;
|
||||||
|
the tracked example is the sanitized backup of its non-secret settings.
|
||||||
|
|
||||||
## Volumes
|
## Volumes
|
||||||
|
|
||||||
| Container path | Host path (convention) |
|
| Container path | Host path (convention) |
|
||||||
|----------------|------------------------|
|
|----------------|------------------------|
|
||||||
| `/data` | `/opt/hermes/data` |
|
| `/data` | `/home/dank/hermes-data` |
|
||||||
|
|
||||||
## Gotchas
|
## Gotchas
|
||||||
|
|
||||||
|
|||||||
@@ -15,6 +15,26 @@ engine for an hour to 15 days and silently drops it from results. That is the
|
|||||||
This page is the fixbook. Read [[SearXNG]] for the stack; the sub-pages below
|
This page is the fixbook. Read [[SearXNG]] for the stack; the sub-pages below
|
||||||
are the detail if you need it.
|
are the detail if you need it.
|
||||||
|
|
||||||
|
## Provider policy
|
||||||
|
|
||||||
|
The default aggregation should maximize **independent, anonymous coverage**
|
||||||
|
without turning one user query into a request storm. Disabled engines remain
|
||||||
|
installed and selectable on demand; `inactive` is reserved for engines that
|
||||||
|
should disappear completely.
|
||||||
|
|
||||||
|
| Category | Default baseline | On-demand / deferred |
|
||||||
|
|----------|------------------|----------------------|
|
||||||
|
| General/web | Bing, DuckDuckGo, Wikipedia | Google CSE, Brave, Startpage, Qwant, Mojeek, Wiby, MWMBL after retesting |
|
||||||
|
| Technical | GitHub, MDN, Stack Overflow, Ask Ubuntu, Superuser, Docker Hub, PyPI | Other technical indexes as needed |
|
||||||
|
| Science | arXiv, PubMed, Semantic Scholar, OpenAlex-compatible sources | Google Scholar until independently tested |
|
||||||
|
| News | Reuters, Bing News, DuckDuckGo News, Wikinews | Google News, Brave News, Startpage News |
|
||||||
|
| Images | Openverse, Wikimedia Commons, Bing Images, Unsplash | Google/Brave/Startpage image engines |
|
||||||
|
| Videos | YouTube, Vimeo, Dailymotion, Bing Videos | Brave video engines |
|
||||||
|
|
||||||
|
This policy uses no API credentials and does not identify the operator to a
|
||||||
|
search provider. It is deliberately conservative: a bundled engine is not
|
||||||
|
necessarily healthy, anonymous, or suitable for every default query.
|
||||||
|
|
||||||
## Root cause (one line)
|
## Root cause (one line)
|
||||||
|
|
||||||
All SearXNG engine traffic shares **one ProtonVPN exit IP** with no rotation, so
|
All SearXNG engine traffic shares **one ProtonVPN exit IP** with no rotation, so
|
||||||
@@ -51,6 +71,12 @@ Most "Google is broken on a VPN host" cases are a **mix of both**.
|
|||||||
|
|
||||||
## Fix A — Engine config (settings.yml `engines:`)
|
## Fix A — Engine config (settings.yml `engines:`)
|
||||||
|
|
||||||
|
The current tracked policy enables Bing and DuckDuckGo for default general
|
||||||
|
search, gives each weight 2, disables pagination, and allows 5 seconds for the
|
||||||
|
VPN path. CAPTCHA/429-prone Google, Brave, and Startpage families are retained
|
||||||
|
but disabled from default aggregation. This limits fan-out while preserving
|
||||||
|
explicit/on-demand selection.
|
||||||
|
|
||||||
Important: SearXNG's engines are configured in the **`engines:` list inside
|
Important: SearXNG's engines are configured in the **`engines:` list inside
|
||||||
`settings.yml`**, not a separate `engines.yml` file. With
|
`settings.yml`**, not a separate `engines.yml` file. With
|
||||||
`use_default_settings: true` (our setting) your `engines:` list is *merged* over
|
`use_default_settings: true` (our setting) your `engines:` list is *merged* over
|
||||||
@@ -137,12 +163,12 @@ Tune the `outgoing` block and the suspend model to survive load:
|
|||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
outgoing:
|
outgoing:
|
||||||
request_timeout: 6.0 # raise from 2.0 so slow engines aren't dropped as unresponsive
|
request_timeout: 4.0 # current Dankclaw value; modest VPN-latency allowance
|
||||||
max_request_timeout: 15.0
|
max_request_timeout: 12.0
|
||||||
pool_connections: 200 # bigger pool = more concurrent connections per engine host
|
pool_connections: 100
|
||||||
pool_maxsize: 40 # keep-alive connections per host
|
pool_maxsize: 20
|
||||||
enable_http2: true # multiplexing = fewer distinct "new connection" events per engine
|
enable_http2: true
|
||||||
retries: 3 # on 429/5xx, retry against a different proxy + source IP
|
retries: 1 # avoid amplifying 429/CAPTCHA responses from one exit IP
|
||||||
useragent_suffix: "ops@yourhomelab.example" # so a per-engine blocker can reach you
|
useragent_suffix: "ops@yourhomelab.example" # so a per-engine blocker can reach you
|
||||||
|
|
||||||
search:
|
search:
|
||||||
@@ -159,8 +185,12 @@ Notes:
|
|||||||
that is a historical-searx concept. The functional equivalent is the suspend
|
that is a historical-searx concept. The functional equivalent is the suspend
|
||||||
schedule above (widen to back off before blocking, narrow to recover faster).
|
schedule above (widen to back off before blocking, narrow to recover faster).
|
||||||
- `request_timeout` interacts with the suspend model: too tight → engines look
|
- `request_timeout` interacts with the suspend model: too tight → engines look
|
||||||
"unresponsive"; too loose → slow pages. Aim per-engine `timeout` at
|
"unresponsive"; too loose → slow pages. Dankclaw uses 4 seconds globally and
|
||||||
*proxy-add-on time + engine latency*, capped by `max_request_timeout`.
|
12 seconds as the cap; raise it only when measurements show real upstream
|
||||||
|
latency rather than 429/CAPTCHA responses.
|
||||||
|
- Keep retries low: retrying a 429/CAPTCHA through the same VPN exit can make the
|
||||||
|
provider's rate limit worse. A retry count of 1 is a bounded compromise for a
|
||||||
|
transient network failure.
|
||||||
- `safe_search: 0` when diagnosing — a too-strict global safe search can make an
|
- `safe_search: 0` when diagnosing — a too-strict global safe search can make an
|
||||||
engine return zero rows and *look* blocked.
|
engine return zero rows and *look* blocked.
|
||||||
|
|
||||||
@@ -226,6 +256,39 @@ gateway.
|
|||||||
> containers/systemd/<unit>.container` `Environment` to inject names into the
|
> containers/systemd/<unit>.container` `Environment` to inject names into the
|
||||||
> tunnel's `/etc/hosts`.
|
> tunnel's `/etc/hosts`.
|
||||||
|
|
||||||
|
## Observed Dankclaw baseline (2026-08-25)
|
||||||
|
|
||||||
|
The read-only audit found:
|
||||||
|
|
||||||
|
- SearXNG image: `ghcr.io/searxng/searxng:latest`; embedded version `1.0.0`.
|
||||||
|
- Gluetun uses `VPN_SERVICE_PROVIDER=custom` with a static WireGuard profile;
|
||||||
|
no `SERVER_COUNTRIES`, `SERVER_CITIES`, `SERVER_REGIONS`, or
|
||||||
|
`SERVER_HOSTNAMES` filters are configured.
|
||||||
|
- The observed exit was `149.22.88.19` (Seattle, Washington, US).
|
||||||
|
- Bing and DuckDuckGo returned results. Mojeek returned no results without an
|
||||||
|
upstream error.
|
||||||
|
- `google cse` and Brave were suspended for too many requests; Startpage was
|
||||||
|
suspended for CAPTCHA; Qwant returned CAPTCHA. SearXNG logs also recorded
|
||||||
|
Google's "unusual traffic from your network" response.
|
||||||
|
- `/healthz` returned `200 OK`; the SearXNG service and Gluetun service were
|
||||||
|
healthy. This isolates the primary failures to upstream provider handling,
|
||||||
|
not application availability.
|
||||||
|
|
||||||
|
This deployment returns the legacy array form for `unresponsive_engines`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"unresponsive_engines": [
|
||||||
|
["brave", "Suspended: too many requests"],
|
||||||
|
["startpage", "Suspended: CAPTCHA"]
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not assume the newer object form (`engine`, `error_type`, `suspended`)
|
||||||
|
until the live response has been checked. The direct published port is the
|
||||||
|
useful diagnostic path because it exercises the ProtonVPN exit. The Caddy
|
||||||
|
hostname resolved to the WAN address during the audit and timed out from the
|
||||||
|
audit host, so it was not used as the baseline path.
|
||||||
|
|
||||||
## Fix D — Verify an engine is unblocked
|
## Fix D — Verify an engine is unblocked
|
||||||
|
|
||||||
Three layers, cheapest first. All queries should hit the path production uses
|
Three layers, cheapest first. All queries should hit the path production uses
|
||||||
@@ -242,11 +305,11 @@ curl -s "$BASE/search?q=test&engines=google&format=json" \
|
|||||||
|
|
||||||
Inspect:
|
Inspect:
|
||||||
- `content[].engine` — which engine produced each result.
|
- `content[].engine` — which engine produced each result.
|
||||||
- `unresponsive_engines[]` — `{engine, error_type, suspended}`. `error_type`
|
- `unresponsive_engines[]` — inspect the live shape first. On Dankclaw it is
|
||||||
tells you *why* (`SearxEngineTooManyRequests`, `SearxEngineCaptcha`,
|
`[engine, message]`, for example `["google cse", "Suspended: too many requests"]`.
|
||||||
`timeout`, `SearxEngineAccessDenied`); `suspended: true` = auto-banned right
|
Newer builds may return objects containing `engine`, `error_type`, and
|
||||||
now.
|
`suspended`.
|
||||||
- `timings[].engine` — per-engine latency.
|
- `timings[].engine` — per-engine latency when the running build emits timings.
|
||||||
|
|
||||||
Full status check (bash + jq):
|
Full status check (bash + jq):
|
||||||
|
|
||||||
@@ -285,16 +348,23 @@ misleading).
|
|||||||
|
|
||||||
## Recommended path for Dankclaw (ranked)
|
## Recommended path for Dankclaw (ranked)
|
||||||
|
|
||||||
1. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
|
1. **Measure individual engines** — use `/json` + `?engines=<name>` and several
|
||||||
|
query classes before enabling a bundled engine in default aggregation.
|
||||||
|
2. **Keep default fan-out small** — prefer independent anonymous providers and
|
||||||
|
disable pagination for rate-sensitive engines.
|
||||||
|
3. **Promote by evidence** — raise `weight` only for engines that return useful
|
||||||
|
results consistently; weight is not a health check.
|
||||||
|
4. **Avoid credentials** — do not add API-backed engines unless the provider is
|
||||||
|
genuinely free, anonymous, and compatible with our privacy requirements.
|
||||||
|
5. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
|
||||||
exactly which engines are down and why. Confirm the running SearXNG version
|
exactly which engines are down and why. Confirm the running SearXNG version
|
||||||
(check whether `parsers`/`rate_limit` exist in the pinned build).
|
(check whether `parsers`/`rate_limit` exist in the pinned build).
|
||||||
2. **Tighten Fix B** — raise `outgoing.request_timeout`, add `retry_on_http_error:
|
6. **Use conservative request tuning** — Dankclaw now allows 5 seconds per
|
||||||
[403, 429]` on the Google engines, widen `suspended_times`. Cheapest, no new
|
upstream request, caps it at 15 seconds, and retries once. This addresses
|
||||||
infra.
|
modest VPN latency without multiplying 429/CAPTCHA traffic. Adjust only
|
||||||
3. **Duplicate the Google engines** (Fix A) so a CAPTCHA ban on one leaves the
|
after measuring the resulting error and latency profile.
|
||||||
other returning results.
|
7. **If Google still re-blocks** — Fix C Option 1: point the Google engines at
|
||||||
4. **If Google still re-blocks** — Fix C Option 1: point the Google engines at a
|
a dedicated-IP proxy (Option 2 dedicated second Gluetun if reputation
|
||||||
dedicated-IP proxy (Option 2 dedicated second Gluetun if reputation
|
|
||||||
isolation is needed). Avoid the same-netns-proxy trap.
|
isolation is needed). Avoid the same-netns-proxy trap.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -61,9 +61,29 @@ valkey:
|
|||||||
url: "valkey://searxng-valkey:6379/0"
|
url: "valkey://searxng-valkey:6379/0"
|
||||||
|
|
||||||
outgoing:
|
outgoing:
|
||||||
request_timeout: 3.0
|
request_timeout: 5.0
|
||||||
max_request_timeout: 10.0
|
max_request_timeout: 15.0
|
||||||
retries: 3
|
retries: 1 # avoid amplifying upstream 429/CAPTCHA responses
|
||||||
|
|
||||||
|
# Anonymous, bounded default aggregation. Disabled engines remain configured
|
||||||
|
# for explicit/on-demand selection; see SearXNG Providers for the policy.
|
||||||
|
engines:
|
||||||
|
- name: bing
|
||||||
|
disabled: false
|
||||||
|
weight: 2
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: duckduckgo
|
||||||
|
disabled: false
|
||||||
|
weight: 2
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: google cse
|
||||||
|
disabled: true
|
||||||
|
- name: brave
|
||||||
|
disabled: true
|
||||||
|
- name: startpage
|
||||||
|
disabled: true
|
||||||
```
|
```
|
||||||
|
|
||||||
## Volumes
|
## Volumes
|
||||||
@@ -96,10 +116,16 @@ Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local s
|
|||||||
|
|
||||||
## Unblocking Google & Search Providers
|
## Unblocking Google & Search Providers
|
||||||
|
|
||||||
SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP. The
|
The current request policy allows a modest amount of VPN latency (`4.0s` per
|
||||||
rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429,
|
request, `12.0s` maximum) while limiting retries to one. This can recover slow
|
||||||
and SearXNG **suspends** the offending engine for hours to days, which is what
|
providers, but it cannot fix an upstream CAPTCHA or 429 caused by the shared
|
||||||
shows up as "Google went unavailable". This is upstream, not local.
|
VPN exit IP. SearXNG queries all ~50 upstream engines from one Gluetun VPN exit
|
||||||
|
IP. The rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429.
|
||||||
|
The default provider policy therefore favors anonymous Bing and DuckDuckGo,
|
||||||
|
disables observed CAPTCHA/429 providers from normal aggregation, and disables
|
||||||
|
pagination to reduce upstream request volume. SearXNG **suspends** offending
|
||||||
|
engines for hours to days, which is what shows up as "Google went unavailable".
|
||||||
|
This is upstream, not local.
|
||||||
|
|
||||||
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
|
Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
|
||||||
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
|
`retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# Entropy Volume
|
||||||
|
|
||||||
|
The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at
|
||||||
|
`/mnt/entropy`. The command-line helper reuses the host's desktop-integrated
|
||||||
|
`udisks2`, `crypttab`, and `fstab` configuration; it does not store or handle
|
||||||
|
the passphrase itself.
|
||||||
|
|
||||||
|
## Command-line mount
|
||||||
|
|
||||||
|
```bash
|
||||||
|
/home/dank/serverAppConfigs/scripts/mount-entropy.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Run it as the normal desktop user (`dank`), not from a root shell. `udisksctl`
|
||||||
|
will securely prompt for the LUKS passphrase. `sudo mount /mnt/entropy` then
|
||||||
|
uses the existing `/etc/fstab` entry, preserving its options:
|
||||||
|
|
||||||
|
```text
|
||||||
|
nosuid,nodev
|
||||||
|
```
|
||||||
|
|
||||||
|
The existing mountpoint is owned by `dank:dankserver` with mode `0755`; the
|
||||||
|
helper deliberately does not run `chmod` or `chown`, so it cannot silently
|
||||||
|
change those permissions.
|
||||||
|
|
||||||
|
## Device path
|
||||||
|
|
||||||
|
The current storage chain is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/dev/md0 -> LUKS UUID cd86c728-459e-4f3a-bcc2-41ecf84685ec
|
||||||
|
-> ext4 label entropy
|
||||||
|
-> /mnt/entropy
|
||||||
|
```
|
||||||
|
|
||||||
|
The helper checks that `/dev/md0` exists and exits successfully if the volume
|
||||||
|
is already mounted. It is intentionally specific to this host and volume:
|
||||||
|
that avoids accidentally unlocking or mounting a different disk.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[Storage Policy]]
|
||||||
|
- [[Dankserver Stack]]
|
||||||
+10
-10
@@ -18,7 +18,7 @@ repository to a running service.
|
|||||||
| Target | Host | Runtime | Service | Port |
|
| Target | Host | Runtime | Service | Port |
|
||||||
|--------|------|---------|---------|------|
|
|--------|------|---------|---------|------|
|
||||||
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
|
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
|
||||||
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT, standalone MTP, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
||||||
|
|
||||||
## Workflow
|
## Workflow
|
||||||
|
|
||||||
@@ -138,21 +138,21 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
|
|||||||
### Dankclaw — frank-default-large
|
### Dankclaw — frank-default-large
|
||||||
|
|
||||||
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
|
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
|
||||||
[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
|
[kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
|
||||||
under the alias `frank-default-large` on port 7770. The main model is
|
under the alias `frank-default-large` on port 7770. The main model is
|
||||||
`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
|
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
|
||||||
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
|
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the Q8_0 vision projector is loaded alongside
|
||||||
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
|
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
|
||||||
service keeps the existing 65536 context / 1-slot profile. The unit file is
|
service uses the 262144 context / 1-slot profile. The unit file is
|
||||||
`scripts/frank-default-large.service`; it launches through
|
`scripts/frank-default-large.service`; it launches through
|
||||||
`scripts/serve-frank-large.sh`, which reads tunables from
|
`scripts/serve-frank-large.sh`, which reads tunables from
|
||||||
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
|
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
|
||||||
|
|
||||||
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
|
> The Q8_0 AGENT experiment was retired on 2026-09-01 after repeated
|
||||||
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
|
> `set_rows` crashes and device-local allocation failures. The former caf-eemin
|
||||||
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
|
> ROCmI4 embedded-MTP model and the v3 PromptForge payload are historical
|
||||||
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
|
> deployments. The intermediate DFlash2 Vulkan deployment was retired on
|
||||||
> See [[dflash2-large-vulkan]] for the archived runbook.
|
> 2026-08-21. See [[kingjones-q8agent-rocmfpx]] and [[dflash2-large-vulkan]].
|
||||||
|
|
||||||
## Validation
|
## Validation
|
||||||
|
|
||||||
|
|||||||
+9
-7
@@ -3,7 +3,7 @@
|
|||||||
Documentation for the LLM inference fleet on two local machines
|
Documentation for the LLM inference fleet on two local machines
|
||||||
(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
|
(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
|
||||||
Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
|
Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
|
||||||
(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
|
(frank-default-large, kingjones Qwen3.8 Uncensored Q6 AGENT). Both
|
||||||
services run the Qwen-Sharp chat template.
|
services run the Qwen-Sharp chat template.
|
||||||
|
|
||||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||||
@@ -26,9 +26,9 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
|
|||||||
|
|
||||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||||
D[gfx1151 iGPU 64 GiB unified memory]
|
D[gfx1151 iGPU 64 GiB unified memory]
|
||||||
E[CIRU ROCmFPX llama.cpp PromptForge v3
|
E[charlie12345/ROCmFPX build-kairic (v81)
|
||||||
frank-default-large, port 7770]
|
frank-default-large, port 7770]
|
||||||
F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
|
F[Qwen3.8 Uncensored Q6 ROCmFPX AGENT
|
||||||
MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
||||||
D --> E --> F
|
D --> E --> F
|
||||||
end
|
end
|
||||||
@@ -48,14 +48,16 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
|||||||
|
|
||||||
- [[deployment]] — deploy and update process for both servers
|
- [[deployment]] — deploy and update process for both servers
|
||||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||||
- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
|
- [kingjones Qwen3.8 Uncensored ROCmFP4 STRIX MTP](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF) — active frank-default-large model on Dankclaw; Q6 AGENT main model with standalone MTP
|
||||||
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
|
- [[kingjones-strix-mtp]] — the active Q6 AGENT runbook: serving profile, standalone MTP, reasoning + preserve_thinking, 114 GiB TTM/GTT
|
||||||
|
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
|
||||||
|
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
|
||||||
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
|
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
|
||||||
on Dankclaw (retired)
|
on Dankclaw (retired)
|
||||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
||||||
Dankserver (frank-default-small, active 2026-08-24)
|
Dankserver (frank-default-small, active 2026-08-24)
|
||||||
- [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
|
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
|
||||||
model on Dankclaw (superseded 2026-08-25 by the ROCmI4 embedded-MTP model)
|
model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)
|
||||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
||||||
Dankserver (frank-default-small, active 2026-08-24)
|
Dankserver (frank-default-small, active 2026-08-24)
|
||||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime
|
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime
|
||||||
|
|||||||
@@ -1,6 +1,9 @@
|
|||||||
# frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
|
# frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
|
||||||
|
|
||||||
**Status:** ACTIVE on Dankclaw (since 2026-08-25).
|
**Status:** Deployed on Dankclaw (since 2026-08-25). Serving is **BLOCKED by
|
||||||
|
the `set_rows` build bug** — the model loads but crashes in
|
||||||
|
`ggml_compute_forward_set_rows` before it can serve. All config goals are met;
|
||||||
|
a build fix is required for the model to run.
|
||||||
|
|
||||||
**Alias:** `frank-default-large`, port **7770**.
|
**Alias:** `frank-default-large`, port **7770**.
|
||||||
|
|
||||||
@@ -26,6 +29,53 @@ fit in the lifted **114 GiB TTM/GTT** budget (see the 114 GiB change in
|
|||||||
| Tunables | `scripts/frank-default-large.env` |
|
| Tunables | `scripts/frank-default-large.env` |
|
||||||
| Serve script | `scripts/serve-frank-large.sh` |
|
| Serve script | `scripts/serve-frank-large.sh` |
|
||||||
|
|
||||||
|
## Serving status: the `set_rows` build bug (blocks serving)
|
||||||
|
|
||||||
|
As of 2026-08-25 the Q8_0 AGENT model will not serve on Dankclaw. It loads,
|
||||||
|
reaches `initializing slots, n_slots = 1`, but never prints `start serving`.
|
||||||
|
Health stays `503` until the process exits. The crash is in
|
||||||
|
`ggml_compute_forward_set_rows` in `libggml-cpu.so.0`, a NULL function-pointer
|
||||||
|
jump (address `0x00000000`).
|
||||||
|
|
||||||
|
The backtrace (gdb). The main thread and one OpenMP worker crash in the same
|
||||||
|
path:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph TD
|
||||||
|
A[llama_context::graph_compute] --> B[ggml_backend_cpu_graph_compute]
|
||||||
|
B --> C[ggml_graph_compute]
|
||||||
|
C --> D[GOMP_parallel]
|
||||||
|
D --> E[ggml_compute_forward_set_rows]
|
||||||
|
E --> F[NULL dispatch -> address 0x0]
|
||||||
|
D --> G[Thread 23 worker: same path]
|
||||||
|
```
|
||||||
|
|
||||||
|
Test matrix. Five ROCmFPX builds and the all-CPU path were tested on a scratch
|
||||||
|
port (`/tmp/g80bench/run.sh`), `gl=45`, `ctx=131072`. All crash in `set_rows`.
|
||||||
|
|
||||||
|
| Build | Binary | Version | Result |
|
||||||
|
| ----- | ------ | ------- | ------ |
|
||||||
|
| build-kairic | `ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server` | v81 (205a3e5f4) | segfault in set_rows |
|
||||||
|
| build-promptforge | `ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` | PromptForge/ActiveFPX lineage | segfault in set_rows |
|
||||||
|
| build-rocmi4-w4a4 | `rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` | v257 (c49ebdb) | segfault in set_rows |
|
||||||
|
| build-strix-rocmfp4 | `GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-server` | v209 (00d5452) | segfault in set_rows |
|
||||||
|
| all-CPU | build-kairic, `-ngl 0` | v81 | segfault in set_rows |
|
||||||
|
|
||||||
|
The crash is not GPU-specific (the all-CPU path fails too), not MTP-specific
|
||||||
|
(`--spec-type none` fails), and not fixed by `LLAMA_SET_ROWS=0`. So it is a
|
||||||
|
build bug: the Q8_0 AGENT GGUF graph needs a `set_rows` dispatch the ROCmFPX
|
||||||
|
backends do not register. The older caf-eemin ROCmI4 and Kairic Edge models
|
||||||
|
served fine on related ROCmFPX builds. The caf-eemin payload was about 14.4
|
||||||
|
GiB; the separate
|
||||||
|
PromptForge/ActiveFPX deployment used a much larger multi-file payload of about
|
||||||
|
36.7 GiB. These are different deployments. The Q8_0 AGENT graph is the trigger
|
||||||
|
for the `set_rows` failure.
|
||||||
|
|
||||||
|
**Fix path:** rebuild charlie12345/ROCmFPX from source with a working
|
||||||
|
`set_rows` for the ROCmFPX backend, or find a build where it is registered.
|
||||||
|
This unblocks the model. See the context budget reality below once `set_rows`
|
||||||
|
is fixed.
|
||||||
|
|
||||||
## Model files
|
## Model files
|
||||||
|
|
||||||
Three files, from the HF repo above:
|
Three files, from the HF repo above:
|
||||||
@@ -93,9 +143,14 @@ Key choices:
|
|||||||
- **f16 K + TurboQuant V** cache. f16 K is the safe long-ctx default; TurboQuant
|
- **f16 K + TurboQuant V** cache. f16 K is the safe long-ctx default; TurboQuant
|
||||||
halves the V cache at 262k while keeping quality (see the ubatch/threads-batch
|
halves the V cache at 262k while keeping quality (see the ubatch/threads-batch
|
||||||
notes in [[qwen3.8-strix-optimizations]]).
|
notes in [[qwen3.8-strix-optimizations]]).
|
||||||
- **Full 262144 context, 1 slot.** The whole context KV cache plus the ~30 GB
|
- **262144 context, 1 slot — the target.** The whole context KV cache plus the
|
||||||
payload (main + draft + mmproj) fit in the 114 GiB budget. Concurrent
|
~30 GB payload (main + draft + mmproj) are meant to fit in the 114 GiB
|
||||||
requests queue; the server never preempts a running generation.
|
budget. Measured reality: `gl=999` (all layers on ROCm0) at 262144 **OOMs**
|
||||||
|
on device-local (`cudaMalloc failed`, the KV cache needs ~10-18 GiB that does
|
||||||
|
not fit once the payload is on the card). `gl=45` (partial offload) fits a
|
||||||
|
131072 context. The env ships `CTX_SIZE=262144`; the proven serving context
|
||||||
|
today is **131072 at `gl=45`** (before the `set_rows` fix). `deploy.sh` waits
|
||||||
|
up to 20 minutes (see `LLM_WAIT`).
|
||||||
- **Reasoning on.** The Sharp template drives thinking (medium effort);
|
- **Reasoning on.** The Sharp template drives thinking (medium effort);
|
||||||
`--reasoning-format none` lets the template project the reasoning content into
|
`--reasoning-format none` lets the template project the reasoning content into
|
||||||
`reasoning_content`. `--reasoning-budget -1` = no budget cap.
|
`reasoning_content`. `--reasoning-budget -1` = no budget cap.
|
||||||
@@ -160,6 +215,11 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
|
|||||||
- **2026-08-25 (today):** switched the large service to the kingjones
|
- **2026-08-25 (today):** switched the large service to the kingjones
|
||||||
Q8_0-AGENT model with 262144 context and reasoning on. Downloaded the three
|
Q8_0-AGENT model with 262144 context and reasoning on. Downloaded the three
|
||||||
payload files from the HF repo, raised the TTM/GTT to 114 GiB for the full
|
payload files from the HF repo, raised the TTM/GTT to 114 GiB for the full
|
||||||
context, and persisted the env/serve script in this repo.
|
context, and persisted the env/serve script in this repo. Measured that the
|
||||||
|
model loads but crashes in `ggml_compute_forward_set_rows` on all five
|
||||||
|
available ROCmFPX builds; serving is BLOCKED pending a rebuild. The config
|
||||||
|
goals (GTT 114 GiB, reasoning on, preserve_thinking/preserve_reasoning on,
|
||||||
|
262144 env context) are in place. Proven serving context today is 131072 at
|
||||||
|
`gl=45`. See the `set_rows` build bug section above.
|
||||||
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
|
- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
|
||||||
retired the same day for the AGENT model). See [[kairic-edge]].
|
retired the same day for the AGENT model). See [[kairic-edge]].
|
||||||
@@ -15,13 +15,12 @@ timestamp: 2026-08-24
|
|||||||
|
|
||||||
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
||||||
|
|
||||||
> **Retired 2026-08-25.** The kingjones ROCmFP4 STRIX MTP payload has been
|
> **Active again 2026-09-01.** The kingjones ROCmFP4 STRIX MTP payload is
|
||||||
> replaced by the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on the same
|
> restored as `frank-default-large` after the Q8_0 AGENT experiment. The Q8_0
|
||||||
> port and iGPU, under a new runtime (charlie12345/ROCmFPX ROCmI4 W4A4). See
|
> model was retired because it crashed before serving. The Q6 AGENT model,
|
||||||
> [[eemin-rocmi4-mtp]]. The old kingjones GGUF + standalone MTP Q4_0 draft are
|
> standalone MTP draft, Q8 projector, and PromptForge runtime are now active.
|
||||||
> retired on this slot. The CIRU ROCmFPX runtime/binary is also retired here.
|
|
||||||
|
|
||||||
> _This page documents the retired model and its runbook for reference._
|
> This page documents the active model and its runbook.
|
||||||
|
|
||||||
`frank-default-large` (port 7770) serves
|
`frank-default-large` (port 7770) serves
|
||||||
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
||||||
@@ -34,7 +33,7 @@ here; the head is loaded with `--model-draft`.
|
|||||||
|
|
||||||
The card ships six text files plus the vision projector. The service uses
|
The card ships six text files plus the vision projector. The service uses
|
||||||
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
||||||
and the Q8 vision projector.
|
and the Q8 vision projector (`mmproj-Qwen3.8-27B-Q8_0.gguf`).
|
||||||
|
|
||||||
| File | Size |
|
| File | Size |
|
||||||
|------|------|
|
|------|------|
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
---
|
||||||
|
type: benchmark plan
|
||||||
|
title: frank-default-large ubatch sweep
|
||||||
|
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
|
||||||
|
tags:
|
||||||
|
- llama.cpp
|
||||||
|
- rocm
|
||||||
|
- benchmark
|
||||||
|
- ubatch
|
||||||
|
timestamp: 2026-09-01
|
||||||
|
---
|
||||||
|
|
||||||
|
# frank-default-large ubatch sweep
|
||||||
|
|
||||||
|
This benchmark sweeps the active `frank-default-large` model on Dankclaw using
|
||||||
|
scratch ports and a 32K-context workload. The production service is stopped
|
||||||
|
before the sweep and restarted by the benchmark's exit handler.
|
||||||
|
|
||||||
|
## Sweep
|
||||||
|
|
||||||
|
The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
|
||||||
|
5120, 6144`. This implements the requested 256-step range through 1536 and
|
||||||
|
1024-step range from 2048 through 6144. Each point runs three 256-token decode
|
||||||
|
requests by default.
|
||||||
|
|
||||||
|
Run on Dankclaw from the repository checkout:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
The script uses the active Q6 AGENT model, standalone MTP draft, a fixed
|
||||||
|
32K batch, and the tracked Sharp
|
||||||
|
template, and `--reasoning-format deepseek`. It reports server timing lines and
|
||||||
|
per-run generation throughput. Run only when the production service can be
|
||||||
|
interrupted; the GPU is shared with live workloads.
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
Record results below after the sweep, including failures or OOM points.
|
||||||
|
|
||||||
|
| ubatch | prefill | generation | notes |
|
||||||
|
| ---: | ---: | ---: | --- |
|
||||||
|
| 256 | — | — | pending |
|
||||||
|
| 512 | — | — | pending |
|
||||||
|
| 768 | — | — | pending |
|
||||||
|
| 1024 | — | — | pending |
|
||||||
|
| 1280 | — | — | pending |
|
||||||
|
| 1536 | — | — | pending |
|
||||||
|
| 2048 | — | — | pending |
|
||||||
|
| 3072 | — | — | pending |
|
||||||
|
| 4096 | — | — | pending |
|
||||||
|
| 5120 | — | — | pending |
|
||||||
|
| 6144 | — | — | pending |
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [LLM fleet index](index.md)
|
||||||
|
- [Deployment](deployment.md)
|
||||||
|
- [Active kingjones runbook](kingjones-strix-mtp.md)
|
||||||
+8
-3
@@ -30,6 +30,10 @@ covers both workloads).
|
|||||||
- [[Deployment Flow]] — git-based deploy, target map, health checks
|
- [[Deployment Flow]] — git-based deploy, target map, health checks
|
||||||
- [[Testing Policy]] — the test seams and how to run them
|
- [[Testing Policy]] — the test seams and how to run them
|
||||||
|
|
||||||
|
## Dankserver Storage
|
||||||
|
|
||||||
|
- [[Entropy Volume]] — command-line unlock and mount helper
|
||||||
|
|
||||||
## Dankserver Services
|
## Dankserver Services
|
||||||
|
|
||||||
- [[RomM]] — retro game manager
|
- [[RomM]] — retro game manager
|
||||||
@@ -97,9 +101,9 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
|
|
||||||
- [[deployment]] — unified deploy and update process for both servers
|
- [[deployment]] — unified deploy and update process for both servers
|
||||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||||
- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
|
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (restored 2026-09-01)
|
||||||
- [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
|
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
|
||||||
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
|
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
|
||||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
|
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
|
||||||
@@ -112,6 +116,7 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
### Benchmarks and Sizing
|
### Benchmarks and Sizing
|
||||||
|
|
||||||
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
|
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
|
||||||
|
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
|
||||||
- [[frank-default-small-benchmark]] — the small-model benchmark suite
|
- [[frank-default-small-benchmark]] — the small-model benchmark suite
|
||||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
|
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
|
||||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
|
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# bench-ubatch-rocmi4.sh — sweep ubatch size at 8k context on the
|
# bench-ubatch-rocmi4.sh — sweep ubatch size at 8k context on the
|
||||||
# frank-default-large ROCmI4 embedded-MTP stack (dankclaw, gfx1151).
|
# frank-default-large ROCmFPX standalone-MTP stack (dankclaw, gfx1151).
|
||||||
#
|
#
|
||||||
# Boots one llama-server instance per ubatch on a scratch port with a fixed
|
# Boots one llama-server instance per ubatch on a scratch port with a fixed
|
||||||
# -b and -c 8192, runs a standardized ~75%-of-ctx prose workload, and reads
|
# -b and -c 32768, runs a standardized ~75%-of-ctx prose workload, and reads
|
||||||
# authoritative prefill/generation tok/s from the server's own print_timing
|
# authoritative prefill/generation tok/s from the server's own print_timing
|
||||||
# lines. Steps from UBATCH_MIN to UBATCH_MAX inclusive by UBATCH_STEP.
|
# lines. Steps from UBATCH_MIN to UBATCH_MAX inclusive by UBATCH_STEP.
|
||||||
#
|
#
|
||||||
@@ -26,21 +26,23 @@ if [[ -f "$ENV_FILE" ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# --- sweep parameters -------------------------------------------------------
|
# --- sweep parameters -------------------------------------------------------
|
||||||
: "${UBATCH_MIN:=512}"
|
: "${UBATCH_MIN:=256}"
|
||||||
: "${UBATCH_MAX:=4096}"
|
: "${UBATCH_MAX:=6144}"
|
||||||
: "${UBATCH_STEP:=512}"
|
: "${UBATCH_STEP:=256}"
|
||||||
: "${CTX:=8192}" # 8k contexts
|
: "${UBATCH_VALUES:=256 512 768 1024 1280 1536 2048 3072 4096 5120 6144}"
|
||||||
: "${BENCH_BATCH:=$CTX}" # fixed batch = ctx, so the full 8k prompt fits
|
: "${CTX:=32768}" # 32k contexts
|
||||||
|
: "${BENCH_BATCH:=32768}" # batch is fixed to the 32k context size
|
||||||
: "${OUT_TOKENS:=256}"
|
: "${OUT_TOKENS:=256}"
|
||||||
: "${RUNS:=3}" # decode runs per ubatch
|
: "${RUNS:=3}" # decode runs per ubatch
|
||||||
: "${PORT_BASE:=7894}" # scratch ports 7894..
|
: "${PORT_BASE:=7894}" # scratch ports 7894..
|
||||||
: "${STOPSVC:=0}" # stop/restart the live unit for a clean GPU
|
: "${STOPSVC:=1}" # stop/restart the live unit for a clean GPU
|
||||||
|
|
||||||
# --- resolved paths (fall back to serve-script defaults) --------------------
|
# --- resolved paths (fall back to serve-script defaults) --------------------
|
||||||
: "${LLAMA_BIN:=/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server}"
|
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server}"
|
||||||
: "${MODEL_DIR:=/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP}"
|
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP}"
|
||||||
: "${MODEL_FILE:=Qwen3.8-27B-Q4_0_ROCMI4.gguf}"
|
: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
|
||||||
: "${MMPROJ_FILE:=mmproj-BF16.gguf}"
|
: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
|
||||||
|
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-Q8_0.gguf}"
|
||||||
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
: "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
|
||||||
: "${ALIAS:=frank-default-large}"
|
: "${ALIAS:=frank-default-large}"
|
||||||
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
|
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
|
||||||
@@ -68,7 +70,14 @@ if [[ -f "$MMPROJ" ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
STopped=0
|
STopped=0
|
||||||
|
SCRATCH_PID=""
|
||||||
cleanup() {
|
cleanup() {
|
||||||
|
if [[ -n "$SCRATCH_PID" ]] && kill -0 "$SCRATCH_PID" 2>/dev/null; then
|
||||||
|
echo "=== stopping scratch llama-server (pid $SCRATCH_PID) ==="
|
||||||
|
kill "$SCRATCH_PID" 2>/dev/null || true
|
||||||
|
wait "$SCRATCH_PID" 2>/dev/null || true
|
||||||
|
SCRATCH_PID=""
|
||||||
|
fi
|
||||||
if [ "$STopped" = 1 ]; then
|
if [ "$STopped" = 1 ]; then
|
||||||
echo "=== restarting frank-default-large ==="
|
echo "=== restarting frank-default-large ==="
|
||||||
sudo systemctl restart frank-default-large || true
|
sudo systemctl restart frank-default-large || true
|
||||||
@@ -109,12 +118,12 @@ run_one() {
|
|||||||
local log="/tmp/ubatch-${ubatch}-p${port}.log"
|
local log="/tmp/ubatch-${ubatch}-p${port}.log"
|
||||||
: > "$log"
|
: > "$log"
|
||||||
|
|
||||||
echo "=== ubatch=${ubatch} ctx=${CTX} port=${port} runs=${runs} ==="
|
echo "=== ubatch=${ubatch} ctx=${CTX} batch=${BENCH_BATCH} port=${port} runs=${runs} ==="
|
||||||
"$LLAMA_BIN" \
|
"$LLAMA_BIN" \
|
||||||
--model "$MODEL" \
|
--model "$MODEL" \
|
||||||
"${MMPROJ_ARGS[@]}" \
|
"${MMPROJ_ARGS[@]}" \
|
||||||
--chat-template-file "$CHAT_TEMPLATE_FILE" \
|
--chat-template-file "$CHAT_TEMPLATE_FILE" \
|
||||||
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
|
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
|
||||||
--alias bench-ubatch \
|
--alias bench-ubatch \
|
||||||
--host 127.0.0.1 --port "$port" \
|
--host 127.0.0.1 --port "$port" \
|
||||||
-dev ROCm0 -ngl 999 -np 1 \
|
-dev ROCm0 -ngl 999 -np 1 \
|
||||||
@@ -122,25 +131,32 @@ run_one() {
|
|||||||
-c "$CTX" -b "$BENCH_BATCH" -ub "$ubatch" \
|
-c "$CTX" -b "$BENCH_BATCH" -ub "$ubatch" \
|
||||||
-fa on -ctk f16 -ctv f16 -ctxcp 16 \
|
-fa on -ctk f16 -ctv f16 -ctxcp 16 \
|
||||||
--cache-ram 8192 --cache-prompt --metrics --no-mmap \
|
--cache-ram 8192 --cache-prompt --metrics --no-mmap \
|
||||||
--spec-type draft-mtp --spec-mtp-strict-qwen \
|
--spec-type draft-mtp --model-draft "$MTP_DRAFT_FILE" \
|
||||||
--spec-draft-device ROCm0 --spec-draft-ngl all \
|
--spec-draft-device ROCm0 --spec-draft-ngl 999 \
|
||||||
--spec-draft-type-k f16 --spec-draft-type-v f16 \
|
--spec-draft-type-k f16 --spec-draft-type-v f16 \
|
||||||
--spec-draft-n-max "$MTP_N_MAX" --spec-draft-n-min 0 \
|
--spec-draft-n-max "$MTP_N_MAX" --spec-draft-n-min 0 \
|
||||||
--spec-draft-p-min "$MTP_P_MIN" \
|
--spec-draft-p-min "$MTP_P_MIN" \
|
||||||
--spec-draft-backend-sampling \
|
--spec-draft-backend-sampling \
|
||||||
--temp 0.7 --top-p 0.95 \
|
--temp 0.7 --top-p 0.95 \
|
||||||
--reasoning on --reasoning-format deepseek --reasoning-budget -1 \
|
--reasoning "$REASONING" --reasoning-format "$REASONING_FORMAT" --reasoning-budget -1 \
|
||||||
-lv 4 >>"$log" 2>&1 &
|
-lv 4 >>"$log" 2>&1 &
|
||||||
local pid=$!
|
local pid=$!
|
||||||
wait_health "$pid" "$port" || { tail -30 "$log"; return 2; }
|
SCRATCH_PID="$pid"
|
||||||
|
if ! wait_health "$pid" "$port"; then
|
||||||
|
tail -30 "$log"
|
||||||
|
kill "$pid" 2>/dev/null || true
|
||||||
|
wait "$pid" 2>/dev/null || true
|
||||||
|
SCRATCH_PID=""
|
||||||
|
return 2
|
||||||
|
fi
|
||||||
|
|
||||||
# one python pass per run; reuse the same boot across runs
|
# one python pass per run; reuse the same boot across runs
|
||||||
local r
|
local r
|
||||||
for r in $(seq 1 "$runs"); do
|
for r in $(seq 1 "$runs"); do
|
||||||
echo "--- run ${r}/${runs} ---"
|
echo "--- run ${r}/${runs} ---"
|
||||||
python3 - "$ubatch" "$port" "$out_tokens" "$CTX" "$log" <<'PY'
|
if ! python3 - "$ubatch" "$port" "$out_tokens" "$CTX" "$log" "$r" <<'PY'
|
||||||
import sys, time, json, urllib.request
|
import sys, time, json, urllib.request
|
||||||
ubatch, port, out_tokens, ctx, log = sys.argv[1], sys.argv[2], int(sys.argv[3]), int(sys.argv[4]), sys.argv[5]
|
ubatch, port, out_tokens, ctx, log, run_number = sys.argv[1], sys.argv[2], int(sys.argv[3]), int(sys.argv[4]), sys.argv[5], sys.argv[6]
|
||||||
|
|
||||||
# Flowing prose at ~75% of ctx so MTP acceptance is realistic.
|
# Flowing prose at ~75% of ctx so MTP acceptance is realistic.
|
||||||
VAR = (
|
VAR = (
|
||||||
@@ -175,8 +191,12 @@ body = json.dumps({
|
|||||||
req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/completions",
|
req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/completions",
|
||||||
data=body, headers={"Content-Type": "application/json"})
|
data=body, headers={"Content-Type": "application/json"})
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
with urllib.request.urlopen(req, timeout=7200) as resp:
|
try:
|
||||||
out = json.load(resp)
|
with urllib.request.urlopen(req, timeout=7200) as resp:
|
||||||
|
out = json.load(resp)
|
||||||
|
except Exception:
|
||||||
|
print("request failed; preserving benchmark cleanup", file=sys.stderr)
|
||||||
|
raise
|
||||||
dt = time.time() - t0
|
dt = time.time() - t0
|
||||||
pt = out.get("usage", {}).get("prompt_tokens", 0)
|
pt = out.get("usage", {}).get("prompt_tokens", 0)
|
||||||
ct = out.get("usage", {}).get("completion_tokens", 0)
|
ct = out.get("usage", {}).get("completion_tokens", 0)
|
||||||
@@ -190,24 +210,33 @@ def grab(arr):
|
|||||||
|
|
||||||
g_tps = ""
|
g_tps = ""
|
||||||
if evals and ct:
|
if evals and ct:
|
||||||
# eval time line: "eval time = .. ms for N tokens"
|
# eval timing line: "eval time = .. ms / N tokens (... tokens per second)"
|
||||||
ev = evals[-1].split("= ")[-1]
|
import re
|
||||||
ms = float(ev.split(" for")[0].strip())
|
match = re.search(r"eval time =\s*([0-9.]+) ms", evals[-1])
|
||||||
g_tps = f"{ct / (ms / 1000):.2f}"
|
if match:
|
||||||
print(f"== RESULT ubatch={ubatch} run={r} "
|
ms = float(match.group(1))
|
||||||
|
g_tps = f"{ct / (ms / 1000):.2f}"
|
||||||
|
print(f"== RESULT ubatch={ubatch} run={run_number} "
|
||||||
f"PREFILL={grab(prefill)} GEN_TPS={g_tps} "
|
f"PREFILL={grab(prefill)} GEN_TPS={g_tps} "
|
||||||
f"pt={pt} ct={ct} wall={dt:.1f}s")
|
f"pt={pt} ct={ct} wall={dt:.1f}s")
|
||||||
PY
|
PY
|
||||||
|
then
|
||||||
|
kill "$pid" 2>/dev/null || true
|
||||||
|
wait "$pid" 2>/dev/null || true
|
||||||
|
SCRATCH_PID=""
|
||||||
|
return 2
|
||||||
|
fi
|
||||||
done
|
done
|
||||||
kill "$pid" 2>/dev/null || true
|
kill "$pid" 2>/dev/null || true
|
||||||
wait "$pid" 2>/dev/null || true
|
wait "$pid" 2>/dev/null || true
|
||||||
|
SCRATCH_PID=""
|
||||||
}
|
}
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
stop_live_service
|
stop_live_service
|
||||||
local idx=0 ubatch port
|
local idx=0 ubatch port
|
||||||
printf "%-10s %-14s %-10s\n" "UBATCH" "PREFILL" "GEN_TPS"
|
printf "%-10s %-14s %-10s\n" "UBATCH" "PREFILL" "GEN_TPS"
|
||||||
for ubatch in $(seq "$UBATCH_MIN" "$UBATCH_STEP" "$UBATCH_MAX"); do
|
for ubatch in $UBATCH_VALUES; do
|
||||||
port=$((PORT_BASE + idx))
|
port=$((PORT_BASE + idx))
|
||||||
run_one "$ubatch" "$port" "$OUT_TOKENS" "$RUNS"
|
run_one "$ubatch" "$port" "$OUT_TOKENS" "$RUNS"
|
||||||
idx=$((idx + 1))
|
idx=$((idx + 1))
|
||||||
|
|||||||
+9
-7
@@ -154,11 +154,11 @@ fi
|
|||||||
if [ '${COPY_SHARED[$TARGET]}' = '1' ]; then
|
if [ '${COPY_SHARED[$TARGET]}' = '1' ]; then
|
||||||
echo '[3/5] Copying shared configs into place...'
|
echo '[3/5] Copying shared configs into place...'
|
||||||
mkdir -p /opt/searxng/config /opt/prometheus/config /opt/loki/config /opt/alloy/config
|
mkdir -p /opt/searxng/config /opt/prometheus/config /opt/loki/config /opt/alloy/config
|
||||||
cp shared/searxng/settings.yml /opt/searxng/config/settings.yml
|
${SUDO} cp shared/searxng/settings.yml /opt/searxng/config/settings.yml
|
||||||
cp shared/searxng/limiter.toml /opt/searxng/config/limiter.toml
|
${SUDO} cp shared/searxng/limiter.toml /opt/searxng/config/limiter.toml
|
||||||
cp shared/prometheus/prometheus.yml /opt/prometheus/config/prometheus.yml
|
${SUDO} cp shared/prometheus/prometheus.yml /opt/prometheus/config/prometheus.yml
|
||||||
cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
|
${SUDO} cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
|
||||||
cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
|
${SUDO} cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
|
||||||
|
|
||||||
echo ' Rendering Caddyfile (basicauth hash from .env)...'
|
echo ' Rendering Caddyfile (basicauth hash from .env)...'
|
||||||
# Extract the hash with grep/cut rather than sourcing .env: sourcing a
|
# Extract the hash with grep/cut rather than sourcing .env: sourcing a
|
||||||
@@ -213,7 +213,9 @@ if [ '${COMPOSE_UP[$TARGET]}' = '1' ]; then
|
|||||||
podman build --no-cache --label cfhash=\$cfhash -t \$img:latest -f dankclaw/\$dir/Containerfile .
|
podman build --no-cache --label cfhash=\$cfhash -t \$img:latest -f dankclaw/\$dir/Containerfile .
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
podman-compose -f '${COMPOSE[$TARGET]}' up -d
|
if ! podman-compose -f '${COMPOSE[$TARGET]}' up -d; then
|
||||||
|
echo ' WARN: compose restart failed; continuing with LLM deployment.' >&2
|
||||||
|
fi
|
||||||
|
|
||||||
# A Caddyfile render is a bind-mount content change, not a container
|
# A Caddyfile render is a bind-mount content change, not a container
|
||||||
# config change: the caddy container often survives 'up -d' and keeps
|
# config change: the caddy container often survives 'up -d' and keeps
|
||||||
@@ -233,7 +235,7 @@ if [ '${COMPOSE_UP[$TARGET]}' = '1' ]; then
|
|||||||
done
|
done
|
||||||
if [ \"\$caddy_reloaded\" -ne 1 ]; then
|
if [ \"\$caddy_reloaded\" -ne 1 ]; then
|
||||||
echo \" Caddy reload failed; restarting the container to load the file.\"
|
echo \" Caddy reload failed; restarting the container to load the file.\"
|
||||||
podman restart caddy
|
podman restart caddy || true
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
echo '[4/5] Skipping compose restart (stack runs as bare containers on this host)'
|
echo '[4/5] Skipping compose restart (stack runs as bare containers on this host)'
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# Tunables for frank-default-large (kingjones Qwen3.8-27B ROCmFPX Q8 AGENT,
|
# Tunables for frank-default-large (kingjones Qwen3.8 Uncensored Q6 AGENT,
|
||||||
# dankclaw, Strix Halo gfx1151).
|
# dankclaw, Strix Halo gfx1151).
|
||||||
#
|
#
|
||||||
# Runtime: ROCmFPX llama.cpp with TurboQuant support. Serves the kingjones
|
# Runtime: ROCmFPX llama.cpp with TurboQuant support. Serves the kingjones
|
||||||
@@ -9,18 +9,22 @@
|
|||||||
# HIP runtime reads those bare env vars (see the 2026-08-13 TEMP incident).
|
# HIP runtime reads those bare env vars (see the 2026-08-13 TEMP incident).
|
||||||
# llama.cpp reads --host/--port/--temp as CLI flags, never as env vars.
|
# llama.cpp reads --host/--port/--temp as CLI flags, never as env vars.
|
||||||
|
|
||||||
# ROCmFPX build with TurboQuant KV-cache support.
|
# PromptForge ROCmFPX build and its historical ROCm 7.15 SDK environment.
|
||||||
LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server
|
LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server
|
||||||
|
ROCM_DEVEL=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_devel
|
||||||
|
ROCM_LIB=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_libraries
|
||||||
|
ROCM_CORE=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_core
|
||||||
|
|
||||||
# Model files from https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF.
|
# Model files from https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF.
|
||||||
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
|
MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP
|
||||||
MODEL_FILE=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
|
MODEL_FILE=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf
|
||||||
MTP_DRAFT_FILE=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf
|
MTP_DRAFT_FILE=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf
|
||||||
MMPROJ_FILE=mmproj-Qwen3.8-27B-BF16.gguf
|
MMPROJ_FILE=mmproj-Qwen3.8-27B-Q8_0.gguf
|
||||||
|
|
||||||
# Qwen-Sharp chat template (tracked in the repo, staged on the host). The Sharp
|
# Qwen-Sharp chat template (tracked in the repo, staged on the host). The Sharp
|
||||||
# template drives thinking on by default (medium effort) and produces the
|
# template drives thinking on by default (medium effort); the server emits
|
||||||
# peg-native + deepseek chat/reasoning shape the Pi agent expects.
|
# reasoning in DeepSeek format so clients receive reasoning_content separately
|
||||||
|
# instead of literal <think> tags in content.
|
||||||
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
|
||||||
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
|
||||||
|
|
||||||
@@ -37,13 +41,13 @@ LISTEN_PORT=7770
|
|||||||
CTX_SIZE=262144
|
CTX_SIZE=262144
|
||||||
PARALLEL=1
|
PARALLEL=1
|
||||||
# Batch/ubatch follow the existing qualified Dankclaw profile.
|
# Batch/ubatch follow the existing qualified Dankclaw profile.
|
||||||
BATCH_SIZE=512
|
BATCH_SIZE=4096
|
||||||
UBATCH_SIZE=512
|
UBATCH_SIZE=4096
|
||||||
NUM_THREADS=16
|
NUM_THREADS=16
|
||||||
THREAD_BATCH=32
|
THREAD_BATCH=32
|
||||||
CACHE_TYPE_K=f16
|
CACHE_TYPE_K=f16
|
||||||
CACHE_TYPE_V=turbo4
|
CACHE_TYPE_V=f16
|
||||||
CTX_CHECKPOINTS=32
|
CTX_CHECKPOINTS=16
|
||||||
CACHE_RAM=8192
|
CACHE_RAM=8192
|
||||||
|
|
||||||
# Standalone MTP draft head, loaded from MTP_DRAFT_FILE by the serve script.
|
# Standalone MTP draft head, loaded from MTP_DRAFT_FILE by the serve script.
|
||||||
@@ -60,10 +64,10 @@ MIN_P=0.0
|
|||||||
PRESENCE_PENALTY=1.5
|
PRESENCE_PENALTY=1.5
|
||||||
|
|
||||||
# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
|
# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
|
||||||
# keep the reasoning trail in the conversation). reasoning-format none lets the
|
# keep the reasoning trail in the conversation). DeepSeek format projects the
|
||||||
# template project the reasoning content into reasoning_content.
|
# reasoning into the API's reasoning_content field instead of content tags.
|
||||||
REASONING=on
|
REASONING=on
|
||||||
REASONING_FORMAT=none
|
REASONING_FORMAT=deepseek
|
||||||
|
|
||||||
# Strix Halo unified memory + native gfx1151 detection for this ROCmFPX build.
|
# Strix Halo unified memory + native gfx1151 detection for this ROCmFPX build.
|
||||||
HSA_OVERRIDE_GFX_VERSION=11.5.1
|
HSA_OVERRIDE_GFX_VERSION=11.5.1
|
||||||
|
|||||||
Executable
+32
@@ -0,0 +1,32 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Unlock and mount Dankserver's encrypted entropy volume.
|
||||||
|
#
|
||||||
|
# The desktop flow uses the system's crypttab and fstab entries. Reuse those
|
||||||
|
# entries here instead of embedding a passphrase, mount options, or ownership
|
||||||
|
# changes in this helper. udisksctl reads the passphrase through its secure
|
||||||
|
# desktop/terminal authentication path.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
MOUNTPOINT='/mnt/entropy'
|
||||||
|
MAPPER='/dev/mapper/luks-cd86c728-459e-4f3a-bcc2-41ecf84685ec'
|
||||||
|
|
||||||
|
if mountpoint -q "$MOUNTPOINT"; then
|
||||||
|
printf '%s is already mounted.\n' "$MOUNTPOINT"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -e /dev/md0 ]; then
|
||||||
|
printf 'ERROR: RAID device /dev/md0 is not available.\n' >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -e "$MAPPER" ]; then
|
||||||
|
udisksctl unlock --block-device /dev/md0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# udisks2 may mount automatically as part of unlocking. Do not attempt a
|
||||||
|
# second mount in that case.
|
||||||
|
if ! mountpoint -q "$MOUNTPOINT"; then
|
||||||
|
sudo mount "$MOUNTPOINT"
|
||||||
|
fi
|
||||||
|
printf 'Mounted %s.\n' "$MOUNTPOINT"
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve the kingjones Qwen3.8-27B ROCmFPX Q8 AGENT model as frank-default-large.
|
# Serve the kingjones Qwen3.8 Uncensored Q6 AGENT model as frank-default-large.
|
||||||
# The deployment uses a standalone MTP draft head and Turbo4 V-side KV cache.
|
# The deployment uses a standalone MTP draft head and f16 KV cache.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
@@ -12,23 +12,23 @@ if [[ -f "$ENV_FILE" ]]; then
|
|||||||
set +a
|
set +a
|
||||||
fi
|
fi
|
||||||
|
|
||||||
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server}"
|
: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server}"
|
||||||
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT}"
|
: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP}"
|
||||||
: "${MODEL_FILE:=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
|
: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
|
||||||
: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
|
: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
|
||||||
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-BF16.gguf}"
|
: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-Q8_0.gguf}"
|
||||||
: "${ALIAS:=frank-default-large}"
|
: "${ALIAS:=frank-default-large}"
|
||||||
: "${BIND_ADDR:=0.0.0.0}"
|
: "${BIND_ADDR:=0.0.0.0}"
|
||||||
: "${LISTEN_PORT:=7770}"
|
: "${LISTEN_PORT:=7770}"
|
||||||
: "${CTX_SIZE:=262144}"
|
: "${CTX_SIZE:=262144}"
|
||||||
: "${PARALLEL:=1}"
|
: "${PARALLEL:=1}"
|
||||||
: "${BATCH_SIZE:=2048}"
|
: "${BATCH_SIZE:=4096}"
|
||||||
: "${UBATCH_SIZE:=512}"
|
: "${UBATCH_SIZE:=4096}"
|
||||||
: "${NUM_THREADS:=16}"
|
: "${NUM_THREADS:=16}"
|
||||||
: "${THREAD_BATCH:=32}"
|
: "${THREAD_BATCH:=32}"
|
||||||
: "${CACHE_TYPE_K:=f16}"
|
: "${CACHE_TYPE_K:=f16}"
|
||||||
: "${CACHE_TYPE_V:=turbo4}"
|
: "${CACHE_TYPE_V:=f16}"
|
||||||
: "${CTX_CHECKPOINTS:=32}"
|
: "${CTX_CHECKPOINTS:=16}"
|
||||||
: "${CACHE_RAM:=8192}"
|
: "${CACHE_RAM:=8192}"
|
||||||
: "${MTP_N_MAX:=4}"
|
: "${MTP_N_MAX:=4}"
|
||||||
: "${MTP_N_MIN:=0}"
|
: "${MTP_N_MIN:=0}"
|
||||||
@@ -55,14 +55,18 @@ if [[ ! -x "$LLAMA_BIN" ]]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
|
SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
|
||||||
ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
|
ROCM_PATH="${ROCM_DEVEL:-${ROCM_PATH:-/opt/rocm}}"
|
||||||
LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib:$ROCM_PATH/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
if [[ -n "${ROCM_LIB:-}" && -n "${ROCM_CORE:-}" ]]; then
|
||||||
|
LIBRARY_PATH="$ROCM_LIB/lib:$ROCM_CORE/lib:$ROCM_DEVEL/lib/llvm/lib:$SERVER_DIR:${LD_LIBRARY_PATH:-}"
|
||||||
|
else
|
||||||
|
LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib:$ROCM_PATH/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
|
||||||
|
fi
|
||||||
|
|
||||||
exec /usr/bin/env \
|
exec /usr/bin/env \
|
||||||
ROCM_PATH="$ROCM_PATH" \
|
ROCM_PATH="$ROCM_PATH" \
|
||||||
HIP_VISIBLE_DEVICES=0 \
|
HIP_VISIBLE_DEVICES=0 \
|
||||||
HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
|
|
||||||
LD_LIBRARY_PATH="$LIBRARY_PATH" \
|
LD_LIBRARY_PATH="$LIBRARY_PATH" \
|
||||||
|
HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
|
||||||
GGML_CUDA_GRAPH_OPT=0 \
|
GGML_CUDA_GRAPH_OPT=0 \
|
||||||
LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 \
|
LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 \
|
||||||
LLAMA_MTP_CPU_ARGMAX_FASTPATH=1 \
|
LLAMA_MTP_CPU_ARGMAX_FASTPATH=1 \
|
||||||
|
|||||||
Executable
+148
@@ -0,0 +1,148 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Run only the Bats tests affected by the staged paths.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# scripts/test-changed.sh # inspect the staged commit
|
||||||
|
# scripts/test-changed.sh --list # print selected test files
|
||||||
|
# scripts/test-changed.sh --paths FILE ...
|
||||||
|
#
|
||||||
|
# --paths is intended for tests and local experimentation. The normal mode is
|
||||||
|
# deliberately staged-only so a pre-commit hook cannot accidentally validate
|
||||||
|
# an unrelated dirty working-tree file.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
LIST_ONLY=0
|
||||||
|
USE_PATHS=0
|
||||||
|
declare -a CHANGED=()
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf 'ERROR: %s\n' "$*" >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--list)
|
||||||
|
LIST_ONLY=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--paths)
|
||||||
|
USE_PATHS=1
|
||||||
|
shift
|
||||||
|
while [ "$#" -gt 0 ] && [[ "$1" != --* ]]; do
|
||||||
|
CHANGED+=("$1")
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
;;
|
||||||
|
--)
|
||||||
|
shift
|
||||||
|
CHANGED+=("$@")
|
||||||
|
break
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option '$1' (use --list or --paths FILE ...)"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
cd "$REPO_ROOT"
|
||||||
|
if [ "$USE_PATHS" -eq 0 ]; then
|
||||||
|
while IFS= read -r -d '' path; do
|
||||||
|
CHANGED+=("$path")
|
||||||
|
done < <(git diff --cached --name-only -z --diff-filter=ACMRTUXB)
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ "${#CHANGED[@]}" -gt 0 ] || {
|
||||||
|
printf 'No staged paths; no focused tests to run.\n'
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
declare -a TESTS=()
|
||||||
|
add_test() {
|
||||||
|
local test="$1"
|
||||||
|
local existing
|
||||||
|
for existing in "${TESTS[@]:-}"; do
|
||||||
|
[ "$existing" != "$test" ] || return 0
|
||||||
|
done
|
||||||
|
TESTS+=("$test")
|
||||||
|
}
|
||||||
|
|
||||||
|
for path in "${CHANGED[@]}"; do
|
||||||
|
case "$path" in
|
||||||
|
tests/*.bats)
|
||||||
|
add_test "$path"
|
||||||
|
;;
|
||||||
|
tests/helpers.bash|tests/run.sh)
|
||||||
|
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
||||||
|
;;
|
||||||
|
.githooks/*)
|
||||||
|
add_test tests/changed_tests.bats
|
||||||
|
;;
|
||||||
|
dankserver/compose.yaml|dankclaw/compose.yaml)
|
||||||
|
add_test tests/compose_validity.bats
|
||||||
|
add_test tests/env_consistency.bats
|
||||||
|
;;
|
||||||
|
dankserver/Caddyfile|dankclaw/Caddyfile)
|
||||||
|
add_test tests/caddyfile.bats
|
||||||
|
;;
|
||||||
|
dankserver/.env.example|dankclaw/.env.example)
|
||||||
|
add_test tests/env_consistency.bats
|
||||||
|
;;
|
||||||
|
shared/homepage/*)
|
||||||
|
add_test tests/homepage_caddy.bats
|
||||||
|
add_test tests/homepage_config.bats
|
||||||
|
add_test tests/homepage_deploy.bats
|
||||||
|
add_test tests/homepage_pages.bats
|
||||||
|
add_test tests/homepage_checker.bats
|
||||||
|
;;
|
||||||
|
shared/searxng/*)
|
||||||
|
add_test tests/searxng_limiter.bats
|
||||||
|
add_test tests/searxng_quadlet.bats
|
||||||
|
;;
|
||||||
|
dankclaw/*.container|dankclaw/*.network)
|
||||||
|
add_test tests/open_webui_quadlet.bats
|
||||||
|
add_test tests/pihole.bats
|
||||||
|
add_test tests/searxng_quadlet.bats
|
||||||
|
;;
|
||||||
|
scripts/mount-entropy.sh)
|
||||||
|
add_test tests/mount_entropy.bats
|
||||||
|
;;
|
||||||
|
scripts/frank-default-large.env|scripts/serve-frank-large.sh)
|
||||||
|
add_test tests/frank_large_config.bats
|
||||||
|
add_test tests/llm_secrets.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
scripts/frank-default-small.env|scripts/serve-frank-small.sh)
|
||||||
|
add_test tests/frank_small_config.bats
|
||||||
|
add_test tests/llm_secrets.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
scripts/*.service)
|
||||||
|
add_test tests/llm_units.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
scripts/*)
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
docs/*)
|
||||||
|
add_test tests/docs_completeness.bats
|
||||||
|
;;
|
||||||
|
dankclaw/hermes/*)
|
||||||
|
add_test tests/docs_completeness.bats
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "no focused-test mapping for '$path'; add one before committing"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$LIST_ONLY" -eq 1 ]; then
|
||||||
|
printf '%s\n' "${TESTS[@]}"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
command -v bats >/dev/null 2>&1 || die 'bats is not installed'
|
||||||
|
printf 'Running focused tests for %d changed path(s):\n' "${#CHANGED[@]}"
|
||||||
|
printf ' %s\n' "${TESTS[@]}"
|
||||||
|
bats --print-output-on-failure "${TESTS[@]}"
|
||||||
Executable
+234
@@ -0,0 +1,234 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# tmux-resurrect.sh — snapshot and restore the exact layout of a tmux session
|
||||||
|
# on Dankclaw, including each pane's start directory and foreground command.
|
||||||
|
#
|
||||||
|
# Use it after a reboot (or when the display session is gone) to reopen the
|
||||||
|
# same windows/panes on the Dankclaw console:
|
||||||
|
#
|
||||||
|
# scripts/tmux-resurrect.sh snapshot # record the live session to JSON
|
||||||
|
# scripts/tmux-resurrect.sh restore # rebuild it, then attach
|
||||||
|
#
|
||||||
|
# Data file: ~/.tmux-resurrect/<session>.json (set TMUX_RES_DIR to relocate).
|
||||||
|
# Re-run snapshot as the session changes so the restore stays faithful.
|
||||||
|
#
|
||||||
|
# How each pane's command is captured: a tmux pane runs one main shell
|
||||||
|
# (#{pane_pid}). The foreground command is that shell's direct child — chosen
|
||||||
|
# as the child that launched its own child (ssh, sudo ...) or the first child
|
||||||
|
# when none did (btop sitting at the top of a pane). An idle shell yields no
|
||||||
|
# command and is left at its prompt on restore.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
TMUX_RES_DIR="${TMUX_RES_DIR:-$HOME/.tmux-resurrect}"
|
||||||
|
DEFAULT_SESSION="0"
|
||||||
|
|
||||||
|
die() { printf 'tmux-resurrect: %s\n' "$*" >&2; exit 1; }
|
||||||
|
b64() { base64 -w0; }
|
||||||
|
b64d() { base64 -d; }
|
||||||
|
|
||||||
|
# fg_command <shell_pid>: print the foreground command of a tmux pane, or the
|
||||||
|
# empty string when the shell is idle.
|
||||||
|
fg_command() {
|
||||||
|
local pid="$1" kids=() kid cmd_pid=""
|
||||||
|
mapfile -t kids < <(pgrep -P "$pid" 2>/dev/null || true)
|
||||||
|
if [ "${#kids[@]}" -eq 0 ]; then return 0; fi
|
||||||
|
# Prefer a child that launched its own child (a launcher such as ssh or
|
||||||
|
# sudo); fall back to the first direct child (e.g. btop at the top).
|
||||||
|
for kid in "${kids[@]}"; do
|
||||||
|
if pgrep -P "$kid" >/dev/null 2>&1; then
|
||||||
|
cmd_pid="$kid"; break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ -z "$cmd_pid" ]; then cmd_pid="${kids[0]}"; fi
|
||||||
|
tr '\0' ' ' < "/proc/$cmd_pid/cmdline" 2>/dev/null | sed -E 's/[[:space:]]+$//'
|
||||||
|
}
|
||||||
|
|
||||||
|
# snapshot <session> [dest.json]
|
||||||
|
snapshot() {
|
||||||
|
local session="${1:-$DEFAULT_SESSION}" dest="${2:-}"
|
||||||
|
local exists
|
||||||
|
exists=$(tmux list-sessions -F '#{session_name}' 2>/dev/null | grep -Fxq "$session" && echo 1 || echo 0)
|
||||||
|
[ "$exists" = 1 ] || die "no tmux session named '$session'"
|
||||||
|
mkdir -p "$TMUX_RES_DIR"
|
||||||
|
[ -z "$dest" ] && dest="$TMUX_RES_DIR/$session.json"
|
||||||
|
python3 - "$dest" "$session" 3< <(
|
||||||
|
{
|
||||||
|
while IFS=$'\t' read -r widx wname wlayout; do
|
||||||
|
printf 'W\t%s\t%s\t%s\n' "$widx" "$(printf '%s' "$wname" | b64)" "$(printf '%s' "$wlayout" | b64)"
|
||||||
|
while IFS=$'\t' read -r pidx ppid pdir; do
|
||||||
|
pcmd=$(fg_command "$ppid")
|
||||||
|
printf 'P\t%s\t%s\t%s\t%s\n' "$widx" "$pidx" \
|
||||||
|
"$(printf '%s' "$pdir" | b64)" "$(printf '%s' "$pcmd" | b64)"
|
||||||
|
done < <(tmux list-panes -t "$session:$widx" \
|
||||||
|
-F $'#{pane_index}\t#{pane_pid}\t#{pane_current_path}' | sort -n -k1,1)
|
||||||
|
done < <(tmux list-windows -t "$session" \
|
||||||
|
-F $'#{window_index}\t#{window_name}\t#{window_layout}' | sort -n -k1,1)
|
||||||
|
}
|
||||||
|
) > /dev/null <<'PY'
|
||||||
|
import sys, json, base64, os
|
||||||
|
dest, session = sys.argv[1], sys.argv[2]
|
||||||
|
wmap, panes = {}, {}
|
||||||
|
for line in os.fdopen(3).read().splitlines():
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
p = line.split('\t')
|
||||||
|
if p[0] == 'W':
|
||||||
|
wmap[int(p[1])] = {"name": base64.b64decode(p[2]).decode(),
|
||||||
|
"layout": base64.b64decode(p[3]).decode()}
|
||||||
|
elif p[0] == 'P':
|
||||||
|
widx = int(p[1])
|
||||||
|
panes.setdefault(widx, []).append({
|
||||||
|
"index": int(p[2]),
|
||||||
|
"dir": base64.b64decode(p[3]).decode(),
|
||||||
|
"cmd": base64.b64decode(p[4]).decode(),
|
||||||
|
})
|
||||||
|
windows = []
|
||||||
|
for widx in sorted(wmap):
|
||||||
|
windows.append({
|
||||||
|
"index": widx,
|
||||||
|
"name": wmap[widx]["name"],
|
||||||
|
"layout": wmap[widx]["layout"],
|
||||||
|
"panes": sorted(panes.get(widx, []), key=lambda x: x["index"]),
|
||||||
|
})
|
||||||
|
with open(dest, "w") as f:
|
||||||
|
json.dump({"session": session, "windows": windows}, f, indent=2, ensure_ascii=False)
|
||||||
|
f.write("\n")
|
||||||
|
PY
|
||||||
|
printf 'snapshot: wrote %s (session %s)\n' "$dest" "$session"
|
||||||
|
}
|
||||||
|
|
||||||
|
# emit_plan <src.json> [session]: python reads the JSON and prints a base64
|
||||||
|
# TSV plan (W/P records) for the bash orchestrator to consume.
|
||||||
|
emit_plan() {
|
||||||
|
python3 - "$1" "$2" <<'PY'
|
||||||
|
import sys, json, base64
|
||||||
|
src, session = sys.argv[1], sys.argv[2]
|
||||||
|
with open(src) as f:
|
||||||
|
data = json.load(f)
|
||||||
|
wins = data.get("windows", [])
|
||||||
|
if session:
|
||||||
|
wins = [w for w in wins if str(w.get("index")) == "0"]
|
||||||
|
def b64(s): return base64.b64encode(s.encode()).decode()
|
||||||
|
for w in wins:
|
||||||
|
print('W\t%s\t%s\t%s' % (w["index"], b64(w["name"]), b64(w["layout"])))
|
||||||
|
for pan in w["panes"]:
|
||||||
|
print('P\t%s\t%s\t%s\t%s' % (w["index"], pan["index"],
|
||||||
|
b64(pan["dir"]), b64(pan["cmd"])))
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
# restore [session] [--no-attach] [--dry-run]
|
||||||
|
restore() {
|
||||||
|
local session="${1:-$DEFAULT_SESSION}"
|
||||||
|
shift || true
|
||||||
|
local keep=0 noattach=0 dryrun=0 a
|
||||||
|
for a in "$@"; do
|
||||||
|
case "$a" in
|
||||||
|
--keep) keep=1 ;;
|
||||||
|
--no-attach) noattach=1 ;;
|
||||||
|
--dry-run) dryrun=1 ;;
|
||||||
|
*) die "unknown restore option: $a" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
local dest="$TMUX_RES_DIR/$session.json"
|
||||||
|
[ -f "$dest" ] || die "no snapshot at $dest (run snapshot first)"
|
||||||
|
|
||||||
|
# Restore wrapper: run tmux, or print the command under --dry-run.
|
||||||
|
tmuxr() {
|
||||||
|
if [ "$dryrun" = 1 ]; then printf ' tmux %s\n' "$*"; else tmux "$@"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
local target_exists
|
||||||
|
target_exists=$(tmux list-sessions -F '#{session_name}' 2>/dev/null | grep -Fxq "$session" && echo 1 || echo 0)
|
||||||
|
if [ "$target_exists" = 1 ]; then
|
||||||
|
if [ "$keep" = 1 ]; then
|
||||||
|
die "session '$session' already exists; rebuild without --keep or save a copy"
|
||||||
|
fi
|
||||||
|
if [ "$dryrun" = 1 ]; then printf ' # kill existing session %s\n' "$session"
|
||||||
|
else tmux kill-session --force "$session"; fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Parse the plan into flat, window-major lists.
|
||||||
|
local -a WNAME=() WLAY=() WSTART=() PD=() PC=()
|
||||||
|
local tag idx nameb layoutb dirb cmdb
|
||||||
|
while IFS=$'\t' read -r tag idx nameb layoutb dirb cmdb; do
|
||||||
|
case "$tag" in
|
||||||
|
W) WNAME+=("$(printf '%s' "$nameb" | b64d)"); WLAY+=("$(printf '%s' "$layoutb" | b64d)"); WSTART+=("${#PD[@]}") ;;
|
||||||
|
P) PD+=("$(printf '%s' "$dirb" | b64d)"); PC+=("$(printf '%s' "$cmdb" | b64d)") ;;
|
||||||
|
esac
|
||||||
|
done < <(emit_plan "$dest" "")
|
||||||
|
|
||||||
|
local n=${#WNAME[@]} w first last npanes i curwin wname wlayout dir cmd
|
||||||
|
for ((w = 0; w < n; w++)); do
|
||||||
|
first=${WSTART[w]}
|
||||||
|
if (( w + 1 < n )); then last=${WSTART[w + 1]}; else last=${#PD[@]}; fi
|
||||||
|
npanes=$(( last - first ))
|
||||||
|
wname="${WNAME[w]}"; wlayout="${WLAY[w]}"
|
||||||
|
if [ "$dryrun" = 1 ]; then
|
||||||
|
if (( w == 0 )); then
|
||||||
|
tmuxr new-session -d -s "$session" -n "$wname" -c "${PD[first]}"
|
||||||
|
else
|
||||||
|
tmuxr new-window -d -t "$session" -n "$wname" -c "${PD[first]}"
|
||||||
|
fi
|
||||||
|
curwin="$session:$wname"
|
||||||
|
elif (( w == 0 )); then
|
||||||
|
curwin=$(tmux new-session -d -P -F '#{window_id}' \
|
||||||
|
-s "$session" -n "$wname" -c "${PD[first]}")
|
||||||
|
else
|
||||||
|
curwin=$(tmux new-window -d -P -F '#{window_id}' \
|
||||||
|
-t "$session" -n "$wname" -c "${PD[first]}")
|
||||||
|
fi
|
||||||
|
# Create the remaining panes in recorded order (index == creation order).
|
||||||
|
for ((i = first + 1; i < last; i++)); do
|
||||||
|
tmuxr split-window -t "$curwin" -c "${PD[i]}"
|
||||||
|
done
|
||||||
|
# Restore the exact geometry for this window.
|
||||||
|
tmuxr select-layout -t "$curwin" "$wlayout"
|
||||||
|
# Launch each pane's recorded command, matched to panes by ascending
|
||||||
|
# pane index (so this is correct regardless of base-index).
|
||||||
|
local -a ordered=()
|
||||||
|
mapfile -t ordered < <(tmux list-panes -t "$curwin" \
|
||||||
|
-F $'#{pane_id}\t#{pane_target}' | sort -n -k1,1)
|
||||||
|
local k=0 tgt
|
||||||
|
for tgt in "${ordered[@]}"; do
|
||||||
|
cmd="${PC[first + k]}"
|
||||||
|
if [ -n "$cmd" ]; then tmuxr send-keys -t "$tgt" "$cmd" Enter; fi
|
||||||
|
k=$((k + 1))
|
||||||
|
done
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$noattach" = 1 ]; then
|
||||||
|
return 0
|
||||||
|
elif [ -t 0 ]; then
|
||||||
|
tmuxr attach -t "$session"
|
||||||
|
elif [ "$dryrun" = 0 ]; then
|
||||||
|
echo "session '$session' rebuilt; attach with: tmux attach -t $session"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'USAGE'
|
||||||
|
tmux-resurrect.sh — snapshot & restore a tmux session on Dankclaw.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
tmux-resurrect.sh snapshot [session] [dest.json] record the live session
|
||||||
|
tmux-resurrect.sh restore [session] [options] rebuild the session
|
||||||
|
options:
|
||||||
|
--keep skip if the session already exists (default: rebuild)
|
||||||
|
--no-attach rebuild without attaching to this terminal
|
||||||
|
--dry-run print the commands instead of running them
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
local cmd="${1:-snapshot}"
|
||||||
|
shift || true
|
||||||
|
case "$cmd" in
|
||||||
|
snapshot) snapshot "$@" ;;
|
||||||
|
restore) restore "$@" ;;
|
||||||
|
-h|--help|help) usage ;;
|
||||||
|
*) die "unknown command: $cmd (use snapshot|restore)" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
@@ -21,13 +21,93 @@ valkey:
|
|||||||
url: "valkey://searxng-valkey:6379/0"
|
url: "valkey://searxng-valkey:6379/0"
|
||||||
|
|
||||||
outgoing:
|
outgoing:
|
||||||
request_timeout: 3.0
|
request_timeout: 5.0
|
||||||
max_request_timeout: 10.0
|
max_request_timeout: 15.0
|
||||||
pool_connections: 100
|
pool_connections: 100
|
||||||
pool_maxsize: 20
|
pool_maxsize: 20
|
||||||
enable_http2: true
|
enable_http2: true
|
||||||
keepalive_expiry: 5.0
|
keepalive_expiry: 5.0
|
||||||
retries: 3
|
retries: 1
|
||||||
|
|
||||||
|
# Keep default aggregation anonymous and bounded. Disabled engines remain
|
||||||
|
# available for explicit selection/on-demand use; inactive engines are removed.
|
||||||
|
# Provider-specific overrides below also reduce pagination fan-out.
|
||||||
|
engines:
|
||||||
|
- name: bing
|
||||||
|
disabled: false
|
||||||
|
weight: 2
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: bing images
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: bing news
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: bing videos
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: duckduckgo
|
||||||
|
disabled: false
|
||||||
|
weight: 2
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: duckduckgo images
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: duckduckgo news
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: duckduckgo videos
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: wikipedia
|
||||||
|
weight: 1
|
||||||
|
timeout: 5.0
|
||||||
|
- name: google cse
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: google cse images
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: google news
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: google scholar
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: brave
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: brave.images
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: brave.news
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: brave.videos
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: startpage
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: startpage images
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
- name: startpage news
|
||||||
|
disabled: true
|
||||||
|
paging: false
|
||||||
|
timeout: 5.0
|
||||||
|
|
||||||
plugins:
|
plugins:
|
||||||
searx.plugins.unit_converter.SXNGPlugin:
|
searx.plugins.unit_converter.SXNGPlugin:
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: the changed-test wrapper selects only the tests associated with
|
||||||
|
# staged paths and refuses to silently run the entire suite.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
@test "changed-test wrapper is executable and uses staged diff by default" {
|
||||||
|
[ -x "$REPO_ROOT/scripts/test-changed.sh" ]
|
||||||
|
[ -x "$REPO_ROOT/.githooks/pre-commit" ]
|
||||||
|
run grep -F 'scripts/test-changed.sh' "$REPO_ROOT/.githooks/pre-commit"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
run grep -F 'git diff --cached' "$REPO_ROOT/scripts/test-changed.sh"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "changed-test wrapper maps a compose change to compose and environment tests" {
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankserver/compose.yaml
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/compose_validity.bats"* ]]
|
||||||
|
[[ "$output" == *"tests/env_consistency.bats"* ]]
|
||||||
|
[[ "$output" != *"tests/homepage_pages.bats"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "changed-test wrapper maps a service script to its focused config tests" {
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/serve-frank-small.sh
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/scripts_syntax.bats"* ]]
|
||||||
|
[[ "$output" == *"tests/frank_small_config.bats"* ]]
|
||||||
|
[[ "$output" != *"tests/frank_large_config.bats"* ]]
|
||||||
|
}
|
||||||
@@ -11,14 +11,15 @@ UNIT_FILE="$REPO_ROOT/scripts/frank-default-large.service"
|
|||||||
TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
|
TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
|
||||||
|
|
||||||
@test "frank-default-large serves the kingjones AGENT model" {
|
@test "frank-default-large serves the kingjones AGENT model" {
|
||||||
grep -q '^MODEL_FILE=Qwen3\.8-27B-Q8_0_ROCMFPX_AGENT\.gguf$' "$ENV_FILE"
|
grep -q '^MODEL_FILE=Qwen3\.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT\.gguf$' "$ENV_FILE"
|
||||||
grep -q '^MTP_DRAFT_FILE=' "$ENV_FILE"
|
grep -q '^MTP_DRAFT_FILE=' "$ENV_FILE"
|
||||||
|
grep -q '^MMPROJ_FILE=mmproj-Qwen3\.8-27B-Q8_0\.gguf$' "$ENV_FILE"
|
||||||
! grep -q -- '--kairic-edge' "$SERVE_FILE"
|
! grep -q -- '--kairic-edge' "$SERVE_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "frank-default-large uses the ROCmFPX runtime" {
|
@test "frank-default-large uses the ROCmFPX runtime" {
|
||||||
grep -q 'ROCmFPX/.*/bin/llama-server' "$ENV_FILE"
|
grep -q 'ROCmFPX/.*/bin/llama-server' "$ENV_FILE"
|
||||||
grep -q 'build-kairic/bin/llama-server' "$ENV_FILE"
|
grep -q 'build-promptforge/bin/llama-server' "$ENV_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "frank-default-large runs standalone MTP" {
|
@test "frank-default-large runs standalone MTP" {
|
||||||
@@ -38,6 +39,11 @@ TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
|
|||||||
grep -q ': "${CTX_SIZE:=262144}"' "$SERVE_FILE"
|
grep -q ': "${CTX_SIZE:=262144}"' "$SERVE_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "frank-default-large uses DeepSeek reasoning output" {
|
||||||
|
grep -q '^REASONING_FORMAT=deepseek$' "$ENV_FILE"
|
||||||
|
grep -q -- '--reasoning-format "$REASONING_FORMAT"' "$SERVE_FILE"
|
||||||
|
}
|
||||||
|
|
||||||
@test "frank-default-large has reasoning and preserved reasoning/thinking enabled" {
|
@test "frank-default-large has reasoning and preserved reasoning/thinking enabled" {
|
||||||
grep -q '^REASONING=on$' "$ENV_FILE"
|
grep -q '^REASONING=on$' "$ENV_FILE"
|
||||||
grep -q 'enable_thinking":true' "$ENV_FILE"
|
grep -q 'enable_thinking":true' "$ENV_FILE"
|
||||||
@@ -45,18 +51,37 @@ TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
|
|||||||
grep -q 'preserve_reasoning":true' "$ENV_FILE"
|
grep -q 'preserve_reasoning":true' "$ENV_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "frank-default-large uses Turbo4 only for the V cache" {
|
@test "frank-default-large uses f16 for both KV cache sides" {
|
||||||
grep -q '^CACHE_TYPE_K=f16$' "$ENV_FILE"
|
grep -q '^CACHE_TYPE_K=f16$' "$ENV_FILE"
|
||||||
grep -q '^CACHE_TYPE_V=turbo4$' "$ENV_FILE"
|
grep -q '^CACHE_TYPE_V=f16$' "$ENV_FILE"
|
||||||
grep -qF -- '-ctk "$CACHE_TYPE_K" -ctv "$CACHE_TYPE_V"' "$SERVE_FILE"
|
grep -qF -- '-ctk "$CACHE_TYPE_K" -ctv "$CACHE_TYPE_V"' "$SERVE_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "frank-default-large exposes a swappable ubatch" {
|
@test "frank-default-large exposes a swappable ubatch" {
|
||||||
grep -q '^UBATCH_SIZE=512$' "$ENV_FILE"
|
grep -q '^UBATCH_SIZE=4096$' "$ENV_FILE"
|
||||||
grep -q ': "${UBATCH_SIZE:=512}"' "$SERVE_FILE"
|
grep -q ': "${UBATCH_SIZE:=4096}"' "$SERVE_FILE"
|
||||||
grep -q -- '-ub "$UBATCH_SIZE"' "$SERVE_FILE"
|
grep -q -- '-ub "$UBATCH_SIZE"' "$SERVE_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "large ubatch bench defaults to the active model and requested sweep" {
|
||||||
|
local bench="$REPO_ROOT/scripts/bench-ubatch-rocmi4.sh"
|
||||||
|
grep -q ': "${UBATCH_MIN:=256}"' "$bench"
|
||||||
|
grep -q ': "${UBATCH_MAX:=6144}"' "$bench"
|
||||||
|
grep -q 'UBATCH_VALUES.*256 512 768 1024 1280 1536 2048 3072 4096 5120 6144' "$bench"
|
||||||
|
grep -q ': "${STOPSVC:=1}"' "$bench"
|
||||||
|
grep -q ': "${CTX:=32768}"' "$bench"
|
||||||
|
grep -q ': "${BENCH_BATCH:=32768}"' "$bench"
|
||||||
|
grep -q 'MODEL_FILE' "$bench"
|
||||||
|
grep -q 'REASONING_FORMAT' "$bench"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "large ubatch bench cleans up scratch server on request failure" {
|
||||||
|
local bench="$REPO_ROOT/scripts/bench-ubatch-rocmi4.sh"
|
||||||
|
grep -q 'SCRATCH_PID=' "$bench"
|
||||||
|
grep -q 'kill.*SCRATCH_PID' "$bench"
|
||||||
|
grep -qF 'eval time =\s' "$bench"
|
||||||
|
}
|
||||||
|
|
||||||
@test "frank-default-large uses the tracked Qwen-Sharp template on :7770" {
|
@test "frank-default-large uses the tracked Qwen-Sharp template on :7770" {
|
||||||
grep -q '^CHAT_TEMPLATE_FILE=' "$ENV_FILE"
|
grep -q '^CHAT_TEMPLATE_FILE=' "$ENV_FILE"
|
||||||
grep -q -- '--jinja' "$SERVE_FILE"
|
grep -q -- '--jinja' "$SERVE_FILE"
|
||||||
|
|||||||
@@ -0,0 +1,25 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: the entropy helper unlocks the known LUKS-on-md device and mounts via
|
||||||
|
# the host's fstab entry, preserving the existing mount options and directory
|
||||||
|
# ownership rather than inventing a second mount configuration.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
@test "entropy helper is executable and has the expected safe shell shape" {
|
||||||
|
run bash -c "test -x '$REPO_ROOT/scripts/mount-entropy.sh'"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
|
||||||
|
run grep -E '^set -euo pipefail$' "$REPO_ROOT/scripts/mount-entropy.sh"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
|
||||||
|
run grep -F 'udisksctl unlock --block-device /dev/md0' "$REPO_ROOT/scripts/mount-entropy.sh"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
|
||||||
|
run grep -F 'sudo mount "$MOUNTPOINT"' "$REPO_ROOT/scripts/mount-entropy.sh"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "entropy helper does not embed a secret or alter mount ownership" {
|
||||||
|
run grep -Eiq 'password|secret|chmod|chown' "$REPO_ROOT/scripts/mount-entropy.sh"
|
||||||
|
[ "$status" -eq 1 ]
|
||||||
|
}
|
||||||
@@ -57,6 +57,38 @@ assert not missing, f"pass_ip missing {missing}; got {pass_ip}"
|
|||||||
PYEOF
|
PYEOF
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "SearXNG outgoing policy allows modest VPN latency without retry amplification" {
|
||||||
|
python3 - "$REPO_ROOT/shared/searxng/settings.yml" <<'PYEOF'
|
||||||
|
import sys, yaml
|
||||||
|
with open(sys.argv[1]) as fh:
|
||||||
|
data = yaml.safe_load(fh)
|
||||||
|
outgoing = data.get("outgoing", {})
|
||||||
|
assert outgoing.get("request_timeout") == 5.0, outgoing
|
||||||
|
assert outgoing.get("max_request_timeout") == 15.0, outgoing
|
||||||
|
assert outgoing.get("retries") == 1, outgoing
|
||||||
|
assert outgoing["request_timeout"] < outgoing["max_request_timeout"], outgoing
|
||||||
|
PYEOF
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "SearXNG default provider policy favors anonymous healthy engines" {
|
||||||
|
python3 - "$REPO_ROOT/shared/searxng/settings.yml" <<'PYEOF'
|
||||||
|
import sys, yaml
|
||||||
|
with open(sys.argv[1]) as fh:
|
||||||
|
data = yaml.safe_load(fh)
|
||||||
|
engines = {entry["name"]: entry for entry in data.get("engines", [])}
|
||||||
|
assert engines["bing"]["disabled"] is False, engines.get("bing")
|
||||||
|
assert engines["bing"]["weight"] > 1, engines["bing"]
|
||||||
|
assert engines["bing"]["paging"] is False, engines["bing"]
|
||||||
|
assert engines["duckduckgo"]["disabled"] is False, engines["duckduckgo"]
|
||||||
|
assert engines["duckduckgo"]["paging"] is False, engines["duckduckgo"]
|
||||||
|
for name in ("google cse", "startpage", "brave"):
|
||||||
|
assert engines[name]["disabled"] is True, engines.get(name)
|
||||||
|
assert engines[name].get("inactive", False) is False, engines[name]
|
||||||
|
for name in ("bing", "duckduckgo", "wikipedia"):
|
||||||
|
assert engines[name]["timeout"] == 5.0, engines[name]
|
||||||
|
PYEOF
|
||||||
|
}
|
||||||
|
|
||||||
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
|
@test "deploy.sh copies limiter.toml into /opt/searxng/config" {
|
||||||
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|
grep -q "shared/searxng/limiter.toml" "$REPO_ROOT/scripts/deploy.sh" \
|
||||||
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }
|
|| { echo "deploy.sh does not copy shared/searxng/limiter.toml"; return 1; }
|
||||||
|
|||||||
Reference in new issue
Block a user