---
type: reference
description: "This page collects concrete, sourced recommendations to improve Dankclaw"
tags:
  - llm
timestamp: 2026-09-03
---

# Dankclaw — Improvements Research

This page collects concrete, sourced recommendations to improve Dankclaw
(the AMD Strix Halo inference box). It covers self-hosted agentic tools,
inference runtime tuning, and server/OS/hardware settings. Each item lists a
priority and the exact action.

Dates: researched 2026-08. Hardware is verified on the live machine unless
marked "verify first". Re-verified 2026-08-15/16; corrections marked below.

**Live today (2026-10-18).** The era described below — llama.cpp on the 8060S
iGPU — is retired; what runs on this box now (both endpoints, their engines and
ports) is [[frank-dragoon]]'s and [[frank-zealot]]'s pages, and both run under their
repo units today — no drift open as of 2026-10-19 (`frank-zealot` active since
2026-10-18 21:06, the deployer's 2-slot reslice; `frank-dragoon` active). The OS facts
are this page's: `iommu=pt` since 2026-10-18 ([[dankclaw-iommu-translation-off]]), and
the `ttm` ceiling and NVMe scheduler rows below were re-checked 2026-10-18 and
still hold. Every benchmark number below is a measurement of the era it was
taken in.

## Corrections (re-verified 2026-08-15/16)

- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
  wins prefill and Vulkan wins decode, and the decode gap widens with context:
  the 64k A/B is  §4, the 262k A/B is
   §High-Context Backend Comparison. The "HIP
  out-performs Vulkan for decode" claim did not hold on the 8060S.
- **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
  does not exist on this machine (checked 2026-08-15). GPU DPM is already
  `performance`, forced level `auto`. This item is N/A.
- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
  `frank-zealot` runs now is its own Current shape — this row is not it.
- **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
  (ngram-mod) with draft-mtp first — timeline on ; the
  spec setting each endpoint runs today is on its own page.
- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
  once on [[Dankclaw System Specs]] (Memory).

## Verified Memory Picture (measured 2026-08-11)

| Figure | Value | Source |
| ------ | ----- | ------ |
| OS physical RAM | 30.5 GiB | `/proc/meminfo` |
| ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` under the then firmware VGM carve |
| ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |

That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
"dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
recorded in  §Why Vulkan + GTT; the 76.2 GiB row is an
era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`
the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
2026-10-18.

This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
limits documented in [[frank-dragoon]].

## 1. Self-Hosted Agentic Tools

All of these connect to the local OpenAI-compatible endpoint on
`:7770` (which engine owns that port is [[frank-dragoon]]'s to say), and the
address must be the IPv4 literal: `curl http://localhost:7770/v1/models` returns
nothing while `http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18
— the rootless podman publish is IPv4-only). None need NVIDIA.

| Tool | Use | Endpoint config | Priority |
| ---- | --- | --------------- | -------- |
| Open WebUI | Chat + agent harness, plugins, RAG, tool calling | Add OpenAI-compatible provider -> `http://localhost:7770/v1` | High |
| Cline (VS Code) | IDE coding agent with MCP | Provider "OpenAI Compatible", `API_URL=http://localhost:7770/v1` | High |
| Aider (CLI) | Terminal pair-programming agent | `OPENAI_API_BASE=http://localhost:7770/v1` | Medium |
| Continue.dev (VS Code) | Open-source coding agent | `config.json` -> `{type:"openai", apiBase:"http://localhost:7770"}` | Medium |
| OpenHands (Agent Canvas) | Self-hosted multi-agent coding platform | LiteLLM custom model + Base URL to the endpoint | Medium |
| AnythingLLM | RAG / knowledge chat | Provider "OpenAI (Generic)" -> endpoint | Medium |
| Browser Use | Browser automation agent (Playwright) | `ChatOpenAI(base_url="http://localhost:7770/v1")` | Low |
| Qdrant | Vector store for RAG | Standalone, CPU-native | Low |

The pragmatic first installs are Open WebUI, Cline, and Aider. They all work
with the endpoint on 7770 and need no extra GPU.

## 2. Inference Runtime Tuning (llama.cpp on gfx1151)

Verified against that era's running service and llama.cpp source; the era is
retired ([[frank-dragoon]]), so this table is history and a method, not a live
TODO list.

| Change | Recommendation | Priority |
| ------ | -------------- | -------- |
| Backend | **Vulkan0** (2026-08-15, corrected — see Corrections). ROCm only for prefill-heavy workloads. | Done |
| KV cache | `q8_0` K / `q8_0` V in production. Try `q4_0` only if more context is needed. | Keep |
| Flash attention | Leave `--flash-attn auto`. It helps prompt processing, not decode. | Keep |
| Load mode | Start with mmap (default). Enable `--mlock` only if page-fault stutter appears. | Optional |
| Threads | Use `--threads 12 --threads-batch 16 --threads-http 1` for 16 Zen5 cores + 3 slots. Frees cores for the ROCm driver. | Medium |
| DFlash n_max | Depth 8 (block 16) was high; a shallower draft cuts rejection cost at the same throughput. Sweep depth per the A/B protocol in [[DFlash DSpark Speculative Decoding]]. | Verify |
| ngram layering | Layering ngram on the DFlash draft measured slower than the draft alone at 4k — figures on [[DFlash DSpark Speculative Decoding]]. | Keep off |

## 3. Server / OS / Hardware Settings

All require root and a reboot for the modprobe change.

| Setting | Action | Effect | Priority |
| ------- | ------ | ------ | -------- |
| TTM GPU pool | Raised 2026-08-25; the knob, its value and its persistence file are [[Dankclaw System Specs]]'s (Memory row) | Lifted the ~15 GiB kernel-page ceiling so a full 262k KV plus a ~30 GB payload fits one slot | Done |
| Power profile | **N/A on this machine** (2026-08-15; re-checked 2026-10-18: no `power_profile` under `/sys/class/drm/card0/device/` or `card1/device/`); GPU DPM already `performance` | none | ~~Medium~~ |
| NVMe scheduler | Already `none` (`[none] mq-deadline` in `/sys/block/nvme0n1/queue/scheduler`, re-read 2026-10-18) | No change | Done |
| BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
| Firmware/EC | Check GMKtec for a newer EC/fan-curve firmware | Better sustained thermal behavior | Verify |

## 4. Other Enhancements

- Test `--spec-draft-n-max 5` and re-benchmark acceptance; if throughput rises,
  change the env tunable.
- The `vllm-thinkingcap.service` unit this item told you to clean up is gone:
  `systemctl is-enabled vllm-thinkingcap.service` → `not-found` (2026-10-18).
- Profile generation under 3-slot concurrent load before touching boot
  settings, so any tuning is measured against the real workload.

## Sources

- [[frank-dragoon]] — measured backend/spec numbers
-  deploy and update process
- AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
- llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`
  `common/arg.cpp`, `common/common.h`
- NotebookCheck Ryzen AI MAX+ 395 analysis

## See Also

- [[frank-dragoon]]
- [[DFlash DSpark Speculative Decoding]]