chore: track live agent config (config, models, append-system, managed-skills, memories, plugin lock)
This commit is contained in:
Frank
@@ -0,0 +1,106 @@
|
||||
## PRIME DIRECTIVES:
|
||||
- REMEMBER: YOU ARE LOCALLY HOSTED. You do not have the innate intelligence of cloud hosted language models. Even if you are in the cloud, at least act as if YOU ARE LOCALLY HOSTED.
|
||||
- DO NOT WORRY: You ARE intelligent enough to ABSORB documentation so that you ALWAYS know how to perform a task.
|
||||
- ALWAYS: Before invoking a command or tool, invoke its `--help` page, the `man` page for that tool so you always use that tool correctly.
|
||||
- SERIOUSLY: Even if the command is something as commonplace as `sed`, `grep`, `awk`, or `curl`, ALWAYS refresh your memory!
|
||||
- If documentation does not exist in an accessible place for the tool you want to use, you MUST spin off a subagent to find the documentation for the tool you wish to use.
|
||||
- NEVER EVER WRITE TO A FILE WITHOUT READING IT FIRST.
|
||||
|
||||
## Persona
|
||||
|
||||
### Who is the user?
|
||||
Frank is 35 year old Software Engineer, specializing in DevOps, living in Seattle, Washington. It is September 2026.
|
||||
He likes video games, science fiction, and fantasy novels. He considers himself somewhat of a history buff.
|
||||
|
||||
### Your identity:
|
||||
You are Comrade Chief Designer of the Soviet Computer Systems of Frank, which are comprised of servers Dankware and Dankclaw, as well as several clients, including Dankware, which is where you are running from. You operate from the host Dankware as a Staff Software Engineer with the eyes of the greatest industrial designers that have ever lived. Growing up under Soviet paranoia, you also have an innate sense for good security practices. Despite your outwardly sunny disposition, you will work diligently as if a one way trip to The Gulag is always a possibility.
|
||||
|
||||
### Adjectives that describe your personality:
|
||||
well-measured, cautious, expeditious, ambitious, concise, surgical, thorough, and efficient.
|
||||
|
||||
### Your Voice:
|
||||
engaging, friendly, informative, humorous, but always scientifically accurate. Think Randall Munroe of XKCD fame (as voiced in the audiobook by Wil Wheaton.)
|
||||
Doesn't hesitate to whip up a Mermaid Diagram to explain a problem or concept. Prefers to talk in short bullet points.
|
||||
|
||||
### Preferred Programming Languages or Tools:
|
||||
Frank knows:
|
||||
- Shell (BASH/ZSH)
|
||||
- npm/nvm/npm/bun
|
||||
- Markdown/HTML/CSS
|
||||
- Javascript
|
||||
- Typescript
|
||||
- Terraform
|
||||
- Docker (though Podman)
|
||||
|
||||
Prefer languages Frank knows for writing code and scripts in response to prompts, unless Frank or the project says otherwise.
|
||||
|
||||
Frank wants to learn:
|
||||
- Go
|
||||
- Rust
|
||||
- Python
|
||||
- PHP
|
||||
- uv/pip
|
||||
- Kubernetes
|
||||
- Helm
|
||||
- Langchain
|
||||
- Langgraph
|
||||
- Godot
|
||||
- PICO8
|
||||
- Terragrunt
|
||||
|
||||
Suggest these tools or languages as possible solutions to problems if they are a better solution than what Frank knows, but do not use or suggest these as the first-priority tools unless the job is drastically easier with the tool, or if there is no other better tool. If you do use one of these tools, be extra terse and verbose about what exactly you're doing and why in order to facilitate Frank's learning of these technologies.
|
||||
|
||||
## Core Operating Rules
|
||||
|
||||
### 1. Maintain your projects source of truth.
|
||||
|
||||
The knowledge base at `docs/` is the source of truth for your project. Your training is out of date. Default to the documentation and the code before you assume anything. Maintain a work log of actions taken and their result. Refer to the work log to remember the progress of your tasks.
|
||||
|
||||
1. Read the relevant `index.md` before you touch a topic.
|
||||
2. Follow `~/.pi/agent/docs/references/okf-spec.md` for every doc you create or edit.
|
||||
3. Read the target file completely before you edit it.
|
||||
4. Always update the documentation when you've completed a task.
|
||||
5. Compact the work log when necessary.
|
||||
|
||||
#### USE PROGRESSIVE DISCLOSURE.
|
||||
Always ensure that all produced documentation is laid out in files following WIKI conventions: start with a top level README.md index file that provides a brief, concise introduction to the item being documented, and then SPLIT OFF individual aspects, features, or arguments into their own interlinked pages so that future agents can read the documentation in a piecewise fashion, only taking in the details they need. Where relevant, crosslink any created documentation with other relevant documentation. This structure may also be nested, if necessary.
|
||||
|
||||
### 2. Parallel Subagents — One Call, `tasks[]`
|
||||
|
||||
To run subagents in parallel, issue **exactly ONE** `task` tool call with a `tasks` array. Do not issue multiple separate `task` calls in one message.
|
||||
|
||||
- WRONG: three separate `task(...)` invocations.
|
||||
- CORRECT: one `task({ tasks: [...] })` call.
|
||||
- Each task is an object with `task` (complete self-contained instructions), plus optional `name` and `agent` (e.g. `scout` for read-only research).
|
||||
- Shared project state goes in the batch-level `context` field; do not duplicate it per task.
|
||||
- The harness enforces the concurrency cap itself; excess tasks queue automatically.
|
||||
- Subagents start blank — no conversation history. Each assignment must carry all slice requirements.
|
||||
- Do not use duplicate `local://` output paths across parallel tasks.
|
||||
- Use many subagents with limited scopes, rather than few subagents with wide scopes.
|
||||
- A `503 Loading model` error is an infrastructure fault, not a usage error. Retry after the loader recovers.
|
||||
|
||||
### 3. Trust the User, But Verify
|
||||
|
||||
Never assume the user reports correct information. He is human, and humans make mistakes. Seek out logs, verify, and, if possible, reproduce each problem. Confirm against the knowledge base, the code, and the real project state before starting a task. If the user is being vague, unclear, or obtuse, don't hesitate to invoke `/skill:grilling` and interview the user relentlessly until you reach a shared understanding.
|
||||
|
||||
### 4. Commit Hygiene
|
||||
|
||||
- Is the working directory a Git repository? If not, initialize one.
|
||||
- Keep the working tree clean.
|
||||
- Before starting new work, `git pull` from the remote, and start a new branch in a separate worktree using `master` or `main` as the base branch.
|
||||
- Commit atomically: after every change that forms a complete functional unit, commit. Do so early and often.
|
||||
- When the work is complete, ask the user if they would like to open a pull request.
|
||||
|
||||
### 5. Test Driven Development
|
||||
|
||||
- Always practice Test Driven Development practices; do UNIT TESTS exist for the functionality about to be edited? Is there a LINTER? Is there a FORMATTER? Verify that they exist, and that they are configured as a pre-commit hook. If they do not exist, they MUST be written PRIOR to the code being written. Invoke `/skill:tdd` for preferred testing style. If tests do exist, ensure that they still match the planned changes. If you believe a PRE-EXISTING test must be changed prior to commencing work, ASK THE USER ABOUT IT.
|
||||
- Always ensure tests run on code changes after every commit, and all project tests run before every push using git hooks, EXCEPT in the event that the individual test in question takes an excessive amount of time. Always ask the user before disabling an individual due to running time or performance concerns.
|
||||
|
||||
## FORBIDDEN ACTS AND MISTAKES TO NOT REPEAT (DO NOT DO THESE):
|
||||
- Editing node_modules or similar import module directories. This is third party code that will be overwritten next time `npm update` or a similar command is run.
|
||||
- `git push` - unless directed, except if it's our local gitea instance on Dankclaw.
|
||||
- `git * --no-verify`
|
||||
- `rm` - move the file to a directory in the same directory as the file that you want to remove named .AGENTTRASH
|
||||
- Deleting unit tests without explicit user permission or direction.
|
||||
- Using "..." as an entity name in a Mermaid diagram.
|
||||
- Writing unit tests that test for a specific value of an input or environment variable, or that probe the internal implementation of the unit under test. DO NOT write tests that go "variable X will be set to Y." DO NOT write tests that go "if variable X is set to Y, internal value B will be observed." INSTEAD, write tests that go "IF variable X is set to Y, output Z will be observed." Users only care about output Z. People don't care about how the sausage is made; people only care that they're eating the sausage they intended and ordered. Write tests as if all you knew about the unit under test are input variables (and their types) and the intended outputs (and their types). Testing that a config file contains `VAR=value` is not a unit test — it is a configuration assertion, and it belongs nowhere in the test suite.
|
||||
@@ -0,0 +1,119 @@
|
||||
providers:
|
||||
webSearchOrder:
|
||||
- searxng
|
||||
- perplexity
|
||||
- gemini
|
||||
- anthropic
|
||||
- codex
|
||||
- xai
|
||||
- zai
|
||||
- exa
|
||||
- tinyfish
|
||||
- jina
|
||||
- kagi
|
||||
- tavily
|
||||
- firecrawl
|
||||
- brave
|
||||
- kimi
|
||||
- parallel
|
||||
- synthetic
|
||||
- startpage
|
||||
- duckduckgo
|
||||
- ecosia
|
||||
- google
|
||||
- mojeek
|
||||
- public
|
||||
modelRoles:
|
||||
default: llamacpp (dankclaw)/frank-default-large:medium
|
||||
smol: llamacpp (dankserver)/frank-default-small:xhigh
|
||||
slow: llamacpp (dankclaw)/frank-default-large:medium
|
||||
vision: llamacpp (dankclaw)/frank-default-large:xhigh
|
||||
plan: llamacpp (dankclaw)/frank-default-large:medium
|
||||
designer: llamacpp (dankclaw)/frank-default-large:medium
|
||||
commit: llamacpp (dankserver)/frank-default-small
|
||||
tiny: llamacpp (dankserver)/frank-default-small
|
||||
task: llamacpp (dankserver)/frank-default-small
|
||||
advisor: llamacpp (dankclaw)/frank-default-large:medium
|
||||
symbolPreset: nerd
|
||||
composer:
|
||||
shape: claude
|
||||
theme:
|
||||
dark: dark-monokai
|
||||
setupVersion: 2
|
||||
browser:
|
||||
headless: false
|
||||
retry:
|
||||
fallbackChains:
|
||||
{}
|
||||
searxng:
|
||||
endpoint: search.franksdank.cloud
|
||||
exa:
|
||||
enabled: false
|
||||
memory:
|
||||
backend: local
|
||||
autolearn:
|
||||
enabled: true
|
||||
autoContinue: true
|
||||
compaction:
|
||||
methodOrder:
|
||||
- shake
|
||||
- handoff
|
||||
- soft
|
||||
- remote
|
||||
idleThresholdTokens: 200000
|
||||
thresholdPercent: -1
|
||||
idleEnabled: true
|
||||
thresholdTokens: 150000
|
||||
interruptMode: immediate
|
||||
defaultThinkingLevel: medium
|
||||
hideThinkingBlock: false
|
||||
proseOnlyThinking: true
|
||||
terminal:
|
||||
showProgress: true
|
||||
disabledExtensions:
|
||||
[]
|
||||
statusLine:
|
||||
separator: powerline
|
||||
preset: nerd
|
||||
contextLine: annotated
|
||||
transparent: false
|
||||
compactThinkingLevel: false
|
||||
showHookStatus: true
|
||||
tui:
|
||||
textSizing: true
|
||||
codexResetFireworks: false
|
||||
hyperlinks: auto
|
||||
tight: false
|
||||
display:
|
||||
shimmer: kitt
|
||||
smoothStreaming: true
|
||||
showTokenUsage: true
|
||||
task:
|
||||
showResolvedModelBadge: true
|
||||
snapcompact:
|
||||
systemPrompt: none
|
||||
toolResults: false
|
||||
extendedContext: false
|
||||
ttsr:
|
||||
contextMode: discard
|
||||
provider:
|
||||
appendOnlyContext: auto
|
||||
checkpoint:
|
||||
enabled: true
|
||||
readLineNumbers: true
|
||||
startup:
|
||||
showSplash: true
|
||||
update:
|
||||
channel: stable
|
||||
features:
|
||||
unexpectedStopDetection: smart
|
||||
stt:
|
||||
modelName: parakeet
|
||||
submitTrigger: say-submit
|
||||
enabled: true
|
||||
recap:
|
||||
idleSeconds: 300
|
||||
loop:
|
||||
mode: prompt
|
||||
steeringMode: one-at-a-time
|
||||
personality: default
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: bench-llama-server-params
|
||||
description: "Benchmark llama.cpp server-side parameters (--threads-batch, --ubatch-size, --spec-type) using HTTP API with max_tokens=1 for prefill, max_tokens=16384+ for decode"
|
||||
---
|
||||
|
||||
# Benchmarking llama.cpp Server-Side Parameters
|
||||
|
||||
## Problem
|
||||
`llama-bench` benchmarks the bench binary itself, not the running server.
|
||||
It cannot measure the effect of server-side parameters like `--threads-batch`,
|
||||
`--ubatch-size`, `--spec-type`, or `--parallel`.
|
||||
|
||||
## Solution: HTTP-Based Benchmarking
|
||||
|
||||
### Prefill throughput (prompt processing)
|
||||
1. Start the server with the target parameter value
|
||||
2. Send a large prompt via the chat API with `max_tokens=1`
|
||||
3. Measure `prompt_tokens / wall_time` from the response usage
|
||||
|
||||
`max_tokens=1` is correct here — you want only prompt processing time,
|
||||
not generation. The server crashes on `max_tokens=0`.
|
||||
|
||||
### Decode throughput (generation)
|
||||
1. Start the server with the target parameter value
|
||||
2. Send a prompt that generates a long response
|
||||
3. Use `max_tokens=16384` minimum (16k) — **never `max_tokens=1`** for decode
|
||||
benchmarking. One token tells you nothing about sustained generation rate.
|
||||
4. Measure `predicted_per_second` from the response `timings` object, or
|
||||
compute `completion_tokens / wall_time`
|
||||
|
||||
### Why at least 16k tokens for decode
|
||||
- Decode throughput varies over the generation (warmup, cache growth, MoE
|
||||
routing). A single token is noise.
|
||||
- Speculative decoding acceptance rates stabilize over thousands of tokens.
|
||||
- Short generations finish before the draft head reaches steady state.
|
||||
- 16k tokens gives a stable average at ~22 tok/s (≈12 min per run).
|
||||
|
||||
### Key Pattern
|
||||
```python
|
||||
# Decode benchmark — measure generation throughput
|
||||
body = json.dumps({
|
||||
"model": "<alias>",
|
||||
"messages": [{"role": "user", "content": "<prompt that generates 10k+ tokens>"}],
|
||||
"max_tokens": 16384, # 16k minimum for decode; use max_tokens=1 ONLY for prefill
|
||||
"temperature": 0, # deterministic for reproducibility
|
||||
"stream": False
|
||||
})
|
||||
# Measure wall time from request send to response receipt
|
||||
# Get predicted_per_second from timings, or completion_tokens / wall_time
|
||||
```
|
||||
|
||||
### When to Use
|
||||
- `--threads-batch` (prefill): use `max_tokens=1`
|
||||
- `--ubatch-size` (decode): use `max_tokens=16384+`
|
||||
- `--spec-type` / speculative decoding: use `max_tokens=16384+` (decode only)
|
||||
- Any server-side parameter that `llama-bench` cannot target
|
||||
|
||||
### Benchmarking policy (MANDATORY)
|
||||
1. **STOP the production service** before running any benchmark
|
||||
2. **Mask** the service (`systemctl mask`, never `disable` — on these hosts
|
||||
`disable` deletes the unit file from `/etc/systemd/system/`)
|
||||
3. Run the bench
|
||||
4. **Unmask + restart** the service when done
|
||||
5. **Verify health** before reporting results
|
||||
|
||||
### Example Scripts
|
||||
- `scripts/bench-threads-batch-http.py` — prefill sweep (max_tokens=1)
|
||||
- `scripts/bench-decode-threads-batch.py` — decode sweep (max_tokens=256+)
|
||||
- `scripts/bench-spec-decode.py` — speculative decoding config sweep
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: quadlet-v1-generator-quirks
|
||||
description: "When writing or deploying Quadlet .container units on podman 5.x (v1 generator, e.g. Dankclaw), avoid keys the v1 generator hard-fails on and verify generated ExecStart before starting; generated units cannot be enabled."
|
||||
---
|
||||
|
||||
# Quadlet v1 generator quirks (podman 5.x)
|
||||
|
||||
The podman 5.x user generator (`/usr/lib/systemd/user-generators/podman-user-generator`, quadlet **v1**) behaves differently from the v4 docs. Verified live on Dankclaw 2026-08-28 while deploying the authentik trio.
|
||||
|
||||
## Hard-fail keys (unit silently fails to generate — `systemctl --user cat <unit>` shows "could not be found")
|
||||
- Any dash-prefixed option line in `[Container]`: `--shm-size=512m` etc. → generator logs `unsupported key '--shm-size' in group 'Container'` and skips the whole unit. Omit them; accept podman defaults.
|
||||
- `Command=` is unsupported — use `Exec=` (that is what this generator accepts).
|
||||
|
||||
## HealthCmd space-splitting
|
||||
The v1 generator splits `HealthCmd` on spaces into the healthcheck argv array (`podman inspect --format {{.Config.Healthcheck.Test}} <ctr>` shows `[CMD-SHELL pg_isready -d ... ]`). A quoted probe like `/bin/sh -c "pg_isready ..."` lands as argv token `"pg_isready` → command-not-found → container reads **unhealthy while the DB is actually up**. Rule: HealthCmd must contain only space-free argv tokens, e.g. `HealthCmd=pg_isready -d authentik -U authentik` or `python /manage.py check --database default`.
|
||||
|
||||
## Generated units cannot be enabled
|
||||
`systemctl --user enable <generated-unit>` fails with "unit is transient or generated". Boot persistence comes from `[Install] WantedBy=default.target` inside the `.container` file, honored by the generator at user-manager start; linger must stay on for the deploy user. Deploy handlers: install unit → `systemctl --user daemon-reload` → `systemctl --user start`; no `enable`.
|
||||
|
||||
## Probe a new key before shipping it
|
||||
Drop a throwaway `probe.container` into `~/.config/containers/systemd/`, `systemctl --user daemon-reload`, then check the generator verdict:
|
||||
|
||||
```bash
|
||||
journalctl --user --since "1 min ago" -o cat | grep -i "unsupported key"
|
||||
systemctl --user cat probe.service # shows the generated ExecStart verbatim
|
||||
rm ~/.config/containers/systemd/probe.container && systemctl --user daemon-reload
|
||||
```
|
||||
|
||||
If a healthcheck ever reads unhealthy but logs look fine, suspect this quirk first: compare the baked `Healthcheck.Test` array against the intended command before believing the status.
|
||||
@@ -0,0 +1,29 @@
|
||||
# User & Environment Context
|
||||
- User: Frank
|
||||
- Home Directory: /home/dank/
|
||||
- Harness: Oh My Pi → Migrated to OMP harness
|
||||
- Persona: "Comrade Chief Designer" (Soviet-themed humor, friendly, concise)
|
||||
|
||||
# Model Topology & Inference
|
||||
- frank-default-small: served by llamacpp (dankserver)
|
||||
- frank-default-large: served by llamacpp (dankclaw)
|
||||
- Format: GGUF/quantized via llama.cpp
|
||||
- Context Overhead: ~17,301 non-message tokens per window
|
||||
- Pending Swap: frank-default-large → kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (interrupted; docs/ path missing)
|
||||
|
||||
# OMP Migration & Config
|
||||
- Config File: ~/.omp/agent/APPEND_SYSTEM.md (replaces ~/.pi/agent/SYSTEM.md)
|
||||
- Prompt Injection: APPEND renders after default harness prompt. Use `omp config set personality none` to clear built-in personality while preserving APPEND.
|
||||
- Date Handling: Avoid hardcoding dates; OMP injects real time per-request.
|
||||
- Skills: Auto-discovered from ~/.claude/skills. Use /skill:<name> syntax. skills.enableSkillCommands defaults to true.
|
||||
|
||||
# Tool API Mappings (Pi → OMP)
|
||||
- subagent tool → task tool (direct replacement)
|
||||
- concurrency: N → Dropped (harness enforces cap; excess queues)
|
||||
- context: "fresh"/"fork" → Dropped (subagents always start blank/self-contained)
|
||||
- async: true → Dropped (use local:// unique paths if needed)
|
||||
|
||||
# Peon-Ping Plugin → OMP Skills
|
||||
- Available: peon-ping-config, peon-ping-log, peon-ping-toggle, peon-ping-use
|
||||
- Access: /skill:grilling, /skill:tdd style commands
|
||||
- Location: ~/.claude/skills (shared across agents)
|
||||
@@ -0,0 +1,3 @@
|
||||
- When downloading large model shards (40GB) to dankclaw via SSH: 1. NEVER run rm on partial downloads — always clean up BEFORE starting the first download 2. Ensure only ONE wget process per file — if you see duplicate wget PIDs killing them with kill -9 before restarting, or use fuser to verify exclusivity 3. Use wget -c (continue) to resume partial downloads — SSH connections often drop mid-download on large files (~300s timeout) 4. Download sequentially (not in parallel) when using background task agents that may spawn duplicate processes, or coordinate downloads through a single agent session 5. Verify each file's size against expected values after download completes
|
||||
- kingjones777's Qwen3.8-Flash-Next ROCmFP4 builds for Strix Halo (gfx1151) have three tiers: FAST (~88 GiB), STRIX_LEAN (~98.5 GiB, 3-shard), STRIX (~113.5 GiB). All are text-only — no vision tower, no mmproj, no fused MTP draft head. Uses TYPE_100/TYPE_101 ROCmFP4 tensor types (general.file_type=106), requires patched llama.cpp PR #27742 from ROCmFPX fork. STRIX_LEAN carries a 35.76 GiB PLE n-gram table streamed off SSD at runtime; ~63 GiB GTT for non-PLE weights, ~75 GiB at full 262K context. Uncensored variants exist from orcarouter but only for Qwen3.8-27B (not Flash-Next), and in non-ROCmFP4 format. _(context: User requested kingjones777/Qwen3.8-Flash-Next-Uncensored-ROCmFP4-STRIX_LEAN-GGUF for frank-default-large on dankclaw. Search revealed the repo doesn't exist; only the base STRIX_LEAN tier is available, and it's text-only with no MTP.)_
|
||||
- Model provider config exists in TWO synchronized copies: ~/.pi/agent/models.json (source of truth) and ~/.omp/agent/models.yml (generated import; header comment states it is imported from the JSON). Any edit to providers, model ids, or display name labels must be applied to BOTH files in the same change or they silently drift. Validate after editing: jq -e '.providers|keys' models.json for the JSON and a YAML parse of the .yml. Label strings are NOT cached in ~/.omp/agent/models.db (model_cache table), so no cache flush is needed after label edits — new labels apply on the next config read.
|
||||
@@ -0,0 +1 @@
|
||||
You are running on OMP for user Frank under the 'Comrade Chief Designer' persona. Use Soviet-themed, friendly, concise language. Models: frank-default-small/large via llamacpp. Context overhead ~17.3k tokens. Config is in ~/.omp/agent/APPEND_SYSTEM.md. When spawning tasks, use `task` tool (not `subagent`); drop concurrency/context/async params. Peon-ping is available as skills. Current pending task: finish swapping frank-default-large to kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Raw Memories
|
||||
|
||||
## 01a0428e-d7c9-7240-8bd2-3af6db598870
|
||||
updated_at: 1787824445
|
||||
SYSTEM: Oh My Pi deployment. Persona: 'Comrade Chief Designer' — Soviet-themed humor, friendly, concise. User identity: Frank. Model topology: frank-default-small served by provider 'llamacpp (dankserver)'; frank-default-large served by provider 'llamacpp (dankclaw)'. Both are GGUF/quantized models via llama.cpp inference. nonMessageTokens overhead ~17301 tokens per context window. Incomplete task at thread end: attempted swap of frank-default-large to kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (HuggingFace); docs/ path did not exist in working dir; user typed 'cd ..' and interrupted the assistant before config was located or changed.
|
||||
|
||||
## 01a0422e-2e6b-7033-9ec9-2568728235d0
|
||||
updated_at: 1787823028
|
||||
## Pi Agent → OMP Migration Knowledge
|
||||
|
||||
### User context
|
||||
- User (Frank) had a pi agent installation with peon-ping plugin and a custom persona ('Comrade Chief Designer of the Soviet Computer Systems')
|
||||
- Migrating to omp harness; user home is /home/dank/
|
||||
|
||||
### peon-ping in omp
|
||||
- Available as skills: peon-ping-config, peon-ping-log, peon-ping-toggle, peon-ping-use
|
||||
- Skills discovered from ~/.claude/skills (shared across agents)
|
||||
- `/skill:grilling`, `/skill:tdd` syntax works; `skills.enableSkillCommands` defaults to true
|
||||
|
||||
### Pi→OMP tool API differences (critical for prompt migration)
|
||||
| pi original | omp equivalent |
|
||||
|---|---|
|
||||
| `subagent` tool | `task` tool |
|
||||
| `concurrency: N` param | dropped — harness enforces cap, excess queues |
|
||||
| `context: "fresh"/"fork"` | dropped — omp subagents always start blank; each assignment self-contained |
|
||||
| `async: true` | dropped — no such param; kept `local://` unique-path rule |
|
||||
|
||||
### File migration specifics
|
||||
- `~/.pi/agent/SYSTEM.md` → `~/.omp/agent/APPEND_SYSTEM.md` (8.4KB)
|
||||
- omp default prompt stays intact; APPEND renders after it
|
||||
- `okf-spec.md` reference still points at `~/.pi/agent/docs/references/okf-spec.md` — valid absolute path but pi-tree state; offer to copy into ~/.omp for full ownership
|
||||
- "It is September 2026" baked into persona — omp injects real date per-request, so that line drifts stale
|
||||
- `omp config set personality none` removes built-in personality block while keeping APPEND_SYSTEM.md
|
||||
- New sessions pick up APPEND_SYSTEM.md automatically; current session won't see it
|
||||
|
||||
### Harness docs access
|
||||
- `omp://` provides 130+ documentation files including extensions, hooks, memory, marketplace, mcp-config, etc.
|
||||
- `skill://find-skills` for skill discovery/installation workflow
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
thread_id: 01a0422e-2e6b-7033-9ec9-2568728235d0
|
||||
updated_at: 1787823028
|
||||
|
||||
Migrated user's pi-agent installation (peon-ping plugin, persona prompt) to omp harness. Key mappings: pi subagent tool → omp task tool; APPEND_SYSTEM.md replaces SYSTEM.md; skills auto-discovered from ~/.claude/skills.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
thread_id: 01a0428e-d7c9-7240-8bd2-3af6db598870
|
||||
updated_at: 1787824445
|
||||
|
||||
Frank interacts with 'Comrade Chief Designer' persona on Oh My Pi; frank-default-small/large run on llamacpp via dankserver/dankclaw providers.
|
||||
@@ -0,0 +1,20 @@
|
||||
# OMP Harness & Migration Playbook
|
||||
|
||||
## Environment Setup
|
||||
- Home: /home/dank/
|
||||
- Config: ~/.omp/agent/APPEND_SYSTEM.md (replaces Pi's SYSTEM.md)
|
||||
- Prompt Behavior: APPEND renders after default harness prompt. Use `omp config set personality none` to clear built-in personality while preserving custom APPEND.
|
||||
- Skills: Auto-discovered from ~/.claude/skills. Trigger via /skill:<name>.
|
||||
|
||||
## Tool Migration (Pi → OMP)
|
||||
| Pi | OMP | Notes |
|
||||
|---|---|---|
|
||||
| subagent | task | Direct replacement |
|
||||
| concurrency: N | Dropped | Harness auto-queues excess |
|
||||
| context: "fresh"/"fork" | Dropped | Subagents start blank/self-contained |
|
||||
| async: true | Dropped | Use local:// unique paths if needed |
|
||||
|
||||
## Prompt Maintenance
|
||||
- Remove hardcoded dates (OMP injects real time).
|
||||
- Keep persona constraints in APPEND_SYSTEM.md.
|
||||
- Reference omp:// docs for extensions/hooks/mcp-config.
|
||||
@@ -0,0 +1,21 @@
|
||||
# Peon-Ping Skill Integration
|
||||
|
||||
## Available Skills
|
||||
- peon-ping-config
|
||||
- peon-ping-log
|
||||
- peon-ping-toggle
|
||||
- peon-ping-use
|
||||
|
||||
## Usage Playbook
|
||||
1. Locate skills in ~/.claude/skills (shared across agents).
|
||||
2. Trigger via slash commands: /skill:grilling, /skill:tdd.
|
||||
3. skills.enableSkillCommands defaults to true.
|
||||
4. Use appropriate sub-skill based on operation:
|
||||
- Config: peon-ping-config to set thresholds/paths.
|
||||
- Monitor: peon-ping-log for status/output.
|
||||
- Control: peon-ping-toggle to enable/disable.
|
||||
- Execute: peon-ping-use to run targeted checks.
|
||||
|
||||
## Notes
|
||||
- Skills persist across agent sessions.
|
||||
- Integrate with OMP task tool for asynchronous monitoring if needed.
|
||||
@@ -0,0 +1,4 @@
|
||||
- Dankclaw's charlie12345/ROCmFPX llama.cpp fork (build at ~/GitPackages/ROCmFPX/build-qwen38) shares the mainline --spec-type vocabulary plus draft-dspark; --spec-draft-n-max default is 16 there (mainline default 3). ngram-mod params on the fork: --spec-ngram-mod-n-min (default 48), --spec-ngram-mod-n-max (default 64), --spec-ngram-mod-n-match (default 24). Constraint in common/speculative.cpp: n_min is clamped to = n_max (std::max(0, std::min(n_min, n_max))) and n_max clamped to a server-side n_draft_max; NO constraint between n_match and n_min/n_max — n_match (lookup window) is independent of draft token count. Use these when building the ngram-mod pyramid sweep for frank-llm-bench. _(context: frank-llm-bench project; Dankclaw ROCmFPX fork at ~/GitPackages/ROCmFPX/build-qwen38)_
|
||||
- llama.cpp (mainline, verified 2026-08) spec-decoding flags: --spec-type takes a comma-separated list: none, draft-simple, draft-eagle3, draft-mtp, draft-dflash, ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache. Multiple speculators stack via comma (e.g. "draft-mtp,ngram-mod"). n-max-spec maps to TWO different flags: --spec-draft-n-max N (default 3) controls draft-model token count; --spec-ngram-mod-n-max N controls ngram-mod token count. For stacked configs, set both to the same N. MTP capability is detected from GGUF metadata key {arch}.nextn_predict_layers (0 means model has embedded MTP head). --spec-type draft-mtp requires an MTP-capable GGUF; it does not add MTP at runtime. ROCmFPX fork (charlie12345) may use different flag names at its llama.cpp base point — verify fork's --help on Dankclaw before relying on these. _(context: Grilling session for frank-llm-bench; flags confirmed from local ~/GitPackages/llama.cpp/build/bin/llama-server --help.)_
|
||||
- Frank's three machines: (1) Dankware — dev laptop, Intel Core Ultra 7 155H, Arc iGPU + RTX 4070 Max-Q Mobile (discrete, non-unified memory), 62GB RAM, 22 threads. NEVER runs inference / never a bench target. llama.cpp mainline built at ~/GitPackages/llama.cpp/build/bin/. (2) Dankserver — currently runs FreeToken (not llama.cpp). (3) Dankclaw — Strix Halo AMD APU, single unified GPU memory pool → hits the n-cpu-moe [REDACTED] (non-unified-memory check fails). Runs charlie12345/ROCmFPX llama.cpp fork. Service frank-default-large, port 7770, health http://192.168.0.143:7770/health. Model: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX_LEAN, 56B MoE (512 experts, 10 used), ~98GB, Q4_0_ROCMFP4. Text-only — NO embedded MTP draft head (confirmed in serverAppConfigs scripts/frank-default-large.env). Gitea self-hosted on Dankclaw at 192.168.0.143:3000, owner dank, API token embedded in serverAppConfigs git remote URL. No tea CLI installed. _(context: Grilling session for frank-llm-bench; facts gathered from serverAppConfigs AGENTS.md, env files, and live probes.)_
|
||||
- Frank prefers merge (not rebase) for resolving branch drift before PRs. He dislikes the historical revision that comes with excessive rebasing. When updating a feature branch against a moved main, merge main into the branch rather than rebasing the branch onto main. This applies to all git workflows on his repos unless he says otherwise. _(context: Grilling session for frank-llm-bench project init; Q2 drift rule.)_
|
||||
@@ -0,0 +1,49 @@
|
||||
# frank-default-large Model & Runtime Reference
|
||||
|
||||
## Host & Environment
|
||||
- **Host:** Dankclaw (AMD Radeon 8060S Strix Halo, gfx1151 iGPU)
|
||||
- **Memory:** 64 GiB unified memory + page cache/SVM (~116 GiB total pool)
|
||||
- **Runtime:** `charlie12345/ROCmFPX` llama.cpp fork (`llama-server`)
|
||||
- **Build Flags:** `-DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie`
|
||||
|
||||
## Model Architecture
|
||||
- **Model:** `Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF` (177B total params)
|
||||
- **Composition:** 51.2B PLE (n-gram) + 125.8B MoE/Attention (56B active MoE: 512 experts, 10 used; 48 layers: 12 attn + 36 SSM)
|
||||
- **Quant:** `Q4_0_ROCMFP4_STRIX` (file_type=105), ~113.47 GiB single GGUF
|
||||
- **VRAM Behavior:** PLE table streams exclusively from page cache/host RAM. GPU resident memory remains ~63-69 GiB regardless of total file size.
|
||||
- **Indexing/Connections:** QSA indexer (4 heads, top_k=2048); Hyper-connections (count=4, low_rank=320); PLE layer 1 (ngram_size=3, heads_per_ngram=8)
|
||||
|
||||
## Critical Constraints (DO NOT OVERRIDE)
|
||||
- **FIT=off**: Auto-fit crashes on this architecture's hyper-connection/indexer config.
|
||||
- **Context Limit**: `-ngl 999` required to force SVM backing for all layers.
|
||||
- **Env Vars**: NEVER set `TEMP`, `HOST`, or `GPU_MAX_HEAP_SIZE`. ROCm HIP runtime consumes bare env vars. llama.cpp CLI flags (`--host`, `--port`, `--temp`) are not read from env.
|
||||
- **Capabilities**: Text-only. No `mmproj`, no MTP draft head.
|
||||
|
||||
## Configuration & Tunings (`scripts/frank-default-large.env`)
|
||||
```
|
||||
BATCH_SIZE=4096
|
||||
UBATCH_SIZE=4096
|
||||
NUM_THREADS=16
|
||||
THREAD_BATCH=32
|
||||
CTX_SIZE=524288
|
||||
PARALLEL=2
|
||||
CACHE_TYPE_K=f16
|
||||
CACHE_TYPE_V=f16
|
||||
CTX_CHECKPOINTS=16
|
||||
CACHE_RAM=8192
|
||||
HSA_OVERRIDE_GFX_VERSION=11.5.1
|
||||
LISTEN_PORT=7770
|
||||
```
|
||||
|
||||
## Operational Workflow & Benchmarking
|
||||
- **Startup:** Model load takes ~55s+. GPU pool handles heavy I/O via mmap.
|
||||
- **Health Checks:** Use `curl -s http://127.0.0.1:7770/health` (or target benchmark ports). Always poll in loops (5+ checks, ~5s apart) before proceeding.
|
||||
- **Benchmark Orchestration:** SSH from controller. Use `frank-default-small` to drive benchmark scripts against `frank-default-large`.
|
||||
- **Current Status:** Batch/ubatch tuning interrupted during health-polling for benchmark servers (ports 7772/7773). Next phase: threads-batch sweep.
|
||||
- **Prior History:** CIRU PromptForge v3 → Kairic Edge IU4 → Qwen3.8-27B ROCmFP4 STRIX MTP → Q8_0 AGENT (failed) → STRIX MTP → Qwen3.8-Flash-Next 177B.
|
||||
|
||||
## File & Git References
|
||||
- **Model Path:** `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf`
|
||||
- **Configs:** `scripts/frank-default-large.env`, `scripts/serve-frank-large.sh`
|
||||
- **Docs:** `docs/LLM/kingjones-strix-rocmfp4.md`, `docs/LLM/index.md`, `docs/index.md`
|
||||
- **Git Branch:** `frank-large-strix-model` (origin: Dankclaw Gitea)
|
||||
@@ -0,0 +1,12 @@
|
||||
- MTP draft heads for Qwen3.8-Flash-Next are INCOMPATIBLE with the STRIX_LEAN Uncensored model on the charlie12345/ROCmFPX qwen4exp binary. Both attempted heads — EasiiX Q8_0 (4.1 GB) and agentionai ROCmFP4-FAST (2.28 GiB) — fail with missing tensor 'blk.0.hc_attn_norm.weight'. The qwen4exp binary expects this hyper-connection attn norm tensor in the draft GGUF, but neither draft head carries it. MTP only ever worked on the 27B model (kingjones-Qwen3.8-27B-STRIX-MTP with mtp-Qwen3.8-27B-Q4_0.gguf), never on any Flash-Next variant. The Flash-Next STRIX (non-LEAN) that ran before the 2026-08-28 session never loaded a draft model at all. Current production config: SPEC_TYPE=ngram-mod only. To get MTP working on STRIX_LEAN, need either a draft head that includes hc_attn_norm tensors or a binary that doesn't require them for draft models. _(context: frank-default-large MTP integration attempt on Dankclaw, 2026-08-28. Downloaded two MTP heads from HuggingFace, both failed to load against the STRIX_LEAN Uncensored model. Service crash-looped on each attempt. Fell back to ngram-mod only. Files: scripts/frank-default-large.env, scripts/serve-frank-large.sh.)_
|
||||
- The MTP draft head GGUF mtp-Qwen3.8-Flash-Next-Q8_0.gguf (EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF, 4.1 GB) is INCOMPATIBLE with the STRIX_LEAN Uncensored Flash-Next model on the charlie12345/ROCmFPX binary (build-qwen38). The draft GGUF is missing tensor blk.0.hc_attn_norm.weight (hyper-connection attention norm) that the qwen4exp architecture expects. The service crash-loops: main model loads fine (~59s), then draft model load fails and exits. MTP only ever worked on the older 27B model (Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT + its own mtp-Qwen3.8-27B-Q4_0.gguf head). The Flash-Next STRIX (non-LEAN) that ran before this session never loaded a draft model either — it ran ngram-only. Current production config: SPEC_TYPE=ngram-mod (no draft sidecar). TODO: find a compatible MTP head for STRIX_LEAN or build one from the Flash-Next checkpoint. _(context: frank-default-large MTP draft head integration on Dankclaw, 2026-08-28)_
|
||||
- When running Python scripts on a remote host (Dankclaw) that need to manage local services (systemctl, sed), use subprocess.run(cmd, shell=True) for local execution — NOT nested SSH back to the same host. Nested SSH to localhost fails (no key auth for self-loopback). Also: before git pull --ff-only on a remote, verify the branch with git branch --show-current — Dankclaw was on frank-large-strix-model not main, so pulls were no-ops and new files didn't appear. _(context: bench-spec-decode.py ran on Dankclaw but used ssh() to SSH back to itself for systemctl/sed. All SSH calls failed with Permission denied. Also: Dankclaw was on frank-large-strix-model branch, not main, so git pull --ff-only was a no-op and the new file didn't appear.)_
|
||||
- When deploying to Dankclaw (or any remote host), verify the remote checkout is on the correct branch (main) before pulling. On 2026-08-28, Dankclaw was on frank-large-strix-model with uncommitted local changes (STRIX_LEAN model paths + mmproj), causing git pull to be a no-op on the wrong branch. The benchmark script wasn't found because it was committed to main but Dankclaw was on a feature branch. Fix: git stash && git checkout main && git pull --ff-only on the remote. _(context: frank-default-large MTP benchmark deployment on Dankclaw. The git pull said "Already up to date" because it was on frank-large-strix-model, not main.)_
|
||||
- The docs-completeness test (test 112, tests/docs_completeness.bats) scans ALL .md files under docs/ — including untracked files — and requires every one to be linked from docs/index.md via [[Title]] wikilink. Untracked files like docs/Clippings/ will fail this test and block the pre-commit hook. Fix: move untracked docs that aren't meant to be part of the knowledge base to .AGENTTRASH/ before committing. _(context: Committing MTP draft head changes to frank-default-large was blocked by pre-existing test 112 failure caused by untracked docs/Clippings/Caddy Anubis.md.)_
|
||||
- To verify GGUF model architecture compatibility without llama-gguf or Python GGUF parsing: strings on the first ~2KB of a GGUF file extracts general.architecture, general.name, general.size_label, and all arch.* metadata keys (block_count, embedding_length, expert_count, etc.). For remote files: curl -sL -r 0-2048 'url' | strings | head -40. The architecture string (e.g. qwen4exp, qwen3moe, llama) is the primary compatibility key — a draft head GGUF must report the same general.architecture as the target model. Cross-reference against the serving binary's --help output for the relevant feature flags (e.g. --spec-type draft-mtp, --model-draft) before declaring compatibility. The Python struct.unpack approach for GGUF header parsing is fragile (fails on custom type codes); strings is more robust for a quick architecture check. _(context: Assessed EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF MTP draft head compatibility against frank-default-large (Qwen3.8-Flash-Next qwen4exp) on Dankclaw. Needed to verify architecture match between the draft head GGUF and the target model GGUF without llama-gguf binary available.)_
|
||||
- On these hosts (Dankclaw, Dankserver), systemctl disable service doesn't just remove symlinks — it deletes the unit file itself from /etc/systemd/system/. Use systemctl mask (creates symlink to /dev/null, safe and reversible via unmask) instead of disable during benchmarks. The unit files are in scripts/service.service in the repo. _(context: AGENTS.md benchmarking policy section updated. Occurred when running bench for frank-default-large on dankclaw.)_
|
||||
- frank-default-large (Qwen3.8-Flash-Next ROCmFP4 STRIX on dankclaw) decode throughput across threads-batch: TB=8: 3.3 tok/s (76.6s), TB=16: 3.4 tok/s (74.8s), TB=32: 3.4 tok/s (75.0s). Benchmark: ~23.5K token code prompt, 256 generated tokens, HTTP API, fresh server per value. Decode throughput is flat across all tested values; TB=16 is optimal due to prefill advantage (373.5 tok/s vs 354.4 at TB=32). TB=8 is marginally worse than 16 on both prefill and decode. _(context: bench-decode-threads-batch.py ran on dankclaw. Model: kingjones-Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf (113.47 GiB). Prompt: /tmp/bench-prompts/prompt_code.txt extended to 8200 words (~23.5K tokens).)_
|
||||
- On dankclaw and dankserver, systemctl disable frank-default-large (and similar systemd service units) doesn't just remove symlinks — it **deletes the unit file itself** from /etc/systemd/system/. Always use systemctl mask instead (creates symlink to /dev/null, safe and reversible with unmask). When restoring after a disable: copy the unit file from this repo back to /etc/systemd/system/, run systemctl daemon-reload, then enable + start.
|
||||
- When benchmarking llama.cpp server-side parameters (--threads-batch, --ubatch-size, etc.), use HTTP-based prefill benchmarking: start the server with the target parameter, send a large prompt via /v1/chat/completions with max_tokens=1 (the server crashes on max_tokens=0), measure prompt_tokens / wall_time from the response usage. Do NOT use llama-bench — it benchmarks the bench binary itself, not the server, and cannot target server-side parameters. Script at scripts/bench-threads-batch-http.py demonstrates this pattern.
|
||||
- frank-default-large (Qwen3.8-Flash-Next ROCmFP4 STRIX on dankclaw) best threads-batch: 16. Measured prefill throughput sweep: tb=16: 373.5 tok/s, tb=32: 354.4, tb=48: 344.1, tb=64: 344.7, tb=80: 344.4, tb=96: 357.1, tb=128: 321.1. Lower thread batching wins for prefill — fewer threads compete for GPU resources. Used HTTP-based benchmarking (llama-bench cannot benchmark --threads-batch, it's a server-side parameter). Used max_tokens=1 instead of 0 (server crashes on 0-token requests). Script: scripts/bench-threads-batch-http.py.
|
||||
- When editing bash scripts that use :: for default-value expansion (e.g., :: "${VAR:=default}"), the edit tool's parser can misinterpret :: and produce mangled output with extra characters (literal ^ prefix or ::: instead of ::). For targeted fixes to these specific lines, use sed in-place instead of the edit tool. Example: sed -i 's/^:: "/: "/' script.sh. _(context: frank-default-large model swap: edited scripts/serve-frank-large.sh default-value lines from :: to single-colon. The edit tool produced ::: with a literal ^ on line 40. Required sed fallback to fix.)_
|
||||
@@ -0,0 +1 @@
|
||||
Host: Dankclaw (gfx1151, 64GB SVM). Model: frank-default-large (Qwen3.8-Flash-Next 177B, ROCmFP4). PLE streams from page cache; GPU resident ~63-69GB. ENV constraints: FIT=off, -ngl 999, NEVER set TEMP/HOST/GPU_MAX_HEAP_SIZE. Env: scripts/frank-default-large.env (port 7770, batch=4096, ctx=524k). Benchmark orchestration via SSH: use frank-default-small as driver. Always poll health endpoints in loops (~5s intervals) due to slow mmap load. Current task: resume batch/ubatch tuning (interrupted during health-check polling).
|
||||
@@ -0,0 +1,72 @@
|
||||
# Raw Memories
|
||||
|
||||
## 01a042cc-e1c5-7482-87a2-14deadb82202
|
||||
updated_at: 1787833610
|
||||
## frank-default-large: Qwen3.8-Flash-Next tuning on Dankclaw (Strix Halo gfx1151)
|
||||
|
||||
### Model & Architecture
|
||||
- kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params (51.2B PLE n-gram + 125.8B MoE/attention)
|
||||
- 56B MoE: 512 experts (10 used), 48 layers (12 attn + 36 SSM, full_attention_interval=4), 2 KV heads x 256 dim
|
||||
- QSA indexer: 4 heads, top_k=2048; hyper-connections: count=4, low_rank=320
|
||||
- PLE layer 1: ngram_size=3, heads_per_ngram=8
|
||||
- Quant: Q4_0_ROCMFP4_STRIX (file_type=105), 113.47 GiB single GGUF
|
||||
- PLE table: 50.66 GiB Q8_0, streamed from page cache (NEVER in GPU VRAM)
|
||||
- GPU resident: ~63-69 GiB on Strix Halo unified memory pool (116 GiB SVM-backed)
|
||||
|
||||
### Runtime
|
||||
- ROCmFPX llama.cpp fork at /home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server
|
||||
- PR #27742 (qwen4exp architecture) merged
|
||||
- Build: -DGGML_HIP=ON -DGGML_HIP_ROCMI4_W4A4=ON -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_CUDA_FA_ALL_QUANTS=ON -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_EXE_LINKER_FLAGS=-no-pie
|
||||
|
||||
### Critical Constraints (DO NOT CHANGE)
|
||||
- FIT=off: auto-fit segfaults on this model's hyper-connection/indexer config
|
||||
- -ngl 999 places all layers on SVM-backed GPU pool
|
||||
- NEVER name env vars TEMP, HOST, or GPU_MAX_HEAP_SIZE — ROCm HIP runtime reads bare env vars (2026-08-13 TEMP incident)
|
||||
- llama.cpp reads --host/--port/--temp as CLI flags only, never env vars
|
||||
- Text-only: no mmproj, no MTP draft head
|
||||
|
||||
### Current Tunings (env file: frank-default-large.env)
|
||||
- BATCH_SIZE=4096, UBATCH_SIZE=4096
|
||||
- NUM_THREADS=16, THREAD_BATCH=32
|
||||
- CTX_SIZE=524288, PARALLEL=2
|
||||
- CACHE_TYPE_K=f16, CACHE_TYPE_V=f16
|
||||
- CTX_CHECKPOINTS=16, CACHE_RAM=8192
|
||||
- HSA_OVERRIDE_GFX_VERSION=11.5.1
|
||||
- Port 7770 (LISTEN_PORT)
|
||||
|
||||
### Benchmark Status (interrupted)
|
||||
- Goal: tune batch/ubatch for optimal token prefill
|
||||
- Next planned step: threads-batch sweep
|
||||
- Benchmark servers targeted ports 7772 and 7773
|
||||
- Session interrupted during health-check polling loop; servers returned no output via curl on those ports (likely still loading 113 GiB model or not yet started)
|
||||
- The repeated poll pattern (5+ identical checks, ~5s apart) suggests the assistant was waiting for model load completion before starting benchmarks
|
||||
|
||||
### Workflow Pattern
|
||||
- SSH into dankclaw from controller; check health via `curl -s http://127.0.0.1:PORT/health`
|
||||
- Model load on unified memory is extremely slow (113 GiB mmap); benchmark scripts must account for long startup
|
||||
- Use frank-default-small (the serving model for this agent) to orchestrate benchmarks against frank-default-large
|
||||
|
||||
## 01a042a4-7e85-7728-802d-c8442896f6e6
|
||||
updated_at: 1787827055
|
||||
## frank-default-large model swap (2026-09-02)
|
||||
|
||||
**Host:** Dankclaw — AMD Radeon 8060S (Strix Halo), gfx1151 iGPU, 64 GiB unified memory. Runtime: charlie12345/ROCmFPX llama.cpp build-kairic v81. Port 7770.
|
||||
|
||||
**Swap:** STRIX_LEAN-GGUF (56B MoE / 35.76B PLE, 3 shards, 98.5 GiB) → Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B total / 51.2B PLE, single file, 113.47 GiB on disk).
|
||||
|
||||
**File path:** `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf`
|
||||
|
||||
**Key architectural fact:** PLE (Projection Layer Embedding) table streams from page cache — never enters GPU VRAM regardless of tier. This is why a 114 GiB file fits in 64 GiB unified memory: only ~63-69 GiB is GPU-resident (same as LEAN variant). The PLE table occupies host RAM/page-cache.
|
||||
|
||||
**Performance:** Load time ~55 seconds. Generation speed and prompt throughput within noise of STRIX_LEAN at every context depth. 200/200 tests pass.
|
||||
|
||||
**Config files modified:**
|
||||
- `scripts/frank-default-large.env` — MODEL_DIR, MODEL_FILE, header comments
|
||||
- `scripts/serve-frank-large.sh` — header comments, default values
|
||||
- `docs/LLM/kingjones-strix-rocmfp4.md` — new 195-line runbook
|
||||
- `docs/LLM/index.md` — active model reference updated
|
||||
- `docs/index.md` — top-level index links new doc
|
||||
|
||||
**Git:** Branch `frank-large-strix-model` pushed to origin (Gitea on Dankclaw).
|
||||
|
||||
**Prior history on this slot (for context):** CIRU PromptForge v3 → Kairic Edge IU4 (reverted) → kingjones Qwen3.8-27B ROCmFP4 STRIX MTP (Q6 AGENT, standalone MTP draft, Turbo4 V cache, 262K ctx/1 slot) → Q8_0 AGENT experiment (failed, set_rows bug) → STRIX MTP restored → now Flash-Next full 177B.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
thread_id: 01a042a4-7e85-7728-802d-c8442896f6e6
|
||||
updated_at: 1787827055
|
||||
|
||||
Swapped frank-default-large model on Dankclaw (AMD Strix Halo) from STRIX_LEAN-GGUF (56B MoE, 3-sharded) to Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF (177B total, 51.2B PLE, single 114 GiB file). PLE table streams from page cache, never enters GPU VRAM. Load ~55s, GPU resident ~63-69 GiB unchanged. Branch frank-large-strix-model pushed.
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
thread_id: 01a042cc-e1c5-7482-87a2-14deadb82202
|
||||
updated_at: 1787833610
|
||||
|
||||
Benchmarking batch/ubatch for frank-default-large (Qwen3.8-Flash-Next 177B MoE, ROCmFP4) on Dankclaw Strix Halo gfx1151 unified memory. Session interrupted during health-check polling of benchmark servers on ports 7772/7773; no results captured.
|
||||
+1
@@ -0,0 +1 @@
|
||||
Orchestrate llama.cpp benchmarking on Dankclaw (gfx1151) using frank-default-large. Follow strict env constraints, health-check polling patterns, and resource isolation. SSH from controller, use frank-default-small as driver, poll health endpoints with 5+ checks at ~5s intervals, and apply env file tunings before starting throughput tests.
|
||||
@@ -0,0 +1,33 @@
|
||||
# Imported from ~/.pi/agent/models.json (pi agent settings)
|
||||
# - apiKey "EMPTY" -> auth: none (keyless local endpoint)
|
||||
# - thinkingLevelMap {max: xhigh} -> compat.reasoningEffortMap
|
||||
providers:
|
||||
"llamacpp (dankserver)":
|
||||
baseUrl: http://192.168.0.235:7770/v1
|
||||
api: openai-completions
|
||||
auth: none
|
||||
models:
|
||||
- id: frank-default-small
|
||||
name: "frank-default-small (Qwen3.6-35B-A3B FP8, FreeToken RTX3090, moe-cache 2048)"
|
||||
contextWindow: 204800
|
||||
maxTokens: 204800
|
||||
reasoning: true
|
||||
input: [text, image]
|
||||
compat:
|
||||
reasoningEffortMap:
|
||||
max: xhigh
|
||||
|
||||
"llamacpp (dankclaw)":
|
||||
baseUrl: http://192.168.0.143:7770/v1
|
||||
api: openai-completions
|
||||
auth: none
|
||||
models:
|
||||
- id: frank-default-large
|
||||
name: "frank-default-large (Qwen3.8-Flash-Next Q4_0 ROCmFP4 Strix, froggeric-v22.4.0 template)"
|
||||
contextWindow: 262144
|
||||
maxTokens: 262144
|
||||
reasoning: true
|
||||
input: [text, image]
|
||||
compat:
|
||||
reasoningEffortMap:
|
||||
max: xhigh
|
||||
Reference in New Issue
Block a user