dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: reflect rollback to v3 on frank-default-large (Kairic experiment retired)

Browse Source 
AGENTS.md topology table + runtime sections, docs/index.md deployment
diagram, docs/kairic-edge.md (retired experiment, perf summary), and
docs/ciru-activefpx-promptforge.md (active again) all restored to the
v3 reality. Remaining Kairic references are inside the retired reference
page itself.
This commit is contained in:
Frank committed 2026-08-22 14:26:32 -07:00
1 parent c60156f3c6
commit c45f40a9b3
4 files changed
+64 -40
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+30 -19
View File
@@ -8,17 +8,19 @@ local machines. Topology (2026-08-22):
| Target | Host | Runtime | Service | Model | Port |
|--------|------|---------|---------|-------|------|
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | dank@dankclaw | Kairic Edge CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-IU4-Kairic-Edge, native IU4 lane + MTP4 + ngram-mod, 262K context / 1 slot | 52625 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw;
the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service swapped to the
Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`); v3
files remain on the machine for rollback. On 2026-08-21 a DFlash2 Vulkan
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service briefly swapped to
the Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`), then
was reverted to v3 the same day (operator decision); the Kairic build and
model files remain on the machine. On 2026-08-21 a DFlash2 Vulkan
deployment briefly replaced CIRU, then was reverted the same day for
speed; the DFlash2 files live in `.AGENTTRASH/`. See
docs/kairic-edge.md and docs/ciru-activefpx-promptforge.md. The old scripts
docs/ciru-activefpx-promptforge.md (active) and docs/kairic-edge.md
(retired experiment). The old scripts
live in `old/scripts/` for reference; they describe the retired PP
topology.
@@ -128,21 +130,29 @@ back to a slow path and prefill collapses ~12x (measured 2026-08-17: 95 t/s
vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README
for the base recipe.
### Kairic Edge llama.cpp (Dankclaw)
### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
(PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
`PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
docs/ciru-activefpx-promptforge.md.
### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)
The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge`
from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`
(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`).
It was deployed and reverted to v3 the same day (operator decision).
The runtime routes qualified prompt and verification shapes through the
RDNA 3.5 native IU4 matrix instruction. The build uses the release script
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
ubatch 4096 with no decode change; see docs/kairic-edge.md before
changing the batch profile.
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`.
**Release gap — CK `config.h`:** the pinned CK commit ships only
`config.h.in`; the release script and guide never generate
@@ -152,15 +162,16 @@ docs/kairic-edge.md). The file is untracked in CK git; a `git clean`
removes it. Local rebuilds are byte-identical (`d7625a0c...`).
**Target greedy argmax fast path:** the pinned runner sets
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`; production sets it to 0 because
the fast path rejects non-greedy requests (400) and the card's own chat
sampling is non-greedy. Do not flip it back without re-testing chat.
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`. The fast path rejects non-greedy
requests (400) and the card's own chat sampling is non-greedy, so the
production attempt ran it at 0. It still rejects reasoning budgets unless
compatibility mode is used. Do not re-deploy Kairic without re-testing
chat first.
**Rollback:** the v3 model files remain in
**Rollback target:** the v3 model files remain in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
The v3 runbook is docs/ciru-activefpx-promptforge.md; the rollback
procedure is in docs/kairic-edge.md.
Restored to v3 on 2026-08-22 (commit `c60156f`).
**Env var naming rule still applies:** the HIP runtime reads `TEMP`,
`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`
@@ -215,7 +226,7 @@ graph TD
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw Kairic loads the 26.05 GiB payload in ~12 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw v3 loads the ~36.7 GiB five-file payload (GGUF + FFN/GDN + Output-K8 .pfs + mmproj) in ~18 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).