dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: reflect rollback to v3 on frank-default-large (Kairic experiment retired)

Browse Source 
AGENTS.md topology table + runtime sections, docs/index.md deployment
diagram, docs/kairic-edge.md (retired experiment, perf summary), and
docs/ciru-activefpx-promptforge.md (active again) all restored to the
v3 reality. Remaining Kairic references are inside the retired reference
page itself.
This commit is contained in:
Frank
2026-08-22 14:26:32 -07:00
parent c60156f3c6
commit c45f40a9b3
4 changed files with 64 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+30 -19
View File
@@ -8,17 +8,19 @@ local machines. Topology (2026-08-22):
| Target | Host | Runtime | Service | Model | Port |
|--------|------|---------|---------|-------|------|
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | dank@dankclaw | Kairic Edge CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-IU4-Kairic-Edge, native IU4 lane + MTP4 + ngram-mod, 262K context / 1 slot | 52625 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw;
the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service swapped to the
Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`); v3
files remain on the machine for rollback. On 2026-08-21 a DFlash2 Vulkan
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service briefly swapped to
the Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`), then
was reverted to v3 the same day (operator decision); the Kairic build and
model files remain on the machine. On 2026-08-21 a DFlash2 Vulkan
deployment briefly replaced CIRU, then was reverted the same day for
speed; the DFlash2 files live in `.AGENTTRASH/`. See
docs/kairic-edge.md and docs/ciru-activefpx-promptforge.md. The old scripts
docs/ciru-activefpx-promptforge.md (active) and docs/kairic-edge.md
(retired experiment). The old scripts
live in `old/scripts/` for reference; they describe the retired PP
topology.
@@ -128,21 +130,29 @@ back to a slow path and prefill collapses ~12x (measured 2026-08-17: 95 t/s
vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README
for the base recipe.
### Kairic Edge llama.cpp (Dankclaw)
### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
(PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
`PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
docs/ciru-activefpx-promptforge.md.
### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)
The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge`
from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`
(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`).
It was deployed and reverted to v3 the same day (operator decision).
The runtime routes qualified prompt and verification shapes through the
RDNA 3.5 native IU4 matrix instruction. The build uses the release script
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
ubatch 4096 with no decode change; see docs/kairic-edge.md before
changing the batch profile.
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`.
**Release gap — CK `config.h`:** the pinned CK commit ships only
`config.h.in`; the release script and guide never generate
@@ -152,15 +162,16 @@ docs/kairic-edge.md). The file is untracked in CK git; a `git clean`
removes it. Local rebuilds are byte-identical (`d7625a0c...`).
**Target greedy argmax fast path:** the pinned runner sets
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`; production sets it to 0 because
the fast path rejects non-greedy requests (400) and the card's own chat
sampling is non-greedy. Do not flip it back without re-testing chat.
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`. The fast path rejects non-greedy
requests (400) and the card's own chat sampling is non-greedy, so the
production attempt ran it at 0. It still rejects reasoning budgets unless
compatibility mode is used. Do not re-deploy Kairic without re-testing
chat first.
**Rollback:** the v3 model files remain in
**Rollback target:** the v3 model files remain in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
The v3 runbook is docs/ciru-activefpx-promptforge.md; the rollback
procedure is in docs/kairic-edge.md.
Restored to v3 on 2026-08-22 (commit `c60156f`).
**Env var naming rule still applies:** the HIP runtime reads `TEMP`,
`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`
@@ -215,7 +226,7 @@ graph TD
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw Kairic loads the 26.05 GiB payload in ~12 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw v3 loads the ~36.7 GiB five-file payload (GGUF + FFN/GDN + Output-K8 .pfs + mmproj) in ~18 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).
docs/ciru-activefpx-promptforge.md
+6 -5
View File
@@ -9,16 +9,17 @@ tags:
- gfx1151
- ciru
- promptforge
timestamp: 2026-08-21
timestamp: 2026-08-22
---
# CIRU ActiveFPX PromptForge on Dankclaw
> **Retired 2026-08-22.** `frank-default-large` now serves the Kairic Edge
> IU4 model. See [[kairic-edge]] for the active deployment and the rollback
> procedure. This page is kept as the v3 runbook and rollback reference.
> **Active 2026-08-22.** `frank-default-large` serves this v3 model again
> after a same-day Kairic Edge IU4 experiment was reverted (operator
> decision). Commit `c60156f` restored the v3 script pair. See
> [[kairic-edge]] for the retired experiment reference.
`frank-default-large` (port 52625) served
`frank-default-large` (port 52625) serves
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
docs/index.md
+12 -11
View File
@@ -2,7 +2,8 @@
Documentation for the LLM inference fleet on two local machines
(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and
Kairic Edge CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large).
CIRU ROCmFPX llama.cpp with the PromptForge v3 runtime on Dankclaw
(frank-default-large).
## Deployment Topology
@@ -19,10 +20,10 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
D[gfx1151 iGPU 64 GiB unified memory]
E[Kairic Edge CIRU ROCmFPX llama.cpp
E[CIRU ROCmFPX llama.cpp PromptForge v3
frank-default-large, port 52625]
F[jcbtc Qwen3.8-27B-IU4-Kairic-Edge
262K ctx / 1 slot, native IU4 lane, MTP4 + ngram-mod]
F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3
262K ctx / 1 slot, full-W8 views, Output-K8 MTP, BF16 vision]
D --> E --> F
end
@@ -41,19 +42,19 @@ frank-default-large, port 52625]
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[kairic-edge]] — the Kairic Edge IU4 runtime on Dankclaw
(frank-default-large, active since 2026-08-22)
- [[ciru-activefpx-promptforge]] — the retired v3 CIRU runtime
(frank-default-large rollback target)
- [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on
Dankclaw (frank-default-large, active; restored 2026-08-22)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
(deployed and reverted 2026-08-22)
- [[Dankserver]] — the upstream llama.cpp box (frank-default-small)
- [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large)
## CIRU Reference (Dankclaw)
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
identity, serving profile, rollback (active deployment)
- [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain,
and build history (retired 2026-08-22, rollback target)
and build history (active deployment)
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
identity, and experiment summary (retired 2026-08-22)
## DFlash2 Reference (archived)
docs/kairic-edge.md
+16 -5
View File
@@ -15,14 +15,25 @@ timestamp: 2026-08-22
# Kairic Edge IU4 Runtime on Dankclaw
`frank-default-large` (port 52625) serves
> **Retired 2026-08-22.** This experiment was deployed, benchmarks were
> collected, and the service reverted to v3 the same day (operator
> decision). `frank-default-large` serves v3 again. This page is kept as
> the retired experiment's build runbook and reference.
`frank-default-large` (port 52625) briefly served
`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX
llama.cpp runtime. The runtime routes qualified prompt and multi-token
verification shapes through the RDNA 3.5 native
`V_WMMA_I32_16X16X16_IU4` matrix instruction. This deployment replaced the
v3 CIRU ActiveFPX PromptForge service on 2026-08-22. See
[[ciru-activefpx-promptforge]] for the retired v3 runbook and rollback
target.
`V_WMMA_I32_16X16X16_IU4` matrix instruction. It was reverted to
[[ciru-activefpx-promptforge]] (v3) on 2026-08-22.
Performance summary of the experiment: the IU4 lane accelerated prefill
1.5-2.3x over v3 at all depths, but prose decode was 15-26% slower
(inherent to the release: the IU4 lane does not accelerate M1 target
decode). Reasoning-off fast-greedy mode was ~8% faster per token than
production chat, and roughly 2x faster per turn from halved output volume.
The v3-era experience, including the 400 errors from the argmax fast path
rejecting non-greedy requests, motivated the revert.
## Identity