docs: reflect rollback to v3 on frank-default-large (Kairic experiment retired)

AGENTS.md topology table + runtime sections, docs/index.md deployment
diagram, docs/kairic-edge.md (retired experiment, perf summary), and
docs/ciru-activefpx-promptforge.md (active again) all restored to the
v3 reality. Remaining Kairic references are inside the retired reference
page itself.
This commit is contained in:
Frank committed 2026-08-22 14:26:32 -07:00
1 parent c60156f3c6
commit c45f40a9b3
4 files changed
+64 -40

No files matched your search

+30 -19
View File
@@ -8,17 +8,19 @@ local machines. Topology (2026-08-22):
| Target | Host | Runtime | Service | Model | Port | | Target | Host | Runtime | Service | Model | Port |
|--------|------|---------|---------|-------|------| |--------|------|---------|---------|-------|------|
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 | | Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | dank@dankclaw | Kairic Edge CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-IU4-Kairic-Edge, native IU4 lane + MTP4 + ngram-mod, 262K context / 1 slot | 52625 | | Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw; NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw;
the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service swapped to the the Strix Halo iGPU (gfx1151). On 2026-08-22 the service briefly swapped to
Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`); v3 the Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`), then
files remain on the machine for rollback. On 2026-08-21 a DFlash2 Vulkan was reverted to v3 the same day (operator decision); the Kairic build and
model files remain on the machine. On 2026-08-21 a DFlash2 Vulkan
deployment briefly replaced CIRU, then was reverted the same day for deployment briefly replaced CIRU, then was reverted the same day for
speed; the DFlash2 files live in `.AGENTTRASH/`. See speed; the DFlash2 files live in `.AGENTTRASH/`. See
docs/kairic-edge.md and docs/ciru-activefpx-promptforge.md. The old scripts docs/ciru-activefpx-promptforge.md (active) and docs/kairic-edge.md
(retired experiment). The old scripts
live in `old/scripts/` for reference; they describe the retired PP live in `old/scripts/` for reference; they describe the retired PP
topology. topology.
@@ -128,21 +130,29 @@ back to a slow path and prefill collapses ~12x (measured 2026-08-17: 95 t/s
vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README
for the base recipe. for the base recipe.
### Kairic Edge llama.cpp (Dankclaw) ### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
(PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
`PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
docs/ciru-activefpx-promptforge.md.
### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)
The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge`
from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server` from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`
(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`). (version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`).
It was deployed and reverted to v3 the same day (operator decision).
The runtime routes qualified prompt and verification shapes through the The runtime routes qualified prompt and verification shapes through the
RDNA 3.5 native IU4 matrix instruction. The build uses the release script RDNA 3.5 native IU4 matrix instruction. The build uses the release script
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile `/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`.
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
ubatch 4096 with no decode change; see docs/kairic-edge.md before
changing the batch profile.
**Release gap — CK `config.h`:** the pinned CK commit ships only **Release gap — CK `config.h`:** the pinned CK commit ships only
`config.h.in`; the release script and guide never generate `config.h.in`; the release script and guide never generate
@@ -152,15 +162,16 @@ docs/kairic-edge.md). The file is untracked in CK git; a `git clean`
removes it. Local rebuilds are byte-identical (`d7625a0c...`). removes it. Local rebuilds are byte-identical (`d7625a0c...`).
**Target greedy argmax fast path:** the pinned runner sets **Target greedy argmax fast path:** the pinned runner sets
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`; production sets it to 0 because `LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`. The fast path rejects non-greedy
the fast path rejects non-greedy requests (400) and the card's own chat requests (400) and the card's own chat sampling is non-greedy, so the
sampling is non-greedy. Do not flip it back without re-testing chat. production attempt ran it at 0. It still rejects reasoning budgets unless
compatibility mode is used. Do not re-deploy Kairic without re-testing
chat first.
**Rollback:** the v3 model files remain in **Rollback target:** the v3 model files remain in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`. `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
The v3 runbook is docs/ciru-activefpx-promptforge.md; the rollback Restored to v3 on 2026-08-22 (commit `c60156f`).
procedure is in docs/kairic-edge.md.
**Env var naming rule still applies:** the HIP runtime reads `TEMP`, **Env var naming rule still applies:** the HIP runtime reads `TEMP`,
`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port` `HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`
@@ -215,7 +226,7 @@ graph TD
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`. - `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`. - Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw Kairic loads the 26.05 GiB payload in ~12 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`). - Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw v3 loads the ~36.7 GiB five-file payload (GGUF + FFN/GDN + Output-K8 .pfs + mmproj) in ~18 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md). - A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md). - Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).
+6 -5
View File
@@ -9,16 +9,17 @@ tags:
- gfx1151 - gfx1151
- ciru - ciru
- promptforge - promptforge
timestamp: 2026-08-21 timestamp: 2026-08-22
--- ---
# CIRU ActiveFPX PromptForge on Dankclaw # CIRU ActiveFPX PromptForge on Dankclaw
> **Retired 2026-08-22.** `frank-default-large` now serves the Kairic Edge > **Active 2026-08-22.** `frank-default-large` serves this v3 model again
> IU4 model. See [[kairic-edge]] for the active deployment and the rollback > after a same-day Kairic Edge IU4 experiment was reverted (operator
> procedure. This page is kept as the v3 runbook and rollback reference. > decision). Commit `c60156f` restored the v3 script pair. See
> [[kairic-edge]] for the retired experiment reference.
`frank-default-large` (port 52625) served `frank-default-large` (port 52625) serves
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21. previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
+12 -11
View File
@@ -2,7 +2,8 @@
Documentation for the LLM inference fleet on two local machines Documentation for the LLM inference fleet on two local machines
(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and (2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and
Kairic Edge CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large). CIRU ROCmFPX llama.cpp with the PromptForge v3 runtime on Dankclaw
(frank-default-large).
## Deployment Topology ## Deployment Topology
@@ -19,10 +20,10 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
D[gfx1151 iGPU 64 GiB unified memory] D[gfx1151 iGPU 64 GiB unified memory]
E[Kairic Edge CIRU ROCmFPX llama.cpp E[CIRU ROCmFPX llama.cpp PromptForge v3
frank-default-large, port 52625] frank-default-large, port 52625]
F[jcbtc Qwen3.8-27B-IU4-Kairic-Edge F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3
262K ctx / 1 slot, native IU4 lane, MTP4 + ngram-mod] 262K ctx / 1 slot, full-W8 views, Output-K8 MTP, BF16 vision]
D --> E --> F D --> E --> F
end end
@@ -41,19 +42,19 @@ frank-default-large, port 52625]
- [[deployment]] — deploy and update process for both servers - [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[kairic-edge]] — the Kairic Edge IU4 runtime on Dankclaw - [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on
(frank-default-large, active since 2026-08-22) Dankclaw (frank-default-large, active; restored 2026-08-22)
- [[ciru-activefpx-promptforge]] — the retired v3 CIRU runtime - [[kairic-edge]] — the retired Kairic Edge IU4 experiment
(frank-default-large rollback target) (deployed and reverted 2026-08-22)
- [[Dankserver]] — the upstream llama.cpp box (frank-default-small) - [[Dankserver]] — the upstream llama.cpp box (frank-default-small)
- [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large) - [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large)
## CIRU Reference (Dankclaw) ## CIRU Reference (Dankclaw)
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
identity, serving profile, rollback (active deployment)
- [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain, - [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain,
and build history (retired 2026-08-22, rollback target) and build history (active deployment)
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
identity, and experiment summary (retired 2026-08-22)
## DFlash2 Reference (archived) ## DFlash2 Reference (archived)
+16 -5
View File
@@ -15,14 +15,25 @@ timestamp: 2026-08-22
# Kairic Edge IU4 Runtime on Dankclaw # Kairic Edge IU4 Runtime on Dankclaw
`frank-default-large` (port 52625) serves > **Retired 2026-08-22.** This experiment was deployed, benchmarks were
> collected, and the service reverted to v3 the same day (operator
> decision). `frank-default-large` serves v3 again. This page is kept as
> the retired experiment's build runbook and reference.
`frank-default-large` (port 52625) briefly served
`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX
llama.cpp runtime. The runtime routes qualified prompt and multi-token llama.cpp runtime. The runtime routes qualified prompt and multi-token
verification shapes through the RDNA 3.5 native verification shapes through the RDNA 3.5 native
`V_WMMA_I32_16X16X16_IU4` matrix instruction. This deployment replaced the `V_WMMA_I32_16X16X16_IU4` matrix instruction. It was reverted to
v3 CIRU ActiveFPX PromptForge service on 2026-08-22. See [[ciru-activefpx-promptforge]] (v3) on 2026-08-22.
[[ciru-activefpx-promptforge]] for the retired v3 runbook and rollback
target. Performance summary of the experiment: the IU4 lane accelerated prefill
1.5-2.3x over v3 at all depths, but prose decode was 15-26% slower
(inherent to the release: the IU4 lane does not accelerate M1 target
decode). Reasoning-off fast-greedy mode was ~8% faster per token than
production chat, and roughly 2x faster per turn from halved output volume.
The v3-era experience, including the 400 errors from the argmax fast path
rejecting non-greedy requests, motivated the revert.
## Identity ## Identity