docs: reflect rollback to v3 on frank-default-large (Kairic experiment retired)
AGENTS.md topology table + runtime sections, docs/index.md deployment diagram, docs/kairic-edge.md (retired experiment, perf summary), and docs/ciru-activefpx-promptforge.md (active again) all restored to the v3 reality. Remaining Kairic references are inside the retired reference page itself.
This commit is contained in:
1 parent
c60156f3c6
commit
c45f40a9b3
4 files changed
+64
-40
No files matched your search
@@ -8,17 +8,19 @@ local machines. Topology (2026-08-22):
|
|||||||
| Target | Host | Runtime | Service | Model | Port |
|
| Target | Host | Runtime | Service | Model | Port |
|
||||||
|--------|------|---------|---------|-------|------|
|
|--------|------|---------|---------|-------|------|
|
||||||
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
|
| Dankserver | dank@dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
|
||||||
| Dankclaw | dank@dankclaw | Kairic Edge CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-IU4-Kairic-Edge, native IU4 lane + MTP4 + ngram-mod, 262K context / 1 slot | 52625 |
|
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 52625 |
|
||||||
|
|
||||||
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
|
The 2026-08-17 redo retired the pipeline-parallel split and the FastFlowLM
|
||||||
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw;
|
NPU endpoint. The 2026-08-18 swap retired the vLLM service on Dankclaw;
|
||||||
the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on
|
the CIRU ROCmFPX runtime then served the v3 ActiveFPX PromptForge model on
|
||||||
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service swapped to the
|
the Strix Halo iGPU (gfx1151). On 2026-08-22 the service briefly swapped to
|
||||||
Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`); v3
|
the Kairic Edge IU4 runtime (pinned tag `kairic-edge-qwen38-27b-v1`), then
|
||||||
files remain on the machine for rollback. On 2026-08-21 a DFlash2 Vulkan
|
was reverted to v3 the same day (operator decision); the Kairic build and
|
||||||
|
model files remain on the machine. On 2026-08-21 a DFlash2 Vulkan
|
||||||
deployment briefly replaced CIRU, then was reverted the same day for
|
deployment briefly replaced CIRU, then was reverted the same day for
|
||||||
speed; the DFlash2 files live in `.AGENTTRASH/`. See
|
speed; the DFlash2 files live in `.AGENTTRASH/`. See
|
||||||
docs/kairic-edge.md and docs/ciru-activefpx-promptforge.md. The old scripts
|
docs/ciru-activefpx-promptforge.md (active) and docs/kairic-edge.md
|
||||||
|
(retired experiment). The old scripts
|
||||||
live in `old/scripts/` for reference; they describe the retired PP
|
live in `old/scripts/` for reference; they describe the retired PP
|
||||||
topology.
|
topology.
|
||||||
|
|
||||||
@@ -128,21 +130,29 @@ back to a slow path and prefill collapses ~12x (measured 2026-08-17: 95 t/s
|
|||||||
vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README
|
vs 1300+ t/s at an 8K prompt on the 3090). See the HauhauCS HF repo README
|
||||||
for the base recipe.
|
for the base recipe.
|
||||||
|
|
||||||
### Kairic Edge llama.cpp (Dankclaw)
|
### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
|
The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
|
||||||
|
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
|
||||||
|
(PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
|
||||||
|
`PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
|
||||||
|
MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
|
||||||
|
batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
|
||||||
|
budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
|
||||||
|
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
|
||||||
|
docs/ciru-activefpx-promptforge.md.
|
||||||
|
|
||||||
|
### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)
|
||||||
|
|
||||||
The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge`
|
The Kairic Edge llama-server serves `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge`
|
||||||
from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`
|
from `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server`
|
||||||
(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`).
|
(version 79, commit `e97b324`, pinned tag `kairic-edge-qwen38-27b-v1`).
|
||||||
|
It was deployed and reverted to v3 the same day (operator decision).
|
||||||
The runtime routes qualified prompt and verification shapes through the
|
The runtime routes qualified prompt and verification shapes through the
|
||||||
RDNA 3.5 native IU4 matrix instruction. The build uses the release script
|
RDNA 3.5 native IU4 matrix instruction. The build uses the release script
|
||||||
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
|
against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
|
||||||
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
|
venv. The model files (GGUF + FFN/GDN/GDN-Output .pfs, 26.05 GiB) live in
|
||||||
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`. The production profile
|
`/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/`.
|
||||||
adds `--no-mmap` and reasoning on (deepseek format, preserve_thinking) on
|
|
||||||
top of the pinned runner; batch 2048 / ubatch 512 are the runner's
|
|
||||||
qualified sizes. Benchmarks: ubatch 512 prefill is 60-90% faster than
|
|
||||||
ubatch 4096 with no decode change; see docs/kairic-edge.md before
|
|
||||||
changing the batch profile.
|
|
||||||
|
|
||||||
**Release gap — CK `config.h`:** the pinned CK commit ships only
|
**Release gap — CK `config.h`:** the pinned CK commit ships only
|
||||||
`config.h.in`; the release script and guide never generate
|
`config.h.in`; the release script and guide never generate
|
||||||
@@ -152,15 +162,16 @@ docs/kairic-edge.md). The file is untracked in CK git; a `git clean`
|
|||||||
removes it. Local rebuilds are byte-identical (`d7625a0c...`).
|
removes it. Local rebuilds are byte-identical (`d7625a0c...`).
|
||||||
|
|
||||||
**Target greedy argmax fast path:** the pinned runner sets
|
**Target greedy argmax fast path:** the pinned runner sets
|
||||||
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`; production sets it to 0 because
|
`LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1`. The fast path rejects non-greedy
|
||||||
the fast path rejects non-greedy requests (400) and the card's own chat
|
requests (400) and the card's own chat sampling is non-greedy, so the
|
||||||
sampling is non-greedy. Do not flip it back without re-testing chat.
|
production attempt ran it at 0. It still rejects reasoning budgets unless
|
||||||
|
compatibility mode is used. Do not re-deploy Kairic without re-testing
|
||||||
|
chat first.
|
||||||
|
|
||||||
**Rollback:** the v3 model files remain in
|
**Rollback target:** the v3 model files remain in
|
||||||
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in
|
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` and the v3 binary in
|
||||||
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
|
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`.
|
||||||
The v3 runbook is docs/ciru-activefpx-promptforge.md; the rollback
|
Restored to v3 on 2026-08-22 (commit `c60156f`).
|
||||||
procedure is in docs/kairic-edge.md.
|
|
||||||
|
|
||||||
**Env var naming rule still applies:** the HIP runtime reads `TEMP`,
|
**Env var naming rule still applies:** the HIP runtime reads `TEMP`,
|
||||||
`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`
|
`HOST`, and `GPU_MAX_HEAP_SIZE`. The serve script passes `--host`/`--port`
|
||||||
@@ -215,7 +226,7 @@ graph TD
|
|||||||
|
|
||||||
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
|
- `sudo` on both boxes needs a tty. Remote sudo steps fail non-interactively. Use `echo '<password>' | sudo -S <command>`.
|
||||||
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
|
- Never run `pkill -f` with a pattern that appears in your own command line. It kills your ssh session. Use `pkill -x <exact-name>`.
|
||||||
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw Kairic loads the 26.05 GiB payload in ~12 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
|
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM (`--no-mmap`, MoE experts on CPU) in a few minutes cold; Dankclaw v3 loads the ~36.7 GiB five-file payload (GGUF + FFN/GDN + Output-K8 .pfs + mmproj) in ~18 s with a warm page cache, a few minutes cold. Health returns 503 while loading. Do not kill the process. `deploy.sh` waits up to 20 minutes on both targets (see `HEALTH_WAIT`).
|
||||||
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
|
- A client abort and a server release look different in the journal. A client abort logs `W srv stop: cancel task, id_task = N`. A natural stop (EOS or limit) logs only `slot release` with `truncated = 0`. The server never preempts a running generation; when all slots are busy, new tasks defer into a queue. See the 2026-08-13 incident in [docs/deployment.md](docs/deployment.md).
|
||||||
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).
|
- Retired hardware history: the Dankserver 35B cuBLAS crash and the DFlash premature-EOS bug are historical; see [docs/dankserver-vram-growth.md](docs/dankserver-vram-growth.md) and [docs/qwen3.8-spec-matrix.md](docs/qwen3.8-spec-matrix.md). The FastFlowLM NPU endpoint is retired; see [docs/FastFlowLM NPU Inference.md](docs/FastFlowLM%20NPU%20Inference.md).
|
||||||
|
|
||||||
|
|||||||
@@ -9,16 +9,17 @@ tags:
|
|||||||
- gfx1151
|
- gfx1151
|
||||||
- ciru
|
- ciru
|
||||||
- promptforge
|
- promptforge
|
||||||
timestamp: 2026-08-21
|
timestamp: 2026-08-22
|
||||||
---
|
---
|
||||||
|
|
||||||
# CIRU ActiveFPX PromptForge on Dankclaw
|
# CIRU ActiveFPX PromptForge on Dankclaw
|
||||||
|
|
||||||
> **Retired 2026-08-22.** `frank-default-large` now serves the Kairic Edge
|
> **Active 2026-08-22.** `frank-default-large` serves this v3 model again
|
||||||
> IU4 model. See [[kairic-edge]] for the active deployment and the rollback
|
> after a same-day Kairic Edge IU4 experiment was reverted (operator
|
||||||
> procedure. This page is kept as the v3 runbook and rollback reference.
|
> decision). Commit `c60156f` restored the v3 script pair. See
|
||||||
|
> [[kairic-edge]] for the retired experiment reference.
|
||||||
|
|
||||||
`frank-default-large` (port 52625) served
|
`frank-default-large` (port 52625) serves
|
||||||
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
|
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
|
||||||
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
|
ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
|
||||||
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
|
previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
|
||||||
|
|||||||
+12
-11
@@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
Documentation for the LLM inference fleet on two local machines
|
Documentation for the LLM inference fleet on two local machines
|
||||||
(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and
|
(2026-08-22): upstream llama.cpp on Dankserver (frank-default-small) and
|
||||||
Kairic Edge CIRU ROCmFPX llama.cpp on Dankclaw (frank-default-large).
|
CIRU ROCmFPX llama.cpp with the PromptForge v3 runtime on Dankclaw
|
||||||
|
(frank-default-large).
|
||||||
|
|
||||||
## Deployment Topology
|
## Deployment Topology
|
||||||
|
|
||||||
@@ -19,10 +20,10 @@ ngram-mod, 36 MoE experts on CPU, 700K ctx]
|
|||||||
|
|
||||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||||
D[gfx1151 iGPU 64 GiB unified memory]
|
D[gfx1151 iGPU 64 GiB unified memory]
|
||||||
E[Kairic Edge CIRU ROCmFPX llama.cpp
|
E[CIRU ROCmFPX llama.cpp PromptForge v3
|
||||||
frank-default-large, port 52625]
|
frank-default-large, port 52625]
|
||||||
F[jcbtc Qwen3.8-27B-IU4-Kairic-Edge
|
F[jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3
|
||||||
262K ctx / 1 slot, native IU4 lane, MTP4 + ngram-mod]
|
262K ctx / 1 slot, full-W8 views, Output-K8 MTP, BF16 vision]
|
||||||
D --> E --> F
|
D --> E --> F
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -41,19 +42,19 @@ frank-default-large, port 52625]
|
|||||||
|
|
||||||
- [[deployment]] — deploy and update process for both servers
|
- [[deployment]] — deploy and update process for both servers
|
||||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||||
- [[kairic-edge]] — the Kairic Edge IU4 runtime on Dankclaw
|
- [[ciru-activefpx-promptforge]] — the v3 CIRU PromptForge runtime on
|
||||||
(frank-default-large, active since 2026-08-22)
|
Dankclaw (frank-default-large, active; restored 2026-08-22)
|
||||||
- [[ciru-activefpx-promptforge]] — the retired v3 CIRU runtime
|
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
|
||||||
(frank-default-large rollback target)
|
(deployed and reverted 2026-08-22)
|
||||||
- [[Dankserver]] — the upstream llama.cpp box (frank-default-small)
|
- [[Dankserver]] — the upstream llama.cpp box (frank-default-small)
|
||||||
- [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large)
|
- [[Dankclaw]] — the CIRU ROCmFPX box (frank-default-large)
|
||||||
|
|
||||||
## CIRU Reference (Dankclaw)
|
## CIRU Reference (Dankclaw)
|
||||||
|
|
||||||
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
|
|
||||||
identity, serving profile, rollback (active deployment)
|
|
||||||
- [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain,
|
- [[ciru-activefpx-promptforge]] — the v3 model, ROCm 7.15 toolchain,
|
||||||
and build history (retired 2026-08-22, rollback target)
|
and build history (active deployment)
|
||||||
|
- [[kairic-edge]] — Kairic Edge IU4 runtime: build runbook, binary
|
||||||
|
identity, and experiment summary (retired 2026-08-22)
|
||||||
|
|
||||||
## DFlash2 Reference (archived)
|
## DFlash2 Reference (archived)
|
||||||
|
|
||||||
|
|||||||
+16
-5
@@ -15,14 +15,25 @@ timestamp: 2026-08-22
|
|||||||
|
|
||||||
# Kairic Edge IU4 Runtime on Dankclaw
|
# Kairic Edge IU4 Runtime on Dankclaw
|
||||||
|
|
||||||
`frank-default-large` (port 52625) serves
|
> **Retired 2026-08-22.** This experiment was deployed, benchmarks were
|
||||||
|
> collected, and the service reverted to v3 the same day (operator
|
||||||
|
> decision). `frank-default-large` serves v3 again. This page is kept as
|
||||||
|
> the retired experiment's build runbook and reference.
|
||||||
|
|
||||||
|
`frank-default-large` (port 52625) briefly served
|
||||||
`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX
|
`jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` through the Kairic Edge CIRU ROCmFPX
|
||||||
llama.cpp runtime. The runtime routes qualified prompt and multi-token
|
llama.cpp runtime. The runtime routes qualified prompt and multi-token
|
||||||
verification shapes through the RDNA 3.5 native
|
verification shapes through the RDNA 3.5 native
|
||||||
`V_WMMA_I32_16X16X16_IU4` matrix instruction. This deployment replaced the
|
`V_WMMA_I32_16X16X16_IU4` matrix instruction. It was reverted to
|
||||||
v3 CIRU ActiveFPX PromptForge service on 2026-08-22. See
|
[[ciru-activefpx-promptforge]] (v3) on 2026-08-22.
|
||||||
[[ciru-activefpx-promptforge]] for the retired v3 runbook and rollback
|
|
||||||
target.
|
Performance summary of the experiment: the IU4 lane accelerated prefill
|
||||||
|
1.5-2.3x over v3 at all depths, but prose decode was 15-26% slower
|
||||||
|
(inherent to the release: the IU4 lane does not accelerate M1 target
|
||||||
|
decode). Reasoning-off fast-greedy mode was ~8% faster per token than
|
||||||
|
production chat, and roughly 2x faster per turn from halved output volume.
|
||||||
|
The v3-era experience, including the 400 errors from the argmax fast path
|
||||||
|
rejecting non-greedy requests, motivated the revert.
|
||||||
|
|
||||||
## Identity
|
## Identity
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user