diff --git a/AGENTS.md b/AGENTS.md index 6693f99..a887949 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,8 +36,11 @@ then was reverted the same day for speed. On 2026-08-22 the service briefly swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day (operator decision). On 2026-08-25 the service switched to the kingjones Qwen3.8-27B-Uncensored-Q6_0-ROCmFPX_AGENT model (standalone MTP draft, -262144 context, reasoning on); the previous Q8_0 AGENT model was retired after -its `set_rows` crash. Runbooks: +262144 context, reasoning on). On 2026-09-01 both services were swapped: +Dankserver switched from ik_llama.cpp Ornith CRACK Q8_0 to stock llama.cpp +Tiel-Coder-35B (qwen35moe hybrid MoE), and Dankclaw switched from the Q6_0 +AGENT model to the Flash-Next 56B MoE (qwen4exp, PR #27742 merged into +charlie's ROCmFPX fork, --fit off, 128k context). Runbooks: [docs/LLM/index.md](docs/LLM/index.md). Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`. @@ -262,23 +265,43 @@ When you rename a service or move a model to a new host, update ## Runtime & Troubleshooting -### kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT llama.cpp (Dankclaw, ACTIVE) +### Qwen3.8-Flash-Next ROCmFP4 STRIX_LEAN llama.cpp (Dankclaw, ACTIVE) -The large service serves `kingjones/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP` -(public) from `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` with a -charlie12345/ROCmFPX `llama-server` (build `build-promptforge`). It runs a -standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full -262144 context / 1 slot, and reasoning on (Sharp template, budget -1, -`--reasoning-format none`; chat kwargs keep `preserve_thinking` + -`preserve_reasoning`). The Q6_0 main model is 24.86 GB. Env -`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`, and -runbook `docs/LLM/kingjones-q8agent-rocmfpx.md` define the deployment. +The large service serves +`kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN-GGUF` (public, 98.5 GiB) +from `/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN/` +with a charlie12345/ROCmFPX `llama-server` built at +`/home/dank/GitPackages/ROCmFPX/build-qwen38` (PR #27742 qwen4exp architecture +merged). The model is a 56B MoE (512 experts, 10 used), 48 layers (12 attn + +36 SSM, full_attention_interval=4), QSA indexer (4 heads, top_k=2048), +hyper-connections (count=4, low_rank=320), PLE (layer 1). Q4_0_ROCMFP4 quant. +It runs f16 KV cache, 131072 context / 1 slot, `--fit off` (auto-fit segfaults +on qwen4exp hyper-connection config), and reasoning on (Sharp template, budget +-1, `--reasoning-format deepseek`; chat kwargs keep `preserve_thinking` + +`preserve_reasoning`). Env `scripts/frank-default-large.env`, serve +`scripts/serve-frank-large.sh`. -**Serving status (2026-09-01):** restored the previously deployed Q6_0 AGENT -model. The Q8_0 AGENT experiment is documented in +**Serving status (2026-09-01):** deployed the Flash-Next 56B MoE model. The +previous kingjones Q6_0 AGENT model (27B, standalone MTP, 262144 context) is +retired. The Q8_0 AGENT experiment is documented in `docs/LLM/kingjones-q8agent-rocmfpx.md`; it was retired because it crashed in `ggml_compute_forward_set_rows` and did not provide a working service. +### Tiel-Coder-35B-A3B llama.cpp (Dankserver, ACTIVE) + +The small service serves `peculiar-ragdoll/Tiel-Coder-35B-A3B-GGUF` (public, +30 GB, UD-Q6_K_XL quant) from `/home/dank/models/tiel-coder-35b/` with +upstream llama.cpp (latest master, CUDA build with +`-DGGML_CUDA_FA_ALL_QUANTS=ON -DCMAKE_CUDA_ARCHITECTURES=86`). The model is a +qwen35moe hybrid MoE: 40 layers (10 attention + 30 SSM/Mamba, +full_attention_interval=4), 256 experts / 8 used, 2 KV heads × 256 dim. It +runs Q8_0 KV cache, 200700 total context (66900/slot × 3 parallel), `--cpu-moe` +for expert offload, `-ngl 999` (non-expert on GPU), and mmap. Env +`scripts/frank-default-small.env`, serve `scripts/serve-frank-small.sh`. + +**Serving status (2026-09-01):** deployed the Tiel-Coder-35B model. The +previous ik_llama.cpp Ornith CRACK Q8_0 deployment is retired. + ### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25) The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,