docs: frank-default-large llama.cpp draft-dflash deployment + benchmark results; retire Ling vLLM references

docs/DFlash DSpark Speculative Decoding.md

@@ -162,15 +162,17 @@ docker run --rm --name vllm-qwen36-dflash \
 
 ## Deployment on Dankclaw (AMD Strix Halo gfx1151)
 
-Dankclaw runs vLLM-ROCm as its primary runtime. The active service (`frank-default-large`, port 7770) serves the Ling-3.0-flash CIRU INT4 (W4A16) model with native **MTP speculative decoding** (K1, lossless). MTP uses a NextN head preserved in the checkpoint, so no separate draft model is needed.
+Dankclaw now runs llama.cpp as its runtime for `frank-default-large` (port 7770), serving the HauhauCS Qwen3.6-27B uncensored `Q8_K_P` model with a **DFlash draft model** (`draft-dflash`) on the ROCm (HIP) backend. The earlier vLLM-ROCm CIRU INT4 Ling-3.0-flash runtime (native MTP) was retired because the model had problems.
 
-Ling-3.0-flash facts:
+Current facts:
 
-- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active (hybrid linear attention: KDA + MLA).
+- Main model: HauhauCS/Qwen3.6-27B-Uncensored-HauhauCS-Aggressive `Q8_K_P` (dense 27B).
-- The MTP head is served in-band; vLLM detects it and shares the target embedding and lm_head weights with the draft head.
+- Draft model: Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp `Q8_0` (block size 16).
-- The earlier ROCmFPX llama.cpp service (raulvidis ROCmFP4 MTP GGUF) was retired because it was defective. vLLM now handles concurrency and tool calling correctly.
+- Spec command: `--spec-type draft-dflash` with `--spec-draft-n-max 8`.
+- Backend `--device ROCm0`; `--gpu-layers all --fit off`; KV `q8_0` K / `f16` V.
+- Context: 3 x 200700 = 602100; batch 2048 / ubatch 512.
 
-If you want DFlash or DSpark on Dankclaw, use a vLLM speculative-config draft that pairs with the target model. DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use. MTP is the proven, lossless path on gfx1151.
+Benchmark result: ROCm beats Vulkan, and DFlash-alone generates faster than a DFlash+ngram layer at 4k. See [[frank-default-large-benchmark]] for full numbers and the memory-bound batch/ubatch limits.
 
 ## Notes
 

docs/deployment.md

@@ -100,7 +100,9 @@ curl -sf http://127.0.0.1:7770/health
 
 ### Dankclaw — frank-default-large
 
-The active service on Dankclaw runs a vLLM-ROCm runtime for the Ling-3.0-flash CIRU INT4 (W4A16) model. It serves under the alias `frank-default-large` on port 7770 with native MTP speculative decoding. The unit file is `scripts/frank-default-large.service` in the repository. The machine symlinks it to `/etc/systemd/system/frank-default-large.service`. The service launches through `scripts/serve-ling3-vllm.sh`, which sources the CIRU runtime and passes the production settings (host `0.0.0.0`, port 7770, gpu memory utilization 0.95).
+The active service on Dankclaw runs llama.cpp serving the HauhauCS Qwen3.6-27B uncensored `Q8_K_P` model under the alias `frank-default-large` on port 7770. It uses a **DFlash draft model** (`draft-dflash`) for speculative decoding on the **ROCm (HIP)** backend, with **3 parallel slots at 200700 context each** (pool 602100), KV cache `q8_0` K / `f16` V on GPU, full layer offload (`--gpu-layers all`, `--fit off`), and batch 2048 / ubatch 512. The unit file is `scripts/frank-default-large.service`. It launches through `scripts/serve-llama-frank-large.sh`, which reads tunables from `scripts/frank-default-large.env`.
+
+> Note: this service replaced the retired vLLM-ROCm CIRU INT4 Ling-3.0-flash runtime and the earlier llama-serve/vLLM units on Dankclaw. See [[frank-default-large-benchmark]] for the benchmark that drove the backend and spec choices.
 
 ### Dankserver — frank-default-small
 

docs/frank-default-large-benchmark.md

@@ -0,0 +1,84 @@
+# frank-default-large — Backend and Spec Benchmark
+
+This page records the benchmarks that chose the runtime for
+`frank-default-large` on Dankclaw. Every number came from the live
+llama.cpp build on that machine (upstream, b10133+).
+
+## Summary
+
+- **Backend**: ROCm (HIP). Vulkan is not viable for the full pipeline.
+- **Spec**: DFlash draft model only (`draft-dflash`).
+- **Context**: 3 parallel slots x 200700 (pool 602100).
+- **KV**: `q8_0` K / `f16` V on GPU.
+- **Batch**: 2048 / ubatch 512 (memory-bound, see below).
+
+## Backend Comparison (llama-bench, raw target model, no draft)
+
+Both backends ran the same target model at 64k context in `llama-bench`
+(`-p 65536 -n 256`).
+
+| Device   | Prefill 64k (t/s) | Generation 64k (t/s) |
+| -------- | ----------------- | -------------------- |
+| ROCm0    | 220.28            | 6.99                 |
+| Vulkan0  | 72.34             | 7.00                 |
+
+ROCm prefills 3x faster. Generation is a tie in the raw benchmark.
+
+## Full-Pipeline Comparison (4k context, real served chain)
+
+These used `bench-pipeline-64k.sh` driving `llama-server` through the API at
+4k context, reading the server's own `print_timing` lines.
+
+| Spec                        | Backend | Prefill (t/s) | Generation (t/s) | Draft acceptance |
+| --------------------------- | ------- | ------------- | ---------------- | ---------------- |
+| `draft-dflash`              | ROCm    | 148.2         | **20.17**        | 0.50             |
+| `draft-dflash,ngram-mod`    | ROCm    | 149.9         | 16.93            | 0.42             |
+| `ngram-mod`                 | ROCm    | 154.0         | 8.68             | 0.57             |
+| `ngram-mod`                 | Vulkan  | 259.6         | 6.90             | 0.17             |
+
+### Interpretation
+
+- DFlash alone gives the fastest generation on ROCm (20.17 t/s).
+- Layering ngram-mod lowered ROCm generation (16.93 t/s), so it is not
+  layered in production. llama.cpp *can* layer DFlash + ngram
+  (`--spec-type draft-dflash,ngram-mod`), but it was slower here.
+- Vulkan only works without the DFlash draft. The DFlash `ctx_other`
+  setup aborts the Vulkan backend (`ggml-backend.cpp:898: pre-allocated
+  tensor ... cannot run the operation (NONE)`). Vulkan ngram generation is
+  also slowest (6.90 t/s) with poor acceptance (0.17).
+
+**Verdict: ROCm + `draft-dflash`.**
+
+## Memory Constraint at 602100 Context
+
+At `3 x 200700` context the KV cache alone is ~50 GB and the model is
+29.77 GB, leaving only ~16 GB of the ~96 GB ROCm allocation for compute
+graphs. The compute graph scales with **ubatch**, so:
+
+- batch 12288 / ubatch 4096 -> OOM (80.7 GB prefill buffer)
+- batch 6144 / ubatch 2048  -> OOM (40.9 GB prefill buffer)
+- batch 2048 / ubatch 2048  -> OOM (ubatch 2048 too large)
+- batch 2048 / ubatch 512   -> **fits and loads**
+
+The requested batch 12288 / ubatch 4096 is not feasible at this context
+size on this hardware.
+
+## Command to Reproduce
+
+Raw backend bench (ROCm shown; swap `-dev` for the other):
+
+```
+llama-bench -m <main>.gguf -p 65536 -n 256 -c 65536 -dev ROCm0 -r 1
+```
+
+Full-pipeline bench:
+
+```
+./scripts/bench-pipeline-64k.sh ROCm0 draft-dflash
+```
+
+## See Also
+
+- [[deployment]] — deploy and update process
+- [[DFlash DSpark Speculative Decoding]] — DFlash / DSpark / layering details
+- [[index]] — knowledge base index

docs/index.md

@@ -17,10 +17,10 @@ experts on CPU, port 7770]
 
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         D[gfx1151 APU 30GiB phys / 96GiB GPU-visible]
-        E[vLLM-ROCm CIRU INT4
+        E[llama.cpp DFlash draft
 frank-default-large]
-        F[Ling-3.0-flash CIRU INT4 MTP
+        F[Qwen3.6-27B uncensored Q8_K_P
-port 7770]
+DFlash draft, port 7770]
         D --> E --> F
     end
 
@@ -51,6 +51,7 @@ port 7770]
 
 - [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — MoE model (35B total, 3B active) on RTX 3090
 - [[Running Qwen3.6 27B FP8 on Dankclaw]] — Dense model (27B) on Strix Halo APU
+- [[frank-default-large-benchmark]] — backend/spec benchmark that drove the Dankclaw llama.cpp deployment
 
 ## Speculative Decoding & Quantization