Configure DeepSeek reasoning and large-model ubatch sweep
This commit is contained in:
@@ -138,7 +138,7 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
|
||||
### Dankclaw — frank-default-large
|
||||
|
||||
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
|
||||
[kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
|
||||
[kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
|
||||
under the alias `frank-default-large` on port 7770. The main model is
|
||||
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
|
||||
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the Q8_0 vision projector is loaded alongside
|
||||
|
||||
@@ -50,6 +50,7 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [kingjones Qwen3.8 Uncensored ROCmFP4 STRIX MTP](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF) — active frank-default-large model on Dankclaw; Q6 AGENT main model with standalone MTP
|
||||
- [[kingjones-strix-mtp]] — the active Q6 AGENT runbook: serving profile, standalone MTP, reasoning + preserve_thinking, 114 GiB TTM/GTT
|
||||
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
|
||||
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
|
||||
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
|
||||
on Dankclaw (retired)
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
type: benchmark plan
|
||||
title: frank-default-large ubatch sweep
|
||||
description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
|
||||
tags:
|
||||
- llama.cpp
|
||||
- rocm
|
||||
- benchmark
|
||||
- ubatch
|
||||
timestamp: 2026-09-01
|
||||
---
|
||||
|
||||
# frank-default-large ubatch sweep
|
||||
|
||||
This benchmark sweeps the active `frank-default-large` model on Dankclaw using
|
||||
scratch ports and an 8K-context workload. The production service is stopped
|
||||
before the sweep and restarted by the benchmark's exit handler.
|
||||
|
||||
## Sweep
|
||||
|
||||
The default points are `256, 512, 768, 1024, 1280, 1536, 2048, 3072, 4096,
|
||||
5120, 6144`. This implements the requested 256-step range through 1536 and
|
||||
1024-step range from 2048 through 6144. Each point runs three 256-token decode
|
||||
requests by default.
|
||||
|
||||
Run on Dankclaw from the repository checkout:
|
||||
|
||||
```bash
|
||||
STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
|
||||
```
|
||||
|
||||
The script uses the active Q6 AGENT model, standalone MTP draft, tracked Sharp
|
||||
template, and `--reasoning-format deepseek`. It reports server timing lines and
|
||||
per-run generation throughput. Run only when the production service can be
|
||||
interrupted; the GPU is shared with live workloads.
|
||||
|
||||
## Results
|
||||
|
||||
Record results below after the sweep, including failures or OOM points.
|
||||
|
||||
| ubatch | prefill | generation | notes |
|
||||
| ---: | ---: | ---: | --- |
|
||||
| 256 | — | — | pending |
|
||||
| 512 | — | — | pending |
|
||||
| 768 | — | — | pending |
|
||||
| 1024 | — | — | pending |
|
||||
| 1280 | — | — | pending |
|
||||
| 1536 | — | — | pending |
|
||||
| 2048 | — | — | pending |
|
||||
| 3072 | — | — | pending |
|
||||
| 4096 | — | — | pending |
|
||||
| 5120 | — | — | pending |
|
||||
| 6144 | — | — | pending |
|
||||
|
||||
## See also
|
||||
|
||||
- [LLM fleet index](index.md)
|
||||
- [Deployment](deployment.md)
|
||||
- [Active kingjones runbook](kingjones-strix-mtp.md)
|
||||
@@ -116,6 +116,7 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
### Benchmarks and Sizing
|
||||
|
||||
- [[frank-default-large-benchmark]] — backend/spec benchmark behind the old head
|
||||
- [[ubatch-benchmark-large]] — active-model ubatch sweep plan and results
|
||||
- [[frank-default-small-benchmark]] — the small-model benchmark suite
|
||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware notes
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]] — Qwen3.6-27B sizing on the iGPU
|
||||
|
||||
Reference in New Issue
Block a user