docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup
- deployment.md: Boot Policy (2026-08-17) section (disabled units table, re-enable recipes, stale frank-restart-small.sh helper, broken deploy.sh dankserver target); Update Dankserver -> frank-pp-worker; condensed retired 35B history block and 2026-08-13 incident - AGENTS.md: boot-policy note, target map fix (Q6_K, split 0.68), deploy.sh dankserver warning, obsolete quirk entries condensed - README.md: rewrite active services/models tables (PP head + worker) - index.md: topology fix (qwythos:9b, Q6_K split 0.68, NPU disabled) - qwen3.8-spec-matrix.md: superseded configs merged into compact history; stale inline env block removed; bundled-head MTP note - FastFlowLM, PP Deployment: boot-policy + enablement notes - System Specs x2, DFlash doc, 35B FP8 doc, Ling docs: stale active claims fixed, retired banners, never-shipped plan stubbed
This commit is contained in:
1 parent
3f88b8e1ed
commit
968066ff23
13 files changed
+190
-246
No files matched your search
@@ -160,9 +160,7 @@ The DFlash draft is not a standalone model. It must pair with its target model.
|
||||
|
||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||
|
||||
## Deployment on Dankserver (NVIDIA RTX 3090)
|
||||
|
||||
The active Dankserver deployment runs **llama.cpp** (`frank-default-small`), not vLLM. It serves the HauhauCS Qwen3.6-35B-A3B uncensored `Q6_K_P` on port 7770 using **ngram-mod** speculative decoding (n_min=48 / n_max=64 / n_match=24) with 3 slots at 200000 context each and MoE experts on CPU. See [[deployment]] and `scripts/serve-llama-frank-small.sh`.
|
||||
The Dankserver llama.cpp deployment (`frank-default-small`, Qwen3.6-35B-A3B) is retired (2026-08-16) and its unit disabled (2026-08-17). The RTX 3090 now runs the PP worker for the Dankclaw head (see [[Qwen3.8 PP Deployment]]). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
|
||||
|
||||
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-default-small` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
|
||||
|
||||
@@ -197,15 +195,7 @@ docker run --rm --name vllm-qwen36-dflash \
|
||||
|
||||
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
|
||||
|
||||
Dankclaw now runs llama.cpp as its runtime for `frank-default-large` (port 7770), serving the unsloth Qwen3.8-27B `UD-Q8_K_XL` model with **no dedicated draft** on the **Vulkan0** backend.
|
||||
|
||||
Current facts (2026-08-15):
|
||||
|
||||
- Main model: `Qwen3.8-27B-UD-Q8_K_XL.gguf` (bundled MTP, unused).
|
||||
- Spec: llama.cpp **`--spec-default`** preset = ngram-mod (n_match 24 / n_min 48 / n_max 64). No draft model.
|
||||
- **Backend: Vulkan0** — at 262144 ctx (~200k prompt) Vulkan decodes 5.68 t/s vs ROCm 3.83 t/s (48% faster); ROCm wins prefill 119 vs 106 t/s. See [[qwen3.8-spec-matrix]] for the bench table.
|
||||
- **PARALLEL=2** — passes the 2x32k parallel stream test.
|
||||
- KV `q8_0` K / `q8_0` V; context 2 x 262144; batch 8192 / ubatch 256.
|
||||
`frank-default-large` on Dankclaw runs the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec is the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. See [[qwen3.8-spec-matrix]] and [[Qwen3.8 PP Deployment]] for the current state and the 2026-08-15 Vulkan0/ROCm0 high-context bench.
|
||||
|
||||
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
|
||||
|
||||
|
||||
Reference in new issue
Block a user