docs + guard: single 262k slot state, split-limits analysis, cache incident
- spec matrix + PP doc: production state = 1 slot @ 262144, split 0.53 (34 blocks), KV 8192/8192, 3090 at 23460/24576. New 'Split limits (measured 2026-08-17)' section: 0.50/0.54/0.55 boundary data, the KV-per-device rounding rule (round(f x 66)), and why 35 blocks is unreachable (9-FA flip at f~0.538). - PP doc: cache incident writeup - full Dankserver disk -> 0-byte cache entries -> corrupt weights -> degenerate output; fix + signature. - deployment.md + AGENTS.md + index.md: numbers brought current. - scripts/prune-rpc-cache.sh: cron guard capping the worker cache at 40 GiB (oldest first) to prevent the disk-full recurrence.
This commit is contained in:
1 parent
c85ff6c6fd
commit
aa1941d592
6 files changed
+132
-45
No files matched your search
@@ -5,9 +5,10 @@
|
||||
This project manages deployment scripts for the LLM inference fleet.
|
||||
Dankclaw runs a llama.cpp pipeline-parallel head for Qwen3.8-27B (unsloth
|
||||
UD-Q8_K_XL): MTP draft (bundled head, `--spec-draft-n-max 4`) over the
|
||||
`--spec-default` ngram-mod fallback, 2 parallel slots at 200704 context
|
||||
(pool 401400), tensors split 0.40/0.60 to the Dankserver RTX 3090
|
||||
`ggml-rpc` worker, head GPU side on ROCm0 (no `--device` flag), port 7770.
|
||||
`--spec-default` ngram-mod fallback, 1 slot at 262144 context (pool
|
||||
262144), tensors split 0.53/0.47 (34 of 65 blocks) to the Dankserver RTX
|
||||
3090 `ggml-rpc` worker, head GPU side on ROCm0 (no `--device` flag), port
|
||||
7770.
|
||||
The Dankclaw NPU endpoint (FastFlowLM, `frank-default-small`, port 52625)
|
||||
serves Qwen3.6-35B-A3B as registry tag `qwen3.6-moe:35b-a3b` at 262144 ctx
|
||||
(swapped in 2026-08-16; the old llama.cpp 35B service on Dankserver is
|
||||
|
||||
Reference in new issue
Block a user