docs + guard: single 262k slot state, split-limits analysis, cache incident

- spec matrix + PP doc: production state = 1 slot @ 262144, split 0.53
  (34 blocks), KV 8192/8192, 3090 at 23460/24576. New 'Split limits
  (measured 2026-08-17)' section: 0.50/0.54/0.55 boundary data, the
  KV-per-device rounding rule (round(f x 66)), and why 35 blocks is
  unreachable (9-FA flip at f~0.538).
- PP doc: cache incident writeup - full Dankserver disk -> 0-byte cache
  entries -> corrupt weights -> degenerate output; fix + signature.
- deployment.md + AGENTS.md + index.md: numbers brought current.
- scripts/prune-rpc-cache.sh: cron guard capping the worker cache at
  40 GiB (oldest first) to prevent the disk-full recurrence.
This commit is contained in:
Frank
2026-08-17 01:24:06 -07:00
parent c85ff6c6fd
commit aa1941d592
6 changed files with 132 additions and 45 deletions
+16 -7
View File
@@ -143,22 +143,31 @@ curl -sf http://127.0.0.1:7770/health
The active service on Dankclaw is the llama.cpp **pipeline-parallel
head**. It serves the unsloth Qwen3.8-27B `UD-Q8_K_XL` model under the
alias `frank-default-large` on port 7770. The head splits the tensors
`0.40,0.60` to the `ggml-rpc` worker on Dankserver (RTX 3090, port 50052).
`0.53,0.47` to the `ggml-rpc` worker on Dankserver (RTX 3090, port 50052):
34 of 65 blocks (0-33) on the 3090 — the maximum provably-safe split
(measured 2026-08-17; see the PP doc for the boundary analysis).
Speculative decoding runs **MTP** (bundled head, `--spec-draft-n-max 4`)
with the `--spec-default` ngram-mod fallback (48/64/24). The head GPU side
runs **ROCm0** (no `--device` flag; default device selection puts RPC
first and picks ROCm for the iGPU). **2 parallel slots** at 200704 context
each (pool 401400), KV cache `f16` K / `f16` V, full layer offload
(`--gpu-layers all`, `--fit off`), batch 8192 / ubatch 256, `--direct-io`.
The unit file is `scripts/frank-default-large.service`. It launches through
first and picks ROCm for the iGPU). **1 slot** at 262144 context (pool
262144, `n_ctx_slot = 262144`), KV cache `f16` K / `f16` V (8192 MiB per
device), full layer offload (`--gpu-layers all`, `--fit off`), batch 8192
/ ubatch 256, `--direct-io`. The unit file is
`scripts/frank-default-large.service`. It launches through
`scripts/serve-pp-head.sh`, which reads tunables from
`scripts/frank-default-large.env`.
> Note: 2026-08-17 — single-slot config: `PARALLEL=1`, pool 262144
> (was 2 slots x 200700 at 401400), split 0.53 (was 0.40). The 3090
> landed at 23460/24576 MiB, ~667 MiB free. Same-day incident: a full
> Dankserver disk corrupted the worker RPC tensor cache -> degenerate
> output; fixed by clearing the cache + freeing the disk. Guard:
> `scripts/prune-rpc-cache.sh` (cron, 40 GiB cap).
> Note: 2026-08-16 — the pipeline-parallel split was promoted to
> production as `frank-default-large` (renamed from `frank-pp-head`);
> the standalone single-node service retired. MTP was re-enabled at
> `n_max 4` over the ngram-mod fallback. The pool shrank from 524288 to
> 401400 (2 slots x 200700) to fit the split KV budget. See
> `n_max 4` over the ngram-mod fallback. See
> [[Pipeline Parallelism/Qwen3.8 PP Deployment]] for the runbook.
> Note: 2026-08-15 — the spec went DSpark (crashed under two concurrent 16k