docs: LLM index + work log for frank-primary shape D, harness window

index.md intro and mermaid nodes still described 4 x 262144 fp8/bf16 and
frank-small at 2 x 262144; the active-deployment list had no frank-primary
entry at all. Fixed both, plus a malformed GLM list item.

log.md gains the shape-D entry (the SEQS=2 constraint, the arena-lever
measurement, the 2x2 probe table, the measured 1.99x), and the frank-small
entry's stale consequence bullet now points at the resolution.

frank-small runbook: the compaction-fit note is corrected - the 200704 window
moved frank-primary's trigger to 140,492, which fits a 180,224 slot again.
This commit is contained in:
Frank
2026-10-02 12:05:39 -07:00
parent 295d5784d3
commit 656012a0fa
4 changed files with 115 additions and 26 deletions
@@ -105,13 +105,13 @@ The per-slot window is the whole price. Two consequences the fleet depends on:
- **A prompt over 180,224 tokens is a hard 400**, so `~/.omp/agent/models.yml`
sets `contextWindow: 180224` for this endpoint — the harness compacts before
it reaches the ceiling instead of discovering it.
- **frank-primary's compaction prompt no longer fits with room to answer.**
Its trigger is 183,501 tokens — already above this slot — and its summary
budget is 32,768 output tokens. On the two-slot shape (262,144) that left
32.8k of output room, which is why that number was chosen; on this shape a
single compaction request must be truncated or split. Nothing in the fleet
routes compaction here automatically, but any runbook that claims frank-small
"fits a compaction request in ONE request" is describing the 262,144 shape.
- **frank-primary's compaction prompt fits inside one slot again.** At
frank-primary's shape-D window (200,704) its compaction trigger is 140,492
tokens (70 %), so one compaction request lands here with 39,732 tokens of
room. At the previous 262,144 window the trigger was 183,501 — *above* this
slot — and the request had to be truncated or split. Nothing in the fleet
routes compaction here automatically; the point is that the 2026-10-02 window
trim restored the fit.
## Client contract
+22 -14
View File
@@ -4,18 +4,19 @@ Documentation for the LLM inference fleet on the two hosts (2026-10-02):
**Dankserver runs the tcclaviger RDNA4 vLLM fork on frank-primary** (the
digest-pinned `tcclaviger/vllm` image driving `podman run` directly, serving
orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both Radeon AI PRO R9700s
on :7770, TP=2 with automatic tiered-expert offload, **4 seqs x 262144 ctx
(the model's full native window)**, fp8 KV (5 boot but leave no runtime
headroom; 6 do not fit), no speculative decode, MNBT 1024, MoE gather 32x64, util 0.98,
reasoning/tool parsers wired (qwen3 / qwen3_xml) with preserve_thinking on,
measured 36.4-39.4 tok/s decode — the 2026-10-01 bf16 no-spec cutover plus
the night ladder; rollback frank-primary-fp8-mtp-20261001), and Dankclaw
on :7770, TP=2 with automatic tiered-expert offload, **shape D since
2026-10-02: 3 slots x 200704 ctx** with the KV pool sized for **~2 full-length
conversations** — measured 398,658 tokens = 1.99x — fp8 KV, no speculative
decode, MNBT 1024, MoE gather 32x64, util 0.98, reasoning/tool parsers wired
(qwen3 / qwen3_xml) with preserve_thinking on. `--max-num-seqs` must stay 3:
SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
and Dankclaw
runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF Q8_0_ROCMFPX_AGENT, a ftype-115
file only that fork can read, with the repo's MTP head drafting 4 tokens, ONE
session at 262,144, f16 KV, vision via the bundled projector)** and
**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q5_K_M
GGUF, 2 x 262,144, asymmetric KV f16 K + q8_0 V, on the RTX 3090)**.
GGUF, 3 x 180,224, asymmetric KV f16 K + q8_0 V, on the RTX 3090)**.
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -31,9 +32,9 @@ graph LR
gfx1201, 32 GiB each]
B2[tcclaviger RDNA4 vLLM fork, digest-pinned
podman run, frank-primary, port 7770]
C[Qwen3.8-Flash-Next FP8 bf16 no-spec
TP=2, expert offload automatic, 4 x 262144 ctx,
fp8 KV, MNBT 1024, gather 32x64, qwen3 parsers]
C[Qwen3.8-Flash-Next FP8, no spec decode
TP=2, expert offload automatic, 3 x 200704 ctx,
fp8 KV pool ~2 full-length, MNBT 1024, gather 32x64, qwen3 parsers]
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -46,7 +47,7 @@ frank-large, port 7770]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
H[MiMo-V2.6 Q5_K_M GGUF
2 x 262144 ctx, f16 K + q8_0 V, flash-attn]
3 x 180224 ctx, f16 K + q8_0 V, flash-attn, mmproj vision]
E2 --> E --> F
E2 --> G --> H
end
@@ -79,10 +80,17 @@ frank-small, port 7771]
THREE slots of 180,224 tokens (pool 540,672) with K f16 + V q8_0 and the
mmproj for vision, ~100 tok/s decode, ~2.7 GiB of the card left free.
The KV arithmetic, the flash-attn dependency, rollback to Q6_K
- [[dankserver-qwen38-fp8-expert-offload-tuning]] — **ACTIVE since
2026-10-02 (shape D)** (root systemd unit, port 7770): the tcclaviger RDNA4
vLLM fork serving orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both
R9700s, TP=2 with automatic expert offload — `--max-model-len 200704` x 3
slots, fp8 KV, pool 3,150,000,000 B/rank (measured 398,658 tokens = 1.99x),
MNBT 1024, gather 32x64, no spec decode, prefetch off. Why `--max-num-seqs`
must stay 3 (SEQS 2 cannot compile this model), why the arena lever is
`max_model_len` rather than the KV pool, and the measured envelope
- [[frank-primary-glm53-flash-w4a16]] — RETIRED 2026-09-30 → 10-01 (the
- failed GLM experiment; frank-primary runs Qwen FP8 again, see the intro):
(2026-09-30, root systemd unit, port 7770): the tcclaviger RDNA4 vLLM fork
(2026-09-30, root systemd unit, port 7770): the tcclaviger RDNA4 vLLM fork
failed GLM experiment; frank-primary runs Qwen FP8 again, see the intro):
the tcclaviger RDNA4 vLLM fork
(digest-pinned `tcclaviger/vllm@sha256:da4029666…`, plain `podman run`,
no shim) serving Intel/GLM-5.3-Flash-W4A16-AutoRound — AutoRound symmetric
int4 group 128, ~170 GiB, 34 shards + 2 extras, 113,074 tensors — on BOTH