frank-primary: qwen cutover docs + deployer identity + seam re-pin
- AGENTS.md: fleet table + frank-primary section back to qwen FP8 + MTP4 (37.4-40.6 tok/s, 54% acceptance); GLM runbook moved to the retired index. - runbook: RETIRED verdict — DFlash2 drafter served and drafted but acceptance pinned 1.00-1.38 on every text class including verbatim repetition (mis-wired aux capture, not a weak drafter); 5x slower than baseline at ~1.2 acceptance. - log.md: 2026-10-01 rollback entry. - deploy-remote.sh: LLM_PRIMARY_CONTAINER back to frank-primary-qwen; glm-probe joins the legacy sweep list. - frank_primary_unit.bats: re-pinned to the qwen shape — KV qualification law (7.2 GiB/rank, 3.65x concurrency; 9/16 GiB OOM), fp8 q8/q8 KV PRESENT, MTP4 PRESENT (3.1-3.4x over the 12.11 no-spec baseline), no tool dialect, automatic offload. The old 'MTP buys zero' law carried the prefetch tax and is retired.
This commit is contained in:
@@ -18,6 +18,22 @@ timestamp: 2026-09-30
|
||||
|
||||
# frank-primary — GLM-5.3-Flash W4A16 int4 (tcclaviger, 2x R9700)
|
||||
|
||||
> **RETIRED 2026-10-01.** Cutover reversed to Qwen3.8-Flash-Next-Uncensored-FP8
|
||||
> + MTP4 (commit `2339fae` trio). The DFlash2 drafter experiment is a
|
||||
> **failure**: after five patches the drafter served and drafted (12.8 tok/s
|
||||
> proposed, 65,536 ctx, both TP ranks capturing FULL dflash2 cuda graphs) but
|
||||
> **acceptance pinned at 1.00-1.38 on every text class including verbatim
|
||||
> repetition** — the signature of a mis-wired aux-hidden-state capture, not a
|
||||
> weak drafter. At ~1.2 acceptance the 8-token verification tax makes decode
|
||||
> 5x SLOWER than the 9.9-10.0 no-draft baseline (1.6-1.8 tok/s measured). The
|
||||
> untested alternative capture semantics: post-layer output state
|
||||
> (DeepSeek's `hidden_states + residual`) or the mHC `comb` stream. Payload
|
||||
> (170 GiB) + drafter (2.2 GiB) staged to
|
||||
> `/home/dank/.AGENTTRASH/20261001-glm-purge/` pending purge approval.
|
||||
> KV-cache work that survives: patches 02/03/04 in `scripts/patches/` are
|
||||
> generic unifier/tailpin/grouping fixes, apply-verified against the image.
|
||||
|
||||
|
||||
> **Question this page answers.** Can the fleet serve the 320B-class
|
||||
> `GLM-5.3-Flash` at int4 on two R9700s with expert offload, and how fast is it?
|
||||
> **Answer.** Yes — it serves, with vision, tool calls and 131,072 ctx. It runs
|
||||
|
||||
@@ -1,5 +1,24 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-01
|
||||
- **frank-primary rolled BACK to Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4;
|
||||
GLM/DFlash2 declared failed and purged.** The DFlash2 drafter booted and
|
||||
served (window 6j, 65,536 ctx, `glm53-flash` on :7773, both TP ranks
|
||||
captured FULL dflash2 cuda graphs — five patches deep: EAGLE3 wrapper
|
||||
interface v5, KV unifier MLA-multiple, kpool tailpin, glm5 KV pool with
|
||||
draft+hidden groups, registry method-signal), but **acceptance pinned at
|
||||
1.00-1.38 on every text class INCLUDING verbatim repetition** — a mis-wired
|
||||
aux-hidden-state capture, not a weak drafter. At ~1.2 acceptance the
|
||||
8-token verify tax put decode at 1.6-1.8 tok/s vs the 9.9-10.0 baseline
|
||||
(5x slower). Frank's independently reported 18 tok/s / ~370 prefill did
|
||||
not originate from this engine (APIServer access log: every completion
|
||||
POST was host-local; his workstation appears once, a /v1/models GET).
|
||||
Restored the verified 37.4-40.6 tok/s arm from the `2339fae` trio
|
||||
(200k x 3 seqs, fp8 KV, prefetch off); GLM payload (170 GiB) + drafter
|
||||
(2.2 GiB) + scratch staged to `/home/dank/.AGENTTRASH/20261001-glm-purge/`.
|
||||
KV-cache patches 02/03/04/04b survive in `scripts/patches/` (generic
|
||||
unifier/tailpin/grouping/registry fixes, apply-verified).
|
||||
|
||||
## 2026-09-30
|
||||
- **frank-primary cut over to GLM-5.3-Flash W4A16 int4 — it serves, but NOT at
|
||||
the 35 tok/s target, and the shortfall is measured rather than inferred.**
|
||||
|
||||
Reference in New Issue
Block a user