AGENTS.md: what can be chained, and what cannot
The fleet now runs draft-dflash,ngram-mod, so the rule needs to be written down
where the next agent will look: draftless types (ngram family) are added to the
spec configuration unconditionally, while draft-mtp/dflash/dspark/eagle3 all
require a draft CONTEXT -- therefore one context-backed drafter plus any number
of ngram types is legal, and two context-backed ones never are ('draft-mtp,draft-dflash'
dies at load with 'model doesn't contain MTP layers'). Plus the operational
trap: ngram-mod prints no startup banner, so a working chain looks unengaged in
the journal.
This commit is contained in:
@@ -298,15 +298,14 @@ len ≈ 3` (MTP engaged); an image request is answered from the projector while
|
||||
the halogen side refuses images with a named reason; 22,362 MiB of the card's
|
||||
24,576 MiB is in use; `n_slots = 1, n_ctx_slot = 131072`.
|
||||
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-small-qwen38-gsq-rco.md`.
|
||||
|
||||
### frank-medium — llama.cpp layer split + DFlash2 (Dankserver, ACTIVE)
|
||||
### frank-medium — llama.cpp layer split, DFlash2 + ngram-mod (Dankserver, ACTIVE)
|
||||
|
||||
Mainline llama.cpp at **`fb27a525`** (a gfx1201 HIP build, `Root=`-less root
|
||||
systemd unit `scripts/frank-medium.service`) serving
|
||||
`ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF` at **UD-Q8_K_XL**
|
||||
(29.29 GiB) with the **z-lab DFlash2** `Q8_0` drafter, in **one process across
|
||||
both R9700s**, `--split-mode layer`, 262144 ctx / 1 slot, text only.
|
||||
(29.29 GiB) with the **z-lab DFlash2** `Q8_0` drafter chained to the draftless
|
||||
**ngram-mod** matcher, in **one process across both R9700s**, `--split-mode
|
||||
layer`, 262144 ctx / 1 slot, text only.
|
||||
|
||||
**Layer split is why both cards now serve ONE model.** Tensor parallelism is
|
||||
impossible (see the PCIe lesson below), and so is vLLM's pipeline parallelism —
|
||||
@@ -469,6 +468,27 @@ linked runbooks.
|
||||
the MAIN model's weights (`shares_model`), so there is no second copy of a
|
||||
12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
|
||||
tensors at all, and pointing `--model-draft` at the file would load it twice.
|
||||
- **What can be chained, and what cannot.** `--spec-type a,b` builds every
|
||||
enabled implementation and chains drafting through them, in a fixed internal
|
||||
order (ngram types first, then the drafters — CLI order is irrelevant). The
|
||||
gate is `ctx_dft`: **draftless** types (the ngram family) are added
|
||||
unconditionally, while `draft-mtp`, `draft-dflash`, `draft-dspark` and
|
||||
`draft-eagle3` all require a draft CONTEXT. So a chain may hold one
|
||||
context-backed drafter plus any number of ngram types — `draft-dflash,ngram-mod`
|
||||
is legal and measured (2.6x the drafter alone on prompt-echo traffic, byte-
|
||||
identical to it on prose at the fleet's 48/64 window) — but
|
||||
`draft-mtp,draft-dflash` dies at load: *"context type MTP requested but model
|
||||
doesn't contain MTP layers"*, because MTP builds its context against whatever
|
||||
model the draft context holds. Two context-backed types cannot share one
|
||||
process, and MTP's head only loads when MTP **is** the draft model.
|
||||
- **ngram-mod prints no startup banner.** The load log shows the drafter's
|
||||
`adding speculative implementation 'draft-dflash'` and nothing at all for the
|
||||
matcher, so a chain can look unengaged while it is working. Identify it
|
||||
behaviourally (`draft_n` > 0 only where text repeats) or by the per-task
|
||||
`draft acceptance = ... mean len = ...` line. It costs no VRAM (a 64 MiB
|
||||
host-RAM hash table) and its window decides which traffic it can see: too
|
||||
short and it proposes on ordinary prose fragments, which costs verification
|
||||
without acceptance.
|
||||
- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
|
||||
`500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
|
||||
then served frank-large: every tool-using agent on that endpoint was broken
|
||||
|
||||
Reference in New Issue
Block a user