docs(frank-zealot): the adapter is a payload row, a shape line, and two laws

- Shape table: an Adapter row (bytes, sha, what it edits, and the PTQ1_0
  provenance asymmetry), the fetch line now names both fetchers, the flag list
  carries --lora-scaled.
- One paragraph: the adapter changes behaviour, not the budget equation, and the
  probe that proves it must budget >= 2048 tokens because the thinking channel
  eats a short budget whole and returns empty content with finish_reason=length.
- Eras: the new era's rollback is one env line on a file already loaded.
- Laws: provenance is metadata, not a gate (tensor names, not hashes; an
  embedding rank would crash, which is why the publisher left token_embd alone);
  the fork pin now has a third reason, since older builds refuse the adapter with
  `unsupported prism.hadamard.version: 2`. Base-alignment law updated: the 9B
  refusal escape hatch was a fetch, and it has now been taken.
- AGENTS.md: topology payload cell and the zealot operator block.
This commit is contained in:
Frank
2026-10-19 14:42:56 -07:00
parent 3eaba57925
commit 5d5cac4da3
2 changed files with 30 additions and 11 deletions
+6 -2
View File
@@ -27,7 +27,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|--------|---------|---------|---------|------|
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj | 7771 |
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
Payload identity, shape, eras, laws and rollback live in ONE page each:
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
@@ -440,7 +440,11 @@ disagrees with it is the bug.
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests).
tower can crash the server with concurrent image requests) + the rank-1
refusal-direction adapter applied at load by `--lora-scaled …:1`. The
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
— and build 10777 is also the gate that lets it load: older forks refuse it
with `unsupported prism.hadamard.version: 2`.
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
+24 -9
View File
@@ -1,6 +1,6 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed"
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed"
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
tags:
- llm
@@ -10,10 +10,10 @@ tags:
- gguf
- rtx3090
- endpoint
timestamp: 2026-10-18
timestamp: 2026-10-19
---
# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 3 × 200,704, f16 K + q8_0 V, MTP armed
# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200,704, f16 K + q8_0 V, MTP armed
Dankclaw's RTX 3090 endpoint — `http://192.168.0.143:7771`, wire id `frank-zealot`, the fleet's **compaction hub**
(three conversations at 200,704 each, the window the other two endpoints also cap at). One page
@@ -29,11 +29,12 @@ is the bug.
|---|---|
| Checkpoint | `/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf` — 2,675,410,720 B, LFS sha256 `c79fcdfaf6b7bf47…` |
| Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) |
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (`--verify-only`; `-f`, byte count, LFS sha256 per file) |
| Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws |
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file |
| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload |
| Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
| Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
**Why this shape.** `general.architecture qwen35`: 33 blocks (32 trunk + `blk.32` = the MTP block)
`full_attention_interval 4` ⇒ 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, `nextn_predict_layers 1`
@@ -57,6 +58,17 @@ and `UBATCH_SIZE=2048` beside an armed drafter is still the shape that halved de
`--spec-draft-model` — the draft rides the main model's weights, so a draft file would load the trunk twice.
`SPEC_TYPE=none` is the rollback and strips the whole set.
**The adapter is behaviour, not shape.** It folds 3.2 MB of rank-1 tensors into the trunk at
load, so it moves nothing in the budget equation above — no KV, no slot, no window. What it
changes is what the endpoint answers, and the only instrument for that is a refusal probe:
the same two prompts the plain pack answered `I cannot provide…` (captured on the live
endpoint before the swap, `temperature 0.1`), re-asked after it. Budget **≥ 2 048 tokens** for
that probe on this payload — the thinking channel swallows a 600-token request whole and
`content` comes back empty with `finish_reason: length`, which reads as a refusal that never
happened. The adapter's own card table is **not** evidence here: it was measured on the PTQ1_0
packing. Rollback is `LORA_FILE=` (empty) in the env and one redeploy — no rebuild, no payload
move; `LORA_SCALE=0` is the same result through the flag.
**Harness contract** (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw-3090)`, id `frank-zealot`):
`contextWindow: 200704` — **the per-slot window**, because llama.cpp divides `--ctx-size` across slots and answers a
longer prompt with a 400, never a truncation — plus `maxTokens: 16384`, `input: [text, image]`
@@ -113,6 +125,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads |
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
| 2026-10-18 03:14 → 13:19 (arm under the unit, then the leftover drop-in) | `Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf` 5,946,648,928 B + hardlinked Q8 mmproj | PrismML fork 10777 | arm: 2 × 200,704, batch/ubatch 3072/1536. The leftover `frank-zealot.service.d/ptq1.conf` then made the 13:16 cutover deploy boot THIS payload at 3 × 133,888 (`new slot, n_ctx = 133888` x3, journal read 2026-10-19) until the 13:19 boot | — | `/home/dank/models/Bonsai2-PTQ1_0/` + `scripts/download-frank-zealot-bonsai2-ptq1.sh` |
@@ -169,13 +182,15 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
- MTP is a property of the **file**: the plain PQ2_0 sibling could not draft at any flag value; only the `-MTP` export carries `blk.N.nextn.*`.
- **The format is the runtime lock.** PQ2_0 is `general.file_type 141` over 202 tensors of custom GGML type **142** plus `prism.hadamard.*` metadata: stock llama.cpp refuses the type, and a loader ignoring the Hadamard transform would dequant **wrong, silently** (same lock class as ROCmFPX). Build ≥ b10776 is additionally the floor that pairs MTP with a quantized V cache.
- Check arch support in the source tree: `grep -c` on the packed binary returns 0 for every arch, including the one it serves.
- **Base alignment.** Bonsai-2 is the censored base lineage. `/home/dank/models/Bonsai2-CRACK/` matches the **27B** geometry (a one-line `MODEL_FILE` swap there, identical tensor census); no dealigned 9B exists, so for this payload the refusal escape hatch is a fetch, not a revert.
- **Base alignment.** Bonsai-2 is the censored base lineage. `/home/dank/models/Bonsai2-CRACK/` matches the **27B** geometry (a one-line `MODEL_FILE` swap there, identical tensor census); no dealigned 9B exists, so for this payload the refusal escape hatch is a fetch, not a revert — which is exactly what was taken on 2026-10-19: the rank-1 adapter in the Adapter row above, behind `LORA_FILE`.
- **A stale bench drop-in hijacks the production boot and no health check sees it.** An `Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` in `frank-zealot.service.d/` booted a 27B PTQ1_0 payload while `/health` answered 200, `/v1/models` echoed the alias flag and the e2e gate passed 33/33 — none of them read the payload.
- After any bench window: `systemctl show -p DropInPaths` → `tr '\0' ' ' </proc/$pid/cmdline` → ledger, in that order; drop-ins retire to `~/.AGENTTRASH/<date>-<topic>/`, and payload decisions land on this page the moment they are made, or an approved arm becomes indistinguishable from a rogue one a day later.
- **An arm env must use the serve script's own variable names** (`CACHE_TYPE_K`/`CACHE_TYPE_V`, `BIND_ADDR`, `NGL`): an invented `KV_CACHE_TYPE` is swallowed by `: "${CACHE_TYPE_K:=f16}"` and the arm boots f16 — an OOM that reads like a model fault.
- **Stop, shebang, GPU node.** llama.cpp rides out the default stop timeout (`TimeoutStopSec=180` is on the unit for that reason) and a hand-booted arm keeps answering 7771 behind a `failed` unit — kill it **by PID**, never `pkill -f llama-server`.
- `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check.
- `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change.
- **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason.
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter.
## Contract
@@ -210,12 +225,12 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
boot ledger and the KV derivation all live on this page now; the header used to hold them, and two of its
rung figures were wrong (`CACHE_TYPE_V=q8_0` and `CTX_SIZE=401408` both free **3,840 MiB** at the 524,288 pool,
not 8,192 and not ~6.7 GiB — the Fallback rungs table above is the corrected statement).
- `scripts/serve-frank-zealot.sh` — the exec line and its rationale comments; honors `FRANK_ZEALOT_ENV_FILE` for arm boots; exits 1 naming a missing binary, GGUF or projector rather than half-loading (a restart policy would turn an unfinished download into a crash loop that reads like a runtime fault).
- `scripts/serve-frank-zealot.sh` — the exec line and its rationale comments; honors `FRANK_ZEALOT_ENV_FILE` for arm boots; exits 1 naming a missing binary, GGUF, projector or adapter rather than half-loading (a restart policy would turn an unfinished download into a crash loop that reads like a runtime fault).
- `scripts/frank-zealot.service` — root unit, relinked into `/etc/systemd/system/`.
- `scripts/download-frank-zealot-bonsai9.sh` (active) · `…-bonsai2.sh` · `…-bonsai2-ptq1.sh` ·
- `scripts/download-frank-zealot-bonsai9.sh` (active payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh` (the refusal adapter; its header is the provenance record) · `…-bonsai2.sh` · `…-bonsai2-ptq1.sh` ·
`scripts/download-frank-zealot.sh` (master-era GGUFs) · `old/download-frank-zealot-sglang-int8.sh` (SGLang).
- `tests/frank_zealot_unit.bats` — stub-argv seam plus the VRAM fit gate.
- `scripts/deploy-remote.sh` — `deploy_llm_zealot` (payload-path gates, MainPID-hardened health);
- `scripts/deploy-remote.sh` — `deploy_llm_zealot` (payload-path gates including the adapter, MainPID-hardened health);
- `shared/prometheus/prometheus.yml`, `shared/prometheus/rules/llm-fleet.yml`
`shared/grafana/provisioning/dashboards/llm-fleet.json` — the scrape, the alert, the dashboard.
- Live state, not repo files: the OpenWebUI picker row (back up `webui.db` before re-keying) and `~/.omp/agent/models.yml`.