diff --git a/AGENTS.md b/AGENTS.md index b1c1d0c..edfb729 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1183,14 +1183,22 @@ linked runbooks. gate sees it.** The sanctioned bench-window method — a transient `Environment=FRANK__ENV_FILE=/tmp/.env` drop-in, the arm env `source`ing the repo env and then overriding it — outlives the bench unless - it is retired. On 2026-10-18 a stale `frank-zealot.service.d/ptq1.conf` from - an unfinished experiment made the cutover deploy boot a **different model at - a different shape** (3 slots of 133,888, not 3 × 262,144) while `/health` - answered 200, `/v1/models` echoed the alias (a flag, not a fact) and the - Playwright gate passed 33/33. After deploying any recently-benched LLM - service: `systemctl show -p DropInPaths` (must be empty), then - `tr '\0' ' ' -/` + `daemon-reload`. + it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` — + the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover + deploy boot a **different model at a different shape** (3 slots of 133,888, + not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias + (a flag, not a fact) and the Playwright gate passed 33/33. After deploying any + recently-benched LLM service: `systemctl show -p DropInPaths` (must be + empty), then `tr '\0' ' ' -/` + + `daemon-reload`. +- **A bench verdict that never reaches the repo is indistinguishable from a + rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real + numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only + in `/tmp` env files and journald, so the next agent on the host read the + surviving drop-in as an unfinished experiment and reported a hijack. Write the + measurement into the runbook and the arm script's header in the session that + takes it — `/tmp` and the journal are not records. **Ops** diff --git a/scripts/download-frank-zealot-bonsai2-ptq1.sh b/scripts/download-frank-zealot-bonsai2-ptq1.sh index f3a2506..a60001e 100755 --- a/scripts/download-frank-zealot-bonsai2-ptq1.sh +++ b/scripts/download-frank-zealot-bonsai2-ptq1.sh @@ -23,6 +23,15 @@ # moving 17 % less weight per step can pay. That is an argument, not a # measurement: this script exists so the decision is made on OUR card, against # the PQ2_0 record (76.6 t/s single-stream decode, 1,258 t/s prefill at 35k). +# MEASURED ON OUR CARD (2026-10-18). Frank ordered this shape at 02:36 — Q8 +# mmproj, batch 3072/ubatch 1536, two contexts — and the unit ran it as +# production 03:14:39→03:38:57 (pid 191358, journal `type ptq1_0: 402 tensors`). +# Its own print_timing lines: decode **48.0 / 47.6 t/s at 512 tokens** (47.9 at +# 229, 49.0 at 451), 25–27 t/s per stream with two streams live, prefill 1,295 +# t/s at 30,637 → 943 at 97,856 → 801 at 141,365. The 1,201 MiB arrived and the +# decode tax came with it: −37 % against the PQ2_0 record, so the Ampere warning +# DOES generalize to GA102. Frank scratched the 27B plan at 03:33 for the 9B; +# this payload is the 27B rollback, not a candidate. # # THE RUNTIME LOCK CARRIES OVER. Both packings are custom GGML tensor types and # stock llama.cpp rejects them ("It rejects PQ2_0 and PTQ1_0 as unknown types" diff --git a/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md b/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md index 82090e0..a9880c7 100644 --- a/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md +++ b/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md @@ -198,8 +198,9 @@ never the projection**. sees it.** The sanctioned bench method is a transient `Environment=FRANK_ZEALOT_ENV_FILE=/tmp/.env` drop-in, and an arm env `source`s the repo env and then overrides it. On 2026-10-18 a stale - `frank-zealot.service.d/ptq1.conf` — written 03:29 for an unfinished 27B - **PTQ1_0** experiment — was still installed, so the cutover deploy restarted + `frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm + Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) — + was still installed, so the cutover deploy restarted the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf --ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said @@ -235,16 +236,30 @@ era's `--parallel 1` law). - OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot `webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also corrected 262,144 → 200,704 — see [[OpenWebUI]]). -- What the bench window left behind, and it is NOT garbage: the **PTQ1_0 - candidate was fetched and armed but never measured** — - `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and its header - states the case (1,201 MiB of freed weights against the publisher's own table - showing dense-ternary unpacking LOSING on Ampere, with no GA102 row to settle - the 3090). `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + a hardlinked - mmproj, 03:02), the arm env `/tmp/zealot-arm-current.env` and the - `/tmp/arm-*.jsonl` 9B results are disk/`/tmp` state a reboot erases. Either - measure the PTQ1_0 arm in a real window or record it as declined — leaving it - armed and unmeasured is exactly what let it hijack a production boot. +- **What the bench window left behind is not garbage — the PTQ1_0 arm was + ordered, measured, and then scratched by Frank himself.** The instruction at + **02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the + **Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in + **two** contexts. The unit booted it at **03:14** (journal: + `type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj + loaded, 2 × 200,704) and served 24 minutes of traffic. Its own + `print_timing` lines measure **~48 t/s** single-stream decode + (48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations), + 25–27 t/s per stream with two streams live, and prefill **1,295 t/s at + 30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the + 1,201 MiB the fetch script promised and paid for it on decode, exactly as the + publisher's Ampere row warned (the PQ2_0 record for the same weights is + 76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the + 9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38. +- The actual failure was **documentation, not measurement.** An approved + payload that survives only as a systemd drop-in plus a journal window is, + a day later, indistinguishable from a rogue arm — which is why the 13:16 boot + read as a hijack. Payload decisions belong in this runbook the moment they + are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and + the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal + (`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0 + numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked + mmproj) stays on disk as the 27B rollback. ## Fallback rungs diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 290c304..eb01660 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -2157,9 +2157,10 @@ every restart since the cutover. revert the swap commit — the 27B payload and the build-10754 fork tree are both still on disk. - Deployed 13:16 PDT (`deploy.sh dankclaw --service frank-zealot`, e2e 33/33) - — and the first boot was **not the shipped payload**: a stale + — and the first boot was **not the shipped payload**: a leftover `frank-zealot.service.d/ptq1.conf` from the bench window (written 03:29 for - an unfinished 27B **PTQ1_0** experiment) pointed + the 27B **PTQ1_0** arm Frank ordered at 02:36 and measured from 03:14 — see + the record below) pointed `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit booted that arm: 3 slots of 133,888, 21,188 MiB used, batch 3072/ubatch 1536. `/health` 200, `/v1/models` echoed the alias, Playwright 33/33 — nothing in @@ -2190,6 +2191,35 @@ every restart since the cutover. 200,704). Note the columns are `params`/`meta` JSON — the table has no `context_length`/`description` columns. +## 2026-10-18 — frank-zealot PTQ1_0 arm: measured, approved, and never written down + +- **Correction to the entry above and to my own report.** The + `frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished experiment. + The session archive (`01a14c93`) carries Frank's own instruction at **02:36**: + replace frank-zealot with `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8** + mmproj, batch/ubatch **3072/1536**, "fit as much context as possible in 2 + contexts". +- It was the production payload **03:14:39 → 03:38:57** (pid 191358; journal + `type ptq1_0: 402 tensors`, `file type = PTQ1_0 — 1.75 bpw ternary`, mmproj + loaded, 2 × 200,704) and served real traffic. Recovered from that process's + own `print_timing` lines (17 timed tasks): single-stream greedy decode + **48.0 and 47.6 t/s at 512 tokens, 47.9 at 229, 49.0 at 451**; 25–27 t/s per + stream with two streams live; prefill **1,295 t/s @ 30,637 → 943 @ 97,856 → + 801 t/s @ 141,365** (one 176 t/s row at 97,024 is a concurrent-slot request, + not a clean probe). +- Against the PQ2_0 27B record (76.6 t/s decode, 691.7 t/s prefill at 166k): + PTQ1_0 freed the **1,201 MiB** its fetch script promised and paid **−37 % + decode** — the publisher's "dense ternary unpacking loses on Ampere" warning, + now confirmed on GA102 — while prefilling deep prompts faster than that + record ever managed. Frank scratched the whole 27B plan at **03:33** ("scratch + the current frank-zealot plan completely… 9B… enable mtp… if you can possibly + fit a third context, do it"), which is what shipped at 13:16. +- The durable failure is mine: those numbers lived only in `/tmp` and the + journal, never in the repo, so the next agent on the host could not tell an + approved payload from a rogue drop-in and called it a hijack. The runbook + deploy ledger now carries the measurement; `/home/dank/models/Bonsai2-PTQ1_0/` + stays on disk as the 27B rollback and the drop-in stays trashed. + ## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot