docs(frank-zealot): PTQ1_0 was ordered, measured and approved — not an unfinished arm
Frank's 02:36 instruction (archive 01a14c93) ordered the 27B PTQ1_0 arm at 2 x 200,704 with the Q8 mmproj and batch/ubatch 3072/1536; it served production 03:14:39-03:38:57 (pid 191358, journal type ptq1_0: 402 tensors). Recovered its numbers from that process's own print_timing lines: ~48 t/s single-stream decode, 25-27 t/s per stream at two streams, prefill 1,295 t/s at 30,637 -> 943 at 97,856 -> 801 at 141,365. It bought the promised 1,201 MiB and lost 37% of decode against the PQ2_0 record, so the publisher's Ampere warning generalizes to GA102; Frank scratched the 27B plan at 03:33 for the 9B. Every 'unfinished experiment' claim is re-keyed, and the AGENTS.md law gains the durable lesson: a bench verdict that never reaches the repo is indistinguishable from a rogue arm.
This commit is contained in:
Frank
@@ -1183,14 +1183,22 @@ linked runbooks.
|
||||
gate sees it.** The sanctioned bench-window method — a transient
|
||||
`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env` drop-in, the arm env
|
||||
`source`ing the repo env and then overriding it — outlives the bench unless
|
||||
it is retired. On 2026-10-18 a stale `frank-zealot.service.d/ptq1.conf` from
|
||||
an unfinished experiment made the cutover deploy boot a **different model at
|
||||
a different shape** (3 slots of 133,888, not 3 × 262,144) while `/health`
|
||||
answered 200, `/v1/models` echoed the alias (a flag, not a fact) and the
|
||||
Playwright gate passed 33/33. After deploying any recently-benched LLM
|
||||
service: `systemctl show <svc> -p DropInPaths` (must be empty), then
|
||||
`tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot ledger —
|
||||
and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` + `daemon-reload`.
|
||||
it is retired. On 2026-10-18 a leftover `frank-zealot.service.d/ptq1.conf` —
|
||||
the arm Frank had ordered at 02:36 and measured from 03:14 — made the cutover
|
||||
deploy boot a **different model at a different shape** (3 slots of 133,888,
|
||||
not 3 × 262,144) while `/health` answered 200, `/v1/models` echoed the alias
|
||||
(a flag, not a fact) and the Playwright gate passed 33/33. After deploying any
|
||||
recently-benched LLM service: `systemctl show <svc> -p DropInPaths` (must be
|
||||
empty), then `tr '\0' ' ' </proc/$(pidof llama-server)/cmdline`, then the boot
|
||||
ledger — and retire drop-ins to `~/.AGENTTRASH/<date>-<topic>/` +
|
||||
`daemon-reload`.
|
||||
- **A bench verdict that never reaches the repo is indistinguishable from a
|
||||
rogue arm.** Same window: PTQ1_0 served production for 24 minutes with real
|
||||
numbers (~48 t/s decode, 801 t/s prefill at 141k), but the figures lived only
|
||||
in `/tmp` env files and journald, so the next agent on the host read the
|
||||
surviving drop-in as an unfinished experiment and reported a hijack. Write the
|
||||
measurement into the runbook and the arm script's header in the session that
|
||||
takes it — `/tmp` and the journal are not records.
|
||||
|
||||
**Ops**
|
||||
|
||||
|
||||
@@ -23,6 +23,15 @@
|
||||
# moving 17 % less weight per step can pay. That is an argument, not a
|
||||
# measurement: this script exists so the decision is made on OUR card, against
|
||||
# the PQ2_0 record (76.6 t/s single-stream decode, 1,258 t/s prefill at 35k).
|
||||
# MEASURED ON OUR CARD (2026-10-18). Frank ordered this shape at 02:36 — Q8
|
||||
# mmproj, batch 3072/ubatch 1536, two contexts — and the unit ran it as
|
||||
# production 03:14:39→03:38:57 (pid 191358, journal `type ptq1_0: 402 tensors`).
|
||||
# Its own print_timing lines: decode **48.0 / 47.6 t/s at 512 tokens** (47.9 at
|
||||
# 229, 49.0 at 451), 25–27 t/s per stream with two streams live, prefill 1,295
|
||||
# t/s at 30,637 → 943 at 97,856 → 801 at 141,365. The 1,201 MiB arrived and the
|
||||
# decode tax came with it: −37 % against the PQ2_0 record, so the Ampere warning
|
||||
# DOES generalize to GA102. Frank scratched the 27B plan at 03:33 for the 9B;
|
||||
# this payload is the 27B rollback, not a candidate.
|
||||
#
|
||||
# THE RUNTIME LOCK CARRIES OVER. Both packings are custom GGML tensor types and
|
||||
# stock llama.cpp rejects them ("It rejects PQ2_0 and PTQ1_0 as unknown types"
|
||||
|
||||
@@ -198,8 +198,9 @@ never the projection**.
|
||||
sees it.** The sanctioned bench method is a transient
|
||||
`Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` drop-in, and an arm env
|
||||
`source`s the repo env and then overrides it. On 2026-10-18 a stale
|
||||
`frank-zealot.service.d/ptq1.conf` — written 03:29 for an unfinished 27B
|
||||
**PTQ1_0** experiment — was still installed, so the cutover deploy restarted
|
||||
`frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm
|
||||
Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) —
|
||||
was still installed, so the cutover deploy restarted
|
||||
the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf
|
||||
--ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three
|
||||
slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said
|
||||
@@ -235,16 +236,30 @@ era's `--parallel 1` law).
|
||||
- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot
|
||||
`webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also
|
||||
corrected 262,144 → 200,704 — see [[OpenWebUI]]).
|
||||
- What the bench window left behind, and it is NOT garbage: the **PTQ1_0
|
||||
candidate was fetched and armed but never measured** —
|
||||
`scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and its header
|
||||
states the case (1,201 MiB of freed weights against the publisher's own table
|
||||
showing dense-ternary unpacking LOSING on Ampere, with no GA102 row to settle
|
||||
the 3090). `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + a hardlinked
|
||||
mmproj, 03:02), the arm env `/tmp/zealot-arm-current.env` and the
|
||||
`/tmp/arm-*.jsonl` 9B results are disk/`/tmp` state a reboot erases. Either
|
||||
measure the PTQ1_0 arm in a real window or record it as declined — leaving it
|
||||
armed and unmeasured is exactly what let it hijack a production boot.
|
||||
- **What the bench window left behind is not garbage — the PTQ1_0 arm was
|
||||
ordered, measured, and then scratched by Frank himself.** The instruction at
|
||||
**02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the
|
||||
**Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in
|
||||
**two** contexts. The unit booted it at **03:14** (journal:
|
||||
`type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj
|
||||
loaded, 2 × 200,704) and served 24 minutes of traffic. Its own
|
||||
`print_timing` lines measure **~48 t/s** single-stream decode
|
||||
(48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations),
|
||||
25–27 t/s per stream with two streams live, and prefill **1,295 t/s at
|
||||
30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the
|
||||
1,201 MiB the fetch script promised and paid for it on decode, exactly as the
|
||||
publisher's Ampere row warned (the PQ2_0 record for the same weights is
|
||||
76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the
|
||||
9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38.
|
||||
- The actual failure was **documentation, not measurement.** An approved
|
||||
payload that survives only as a systemd drop-in plus a journal window is,
|
||||
a day later, indistinguishable from a rogue arm — which is why the 13:16 boot
|
||||
read as a hijack. Payload decisions belong in this runbook the moment they
|
||||
are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and
|
||||
the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal
|
||||
(`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0
|
||||
numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked
|
||||
mmproj) stays on disk as the 27B rollback.
|
||||
|
||||
## Fallback rungs
|
||||
|
||||
|
||||
@@ -2157,9 +2157,10 @@ every restart since the cutover.
|
||||
revert the swap commit — the 27B payload and the build-10754 fork tree are
|
||||
both still on disk.
|
||||
- Deployed 13:16 PDT (`deploy.sh dankclaw --service frank-zealot`, e2e 33/33)
|
||||
— and the first boot was **not the shipped payload**: a stale
|
||||
— and the first boot was **not the shipped payload**: a leftover
|
||||
`frank-zealot.service.d/ptq1.conf` from the bench window (written 03:29 for
|
||||
an unfinished 27B **PTQ1_0** experiment) pointed
|
||||
the 27B **PTQ1_0** arm Frank ordered at 02:36 and measured from 03:14 — see
|
||||
the record below) pointed
|
||||
`FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit booted
|
||||
that arm: 3 slots of 133,888, 21,188 MiB used, batch 3072/ubatch 1536.
|
||||
`/health` 200, `/v1/models` echoed the alias, Playwright 33/33 — nothing in
|
||||
@@ -2190,6 +2191,35 @@ every restart since the cutover.
|
||||
200,704). Note the columns are `params`/`meta` JSON — the table has no
|
||||
`context_length`/`description` columns.
|
||||
|
||||
## 2026-10-18 — frank-zealot PTQ1_0 arm: measured, approved, and never written down
|
||||
|
||||
- **Correction to the entry above and to my own report.** The
|
||||
`frank-zealot.service.d/ptq1.conf` drop-in was NOT a half-finished experiment.
|
||||
The session archive (`01a14c93`) carries Frank's own instruction at **02:36**:
|
||||
replace frank-zealot with `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, use the **Q8**
|
||||
mmproj, batch/ubatch **3072/1536**, "fit as much context as possible in 2
|
||||
contexts".
|
||||
- It was the production payload **03:14:39 → 03:38:57** (pid 191358; journal
|
||||
`type ptq1_0: 402 tensors`, `file type = PTQ1_0 — 1.75 bpw ternary`, mmproj
|
||||
loaded, 2 × 200,704) and served real traffic. Recovered from that process's
|
||||
own `print_timing` lines (17 timed tasks): single-stream greedy decode
|
||||
**48.0 and 47.6 t/s at 512 tokens, 47.9 at 229, 49.0 at 451**; 25–27 t/s per
|
||||
stream with two streams live; prefill **1,295 t/s @ 30,637 → 943 @ 97,856 →
|
||||
801 t/s @ 141,365** (one 176 t/s row at 97,024 is a concurrent-slot request,
|
||||
not a clean probe).
|
||||
- Against the PQ2_0 27B record (76.6 t/s decode, 691.7 t/s prefill at 166k):
|
||||
PTQ1_0 freed the **1,201 MiB** its fetch script promised and paid **−37 %
|
||||
decode** — the publisher's "dense ternary unpacking loses on Ampere" warning,
|
||||
now confirmed on GA102 — while prefilling deep prompts faster than that
|
||||
record ever managed. Frank scratched the whole 27B plan at **03:33** ("scratch
|
||||
the current frank-zealot plan completely… 9B… enable mtp… if you can possibly
|
||||
fit a third context, do it"), which is what shipped at 13:16.
|
||||
- The durable failure is mine: those numbers lived only in `/tmp` and the
|
||||
journal, never in the repo, so the next agent on the host could not tell an
|
||||
approved payload from a rogue drop-in and called it a hijack. The runbook
|
||||
deploy ledger now carries the measurement; `/home/dank/models/Bonsai2-PTQ1_0/`
|
||||
stays on disk as the 27B rollback and the drop-in stays trashed.
|
||||
|
||||
|
||||
## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot
|
||||
|
||||
|
||||
Reference in New Issue
Block a user