docs(frank-zealot): PTQ1_0 was ordered, measured and approved — not an unfinished arm
Frank's 02:36 instruction (archive 01a14c93) ordered the 27B PTQ1_0 arm at 2 x 200,704 with the Q8 mmproj and batch/ubatch 3072/1536; it served production 03:14:39-03:38:57 (pid 191358, journal type ptq1_0: 402 tensors). Recovered its numbers from that process's own print_timing lines: ~48 t/s single-stream decode, 25-27 t/s per stream at two streams, prefill 1,295 t/s at 30,637 -> 943 at 97,856 -> 801 at 141,365. It bought the promised 1,201 MiB and lost 37% of decode against the PQ2_0 record, so the publisher's Ampere warning generalizes to GA102; Frank scratched the 27B plan at 03:33 for the 9B. Every 'unfinished experiment' claim is re-keyed, and the AGENTS.md law gains the durable lesson: a bench verdict that never reaches the repo is indistinguishable from a rogue arm.
This commit is contained in:
1 parent
e7841bf9d6
commit
06a9cb60eb
4 files changed
+84
-22
No files matched your search
@@ -198,8 +198,9 @@ never the projection**.
|
||||
sees it.** The sanctioned bench method is a transient
|
||||
`Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` drop-in, and an arm env
|
||||
`source`s the repo env and then overrides it. On 2026-10-18 a stale
|
||||
`frank-zealot.service.d/ptq1.conf` — written 03:29 for an unfinished 27B
|
||||
**PTQ1_0** experiment — was still installed, so the cutover deploy restarted
|
||||
`frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm
|
||||
Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) —
|
||||
was still installed, so the cutover deploy restarted
|
||||
the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf
|
||||
--ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three
|
||||
slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said
|
||||
@@ -235,16 +236,30 @@ era's `--parallel 1` law).
|
||||
- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot
|
||||
`webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also
|
||||
corrected 262,144 → 200,704 — see [[OpenWebUI]]).
|
||||
- What the bench window left behind, and it is NOT garbage: the **PTQ1_0
|
||||
candidate was fetched and armed but never measured** —
|
||||
`scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and its header
|
||||
states the case (1,201 MiB of freed weights against the publisher's own table
|
||||
showing dense-ternary unpacking LOSING on Ampere, with no GA102 row to settle
|
||||
the 3090). `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + a hardlinked
|
||||
mmproj, 03:02), the arm env `/tmp/zealot-arm-current.env` and the
|
||||
`/tmp/arm-*.jsonl` 9B results are disk/`/tmp` state a reboot erases. Either
|
||||
measure the PTQ1_0 arm in a real window or record it as declined — leaving it
|
||||
armed and unmeasured is exactly what let it hijack a production boot.
|
||||
- **What the bench window left behind is not garbage — the PTQ1_0 arm was
|
||||
ordered, measured, and then scratched by Frank himself.** The instruction at
|
||||
**02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the
|
||||
**Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in
|
||||
**two** contexts. The unit booted it at **03:14** (journal:
|
||||
`type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj
|
||||
loaded, 2 × 200,704) and served 24 minutes of traffic. Its own
|
||||
`print_timing` lines measure **~48 t/s** single-stream decode
|
||||
(48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations),
|
||||
25–27 t/s per stream with two streams live, and prefill **1,295 t/s at
|
||||
30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the
|
||||
1,201 MiB the fetch script promised and paid for it on decode, exactly as the
|
||||
publisher's Ampere row warned (the PQ2_0 record for the same weights is
|
||||
76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the
|
||||
9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38.
|
||||
- The actual failure was **documentation, not measurement.** An approved
|
||||
payload that survives only as a systemd drop-in plus a journal window is,
|
||||
a day later, indistinguishable from a rogue arm — which is why the 13:16 boot
|
||||
read as a hijack. Payload decisions belong in this runbook the moment they
|
||||
are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and
|
||||
the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal
|
||||
(`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0
|
||||
numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked
|
||||
mmproj) stays on disk as the 27B rollback.
|
||||
|
||||
## Fallback rungs
|
||||
|
||||
|
||||
Reference in new issue
Block a user