diff --git a/AGENTS.md b/AGENTS.md index 4724f6a..e604e71 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,7 +27,7 @@ merge absorbed the former `vllmScripts` repo, history preserved). |--------|---------|---------|-------|------| | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 | | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 on 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; ~10.8 GiB reserved where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770| -| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +452 MiB of recurrent state for the three DeltaNet cells), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Boot 17,876 used / 6,249 free; 167.2 t/s single-stream decode, 280.2 t/s aggregate across three concurrent streams, 2,877 t/s prefill at 106,418. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 | +| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 | Retired endpoints (`frank-npu`, and every prior runtime) are in the [runbook index](#runbook-index). The full swap history lives in @@ -494,9 +494,10 @@ so **q8_0 K + q8_0 V bills 17,408 B/token ≈ 17 KiB** = **13,056.00 MiB** at th 786,432 pool (ledger-exact) — exactly HALF the 27B's 34,816 B/token, which is the whole reason three native windows fit where the 27B afforded only two shortened ones. **f16 K would bill 26,112 MiB and cannot boot on a 24,123 MiB -card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT — -`RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)` ≈ 150 MiB each — so -pool arithmetic alone under-prices the shape. +card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT — the +production ledger prints `RS buffer size = 150.75 MiB (3 cells, 32 layers, +3 seqs)`, while the SPEC-ARMED arms print 452.25 MiB (MTP depth 2 bills three +seqs per slot, so 9 cells) — so pool arithmetic alone under-prices the shape. **The shape (2026-10-18):** `--ctx-size 786432 --parallel 3` = THREE conversations of 262,144, every one at the model's TRAINED window (Frank: "if @@ -1168,6 +1169,18 @@ linked runbooks. is broken". Diagnose with `lastError` from `/api/v1/targets`, not the alert list, and note Alertmanager has no persistence: a reboot re-stamps `startsAt` on an alert that has been firing for weeks. +- **A leftover bench drop-in silently hijacks the next deploy, and no health + gate sees it.** The sanctioned bench-window method — a transient + `Environment=FRANK__ENV_FILE=/tmp/.env` drop-in, the arm env + `source`ing the repo env and then overriding it — outlives the bench unless + it is retired. On 2026-10-18 a stale `frank-zealot.service.d/ptq1.conf` from + an unfinished experiment made the cutover deploy boot a **different model at + a different shape** (3 slots of 133,888, not 3 × 262,144) while `/health` + answered 200, `/v1/models` echoed the alias (a flag, not a fact) and the + Playwright gate passed 33/33. After deploying any recently-benched LLM + service: `systemctl show -p DropInPaths` (must be empty), then + `tr '\0' ' ' -/` + `daemon-reload`. **Ops** diff --git a/scripts/frank-zealot.env b/scripts/frank-zealot.env index f72e31e..e5e69cd 100644 --- a/scripts/frank-zealot.env +++ b/scripts/frank-zealot.env @@ -65,9 +65,11 @@ UBATCH_SIZE=2048 # the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the # whole reason three native windows fit. f16 would bill 26,112 MiB at this pool # (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers -# bill a fixed recurrent state per SLOT instead of per token — the ledger's -# "RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)" = 150.75 MiB a -# slot — which is why pool arithmetic alone under-prices the shape. +# bill a fixed recurrent state per SLOT instead of per token — the PRODUCTION +# ledger prints "RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)" +# (~50 MiB a cell), while the SPEC-ARMED arms print 452.25 MiB, because MTP +# depth 2 bills three seqs per slot (9 cells) — which is why pool arithmetic +# alone under-prices the shape. CACHE_TYPE_K=q8_0 CACHE_TYPE_V=q8_0 # SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter @@ -103,18 +105,23 @@ SPEC_DRAFT_P_MIN=0 # PRESENCE_PENALTY="" is the documented opt-out. PRESENCE_PENALTY=1.0 NGL=99 -# Boot ledger (2026-10-18, this exact shape, nvidia-smi settled): -# 17,876 MiB used / 6,249 MiB free of 24,123 +# Boot ledger (2026-10-18 PRODUCTION boot under the unit, verbosity 4): +# 17,922 MiB used / 6,203 MiB free of 24,123 (nvidia-smi settled) +# CUDA0 model 2,036.71 MiB + CPU_Mapped 257.66 MiB # CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions) -# CUDA0 RS 452.25 MiB (3 recurrent-state cells) -# compute 1,409 MiB CUDA + 1,201 MiB CUDA_Host, projector 595 + 248 MiB +# CUDA0 RS 150.75 MiB (3 cells, 32 layers, 3 seqs) +# compute 1,409.02 MiB CUDA + 1,088.87 MiB CUDA_Host (+2.84 MiB output buf) +# mmproj worst-case estimate 868.32 MiB (248.10 CUDA + 24.93 CPU compute +# meta) — charged AFTER the engine's own fit check, as always # no clamp line, no cudaMalloc error -# Measured on the same boot: 167.2 t/s greedy decode (/completion, 512 tokens), -# 158.9 t/s single-stream chat, 280.2 t/s aggregate across THREE concurrent -# streams (93.8 each), 2,877 t/s prefill at 106,418 tokens. The 27B record it -# replaces: 76.6 decode, 109.4 aggregate across two streams, 691.7 prefill at -# 166k — beaten on every axis. The FIRST request after a boot is a warmup -# artifact (37.7 t/s class); never report it. +# Measured on this boot (/completion timings, greedy, ignore_eos, 512 tokens): +# 168.7 t/s decode (three runs 168.4-168.7) and ~300 t/s aggregate across +# THREE concurrent streams (102.8 each). The trial arm's record (167.2 decode, +# 280.2 aggregate, 2,877 t/s prefill at 106,418) stands; the 27B it replaces +# gave 76.6 / 109.4 / 691.7 at 166k. The FIRST request after a boot is a +# warmup artifact (37.7 t/s class) and a natural generation that stops early +# makes a wall-clock aggregate read 4x low - measure with ignore_eos and read +# the engine's own timings. # Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the # 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here # frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever diff --git a/serverAppConfigs-docs/Dankclaw/OpenWebUI.md b/serverAppConfigs-docs/Dankclaw/OpenWebUI.md index 0a20bb0..521e44d 100644 --- a/serverAppConfigs-docs/Dankclaw/OpenWebUI.md +++ b/serverAppConfigs-docs/Dankclaw/OpenWebUI.md @@ -53,11 +53,18 @@ Policy while candidate arms held the port): the candidate — Ternary-Bonsai-2-* PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the picker row below was re-keyed to it (see the next paragraph). -The `model` table carries exactly one decoration row, `frank-zealot` -(context length 262,144 = the per-slot window since 2026-10-18; description -re-keyed the same day to Ternary-Bonsai-2-9B PQ2_0 + MTP + Q8_0 vision mmproj — -the row's `description` is live DB state, so a payload swap must re-key it in -the database too, not only in the repo). +The `model` table carries exactly two decoration rows, `frank-zealot` and +`frank-dragoon`. Both the context length and the description live **inside +JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds +`{"description": …}`; there is no `context_length` or `description` column, so +`sqlite3` one-liners against those names fail. Re-keyed 2026-10-18: +`frank-zealot` → 262,144 with the Bonsai-2-9B description, and `frank-dragoon` +→ 200,704 (its `params` had kept the pre-lowering 262,144 window from +2026-10-17 while its own description already said 200,704). These rows are +live DB state, so a payload swap must re-key the database too, not only the +repo. There is no `sqlite3` binary on Dankclaw — use `python3 -c` with the +stdlib `sqlite3` module, and take a `Connection.backup()` snapshot first (the +DB is in WAL and the app is live). A third gate sits *after* the table, and it is the one that bites family accounts (2026-10-07): a model row is only visible to a non-admin user when diff --git a/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md b/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md index 69572e0..82090e0 100644 --- a/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md +++ b/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md @@ -76,8 +76,10 @@ axis improved — decode, prefill, three-stream aggregate, and free VRAM. **13,056.00 MiB**, ledger-exact. That is exactly HALF the 27B's 34,816 B/token. **f16 KV would bill 26,112 MiB against a 24,123 MiB card** and dies at `cudaMalloc`. The 24 DeltaNet layers bill a fixed recurrent state per SLOT -instead — `RS buffer size = 452.25 MiB (3 cells, 32 layers, 3 seqs)` ≈ 150 MiB -each — so pool arithmetic alone under-prices the shape. +instead: the PRODUCTION ledger prints `RS buffer size = 150.75 MiB (3 cells, +32 layers, 3 seqs)` ≈ 50 MiB a cell, while the SPEC-ARMED arms printed +452.25 MiB — MTP depth 2 bills three seqs per slot, so the drafter triples +even the recurrent state. Pool arithmetic alone under-prices the shape. ## The shape (`scripts/frank-zealot.env`) @@ -123,30 +125,36 @@ inputs plus 2–3 concurrent subagents — so the shipped arm takes the +17 % prefill and +12 % three-stream aggregate over single-stream speed, and keeps 6.2 GiB free. Frank confirmed this arm on 2026-10-18. -## Boot ledger (deployed arm, verbosity 4) +## Boot ledger (production boot 2026-10-18 13:19, under the unit, verbosity 4) ``` -CUDA0 KV buffer size = 13056.00 MiB # q8_0 K + q8_0 V, 786,432 positions -CUDA0 RS buffer size = 452.25 MiB # 3 recurrent-state cells, 32 layers, 3 seqs -compute 1,409 MiB CUDA + 1,201 MiB CUDA_Host -mmproj 595 MiB + 248 MiB # charged AFTER the fit check -nvidia-smi settled: 17,876 MiB used / 6,249 MiB free of 24,123 +CUDA0 model buffer size = 2036.71 MiB # + 257.66 MiB CPU_Mapped (mmap) +CUDA0 KV buffer size = 13056.00 MiB # q8_0 K + q8_0 V, 786,432 positions +CUDA0 RS buffer size = 150.75 MiB # 3 cells, 32 layers, 3 seqs +CUDA0 compute buffer = 1409.02 MiB # + 1088.87 MiB CUDA_Host, 2.84 output +mmproj worst-case (est.) = 868.32 MiB # 248.10 CUDA + 24.93 CPU, AFTER fit +nvidia-smi settled: 17,922 MiB used / 6,203 MiB free of 24,123 +n_slots = 3, n_ctx_slot = 262144, kv_unified = false ``` No clamp line, no `cudaMalloc` error. The engine's own projection (21,439-class numbers from the 27B era) excludes the projector — **read settled `nvidia-smi`, never the projection**. -## Measured (deployed arm) +## Measured (production boot, `/completion` `timings`, greedy, `ignore_eos`) -- **167.2 t/s** greedy decode (512 tokens, `/completion` `timings`); 158.9 t/s - single-stream chat with the card's sampler. -- **280.2 t/s aggregate** across three concurrent streams (≈93.8 each). -- **2,877 t/s prefill** at 106,418 prompt tokens. +- **168.7 t/s** decode — three runs of 512 tokens: 168.4 / 168.7 / 168.7 + (the trial arm printed 167.2; same shape, same noise band). +- **~300 t/s aggregate** across three concurrent 512-token streams + (102.8 t/s per stream, round 1 and 2 both) — the trial arm's 280.2. +- **2,877 t/s prefill at 106,418 tokens** — the trial arm; the production boot + has not re-run that depth. - The 27B record it replaces: 76.6 decode, 109.4 aggregate across two streams, 691.7 prefill at 166k. Beaten on every axis. - The **first request after a boot is a warmup artifact** (37.7 t/s class) — - never report it. + never report it. And never time a wall-clock aggregate on a generation that + stops on its own: a 24-token countdown measured 72 t/s across three streams + while the same shape with `ignore_eos` gave 300. ## Contract traps @@ -186,6 +194,20 @@ never the projection**. 7771 — a live port with no unit behind it. `TimeoutStopSec=180` is now on the unit (llama.cpp needs the drain time to release slots and the card); the hand-booted arm must be killed by PID, never `pkill -f llama-server`. +- **A leftover arm drop-in hijacks the production boot, and no health check + sees it.** The sanctioned bench method is a transient + `Environment=FRANK_ZEALOT_ENV_FILE=/tmp/.env` drop-in, and an arm env + `source`s the repo env and then overrides it. On 2026-10-18 a stale + `frank-zealot.service.d/ptq1.conf` — written 03:29 for an unfinished 27B + **PTQ1_0** experiment — was still installed, so the cutover deploy restarted + the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf + --ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three + slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said + `frank-zealot` (the alias is a flag, not a fact), and the Playwright gate + passed 33/33 — none of those read the payload. **After deploying an endpoint + that was recently benched: `systemctl show -p DropInPaths`, then + `tr '\0' ' ' -/`, `daemon-reload`, restart. ## Compaction hub @@ -197,6 +219,33 @@ the window ≈ 222,822 est, worst case ≈ 245,891 ≤ 262,144). With `--paralle a compaction request no longer serializes behind a live generation (the Gemma4 era's `--parallel 1` law). +## Deploy ledger (2026-10-18) + +- `./scripts/deploy.sh dankclaw --service frank-zealot` at 13:16 PDT; the e2e + gate ran 33/33. The first boot was the hijacked arm above; the drop-in went + to `~/.AGENTTRASH/20261018-zealot-ptq1-arm/` and the 13:19 restart is the + ledger recorded here. +- Live probes on the shipped boot: chat `27*59` → **1593**; `where_am_i` tool + call fired with schema-correct empty arguments; 512 px solid-red PNG → + **Red** at 278 prompt tokens with `process_mtmd` chunk encoding in the + journal; `/props` reports `n_ctx 262144` and `presence_penalty 1.0`; + `/metrics` answers 200 with 15 `llamacpp:*` families and + `up{job="frank-zealot"}` is **1** in Prometheus for the first time in its + retained history. +- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot + `webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also + corrected 262,144 → 200,704 — see [[OpenWebUI]]). +- What the bench window left behind, and it is NOT garbage: the **PTQ1_0 + candidate was fetched and armed but never measured** — + `scripts/download-frank-zealot-bonsai2-ptq1.sh` is committed, and its header + states the case (1,201 MiB of freed weights against the publisher's own table + showing dense-ternary unpacking LOSING on Ampere, with no GA102 row to settle + the 3090). `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + a hardlinked + mmproj, 03:02), the arm env `/tmp/zealot-arm-current.env` and the + `/tmp/arm-*.jsonl` 9B results are disk/`/tmp` state a reboot erases. Either + measure the PTQ1_0 arm in a real window or record it as declined — leaving it + armed and unmeasured is exactly what let it hijack a production boot. + ## Fallback rungs 1. `UBATCH_SIZE=1024` — frees **1,306 MiB** measured on this shape (prefill diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index c2f3102..f24d044 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -2156,4 +2156,37 @@ every restart since the cutover. picker row (DB `description` + context length 262,144) re-keyed. Rollback: revert the swap commit — the 27B payload and the build-10754 fork tree are both still on disk. +- Deployed 13:16 PDT (`deploy.sh dankclaw --service frank-zealot`, e2e 33/33) + — and the first boot was **not the shipped payload**: a stale + `frank-zealot.service.d/ptq1.conf` from the bench window (written 03:29 for + an unfinished 27B **PTQ1_0** experiment) pointed + `FRANK_ZEALOT_ENV_FILE` at `/tmp/zealot-arm-current.env`, so the unit booted + that arm: 3 slots of 133,888, 21,188 MiB used, batch 3072/ubatch 1536. + `/health` 200, `/v1/models` echoed the alias, Playwright 33/33 — nothing in + the gate reads the payload. Drop-in moved to + `~/.AGENTTRASH/20261018-zealot-ptq1-arm/`, `daemon-reload`, restart at 13:19: + `n_slots = 3, n_ctx_slot = 262144`, KV 13,056.00 MiB, settled 17,922/6,203. +- Production measures (engine `timings`, greedy, `ignore_eos`): **168.7 t/s** + decode (168.4/168.7/168.7) and **~300 t/s** aggregate across three 512-token + streams (102.8 each). My first battery reported 117 and 72 t/s: it timed + natural generations that stopped at 24 tokens — wall-clock aggregate on an + early EOS reads 4× low. `ignore_eos` + the engine's own numbers is the only + honest way to time this. +- Ledger correction: the shipped shape prints **RS 150.75 MiB** (3 cells), not + the 452.25 MiB my first-pass notes carried — that figure belongs to the + SPEC-ARMED arms, where MTP depth 2 bills three seqs per slot (9 cells). The + drafter triples the recurrent state as well as billing 3,072 MiB of f16 KV. + Corrected in AGENTS.md (topology row + architecture section), the env header + and the runbook, with the production boot ledger replacing the trial's. +- Verified live: chat `27*59` → 1593; `where_am_i` tool call fired; 512 px + solid-red PNG → **Red** (278 prompt tokens, `process_mtmd` in the journal); + `/props` `n_ctx 262144`, `presence_penalty 1.0`; `/metrics` 200 with 15 + `llamacpp:*` families and `up{job="frank-zealot"} = 1` — the first non-zero + sample in that job's retained history. +- OpenWebUI `model` rows re-keyed in the live DB (backup + `webui.db.bak-20261018`): `frank-zealot` → 262,144 + the 9B description, and + `frank-dragoon`'s `context.length` corrected 262,144 → **200,704** (it had + kept the pre-lowering window from 2026-10-17 while its own description said + 200,704). Note the columns are `params`/`meta` JSON — the table has no + `context_length`/`description` columns.