docs(frank-zealot): Prism Bonsai-2 PQ2_0 runbook; indexes and log re-keyed
- new runbook: runtime lock (build 10754 / type 142 door), geometry census, four-arm VRAM table (f16 wall, q8_0 deployed, q4_0 rung, 2x262,144), measured rate ladder, behavior contract, ops notes - LLM/index.md + vault root index: new ACTIVE entry, 9B era retired in place - log.md: the trial+cutover wave record, incl. the env-var-name law
This commit is contained in:
1 parent
f383038437
commit
7706e06775
4 files changed
+262
-14
No files matched your search
@@ -20,11 +20,13 @@ digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
|
||||
ctx, vision ON)** and
|
||||
**llama.cpp CUDA (frank-zealot, port 7771: HauhauCS Gemma4-26B-A4B QAT
|
||||
Uncensored Q4_K_M GGUF — 26B MoE hybrid (25 sliding-window + 5 full-attention
|
||||
layers), ONE 200,704-token conversation, MTP disabled,
|
||||
q8_0 K + q8_0 V (FP8 class), vision via the publisher's own BF16
|
||||
mmproj, on the RTX 3090)**.
|
||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
|
||||
the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom
|
||||
tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the
|
||||
file)**.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -106,13 +108,25 @@ frank-zealot, port 7771]
|
||||
(`slow`/`advisor` → frank-archon) carried forward. Superseded by the
|
||||
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
|
||||
engine moved with the model
|
||||
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — **ACTIVE since 2026-10-07**
|
||||
(root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
|
||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090
|
||||
— TWO 200,704-token conversations (`--ctx-size 401408 --parallel 2`), exact
|
||||
f16 K + f16 V, vision via the publisher's BF16 mmproj. MTP is OFF because
|
||||
the file has no MTP tensors at all. Priced from the MiMo Q6_K boots (same
|
||||
qwen35 geometry, same card, same shape). Supersedes the Gemma4 era (below).
|
||||
- [[frank-zealot-llama-prism-bonsai2-pq20]] — **ACTIVE since 2026-10-17**
|
||||
(root systemd unit, port 7771): the **PrismML fork** build of llama.cpp
|
||||
(vendor 10754, commit `2459f68b5` — the only loader for the payload's custom
|
||||
tensor type 142 + `prism.hadamard.*` transform) serving
|
||||
prism-ml/Ternary-Bonsai-2-27B-gguf **PQ2_0** (2.13 bpw) on the RTX 3090 —
|
||||
TWO 200,704-token conversations (`--ctx-size 401408 --parallel 2`), **q8_0 K
|
||||
+ q8_0 V** (34 KiB/token over the 16 full layers = 13,328 MiB; the f16 pair
|
||||
needs 25,088 MiB and dies), settled 22,664 used / 1,461 free, vision via the
|
||||
publisher's Q8_0 mmproj (+873 MiB, charged AFTER the engine's fit check), MTP
|
||||
OFF (no MTP tensors). 76.6 t/s single-stream decode, 691.7 t/s prefill at
|
||||
166k, 109.4 t/s aggregate at two streams. Supersedes the Qwen3.5-9B era below.
|
||||
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — **RETIRED 2026-10-07 →
|
||||
10-17** (payload + master CUDA build on disk as rollback): llama.cpp master
|
||||
(CUDA sm_86) serving HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive
|
||||
**Q6_K** on the RTX 3090 — TWO 200,704-token conversations
|
||||
(`--ctx-size 401408 --parallel 2`), exact f16 K + f16 V, vision via the
|
||||
publisher's BF16 mmproj. MTP is OFF because the file has no MTP tensors at
|
||||
all. Priced from the MiMo Q6_K boots (same qwen35 geometry, same card, same
|
||||
shape). Supersedes the Gemma4 era (below).
|
||||
- [[frank-zealot-llama-gemma4-q4km-200704]] — **RETIRED 2026-10-07** (same
|
||||
day; superseded by qwen35-9b-q6k; payload on disk as rollback): llama.cpp
|
||||
master (CUDA sm_86) serving HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-
|
||||
|
||||
Reference in new issue
Block a user