From c0759492e6fc5430233f6e2cc2cd77bc7b2d737e Mon Sep 17 00:00:00 2001 From: Frank Date: Sun, 18 Oct 2026 13:33:04 -0700 Subject: [PATCH] docs(frank-dragoon): the 401,408-position pool is live; every figure re-keyed to the boot ledger MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deployed 2026-10-18 (e2e 33/33). The boot ledger prints `reserving the KV pool, 401408 positions (about 11.0 GiB)` and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`, so the ~10.8 GiB that every page quoted from the ~28 KiB/position arithmetic is replaced by the reservation the engine actually makes (11.0) and its preflight bill (11.3). Host MemAvailable fell 26 -> 22 GiB across the raise: the FLAGS.md collision (pool RAM is n-gram page cache) paid in full, so the warm prompt-cache ratio is flagged as the number to watch. All "committed but not yet deployed" clauses retired: AGENTS.md topology row, KV-pool law and memory ledger, frank-dragoon.env lore, both index pages, the iommu page, and the runbook chronology (third production boot recorded). The sweep also caught the LLM index still describing frank-zealot as Bonsai-2-27B at 2 x 200,704 in its intro and topology diagram - re-keyed to the live 9B: 8 full-attention + 24 Gated DeltaNet + blk.32 MTP, 3 x 262,144, tensor type 141. --- AGENTS.md | 20 +++---- scripts/frank-dragoon.env | 27 +++++----- .../LLM/dankclaw-iommu-translation-off.md | 7 ++- .../frank-dragoon-flash-server-v2-native.md | 54 +++++++++++-------- serverAppConfigs-docs/LLM/index.md | 30 +++++------ serverAppConfigs-docs/index.md | 5 +- serverAppConfigs-docs/log.md | 36 +++++++++++++ 7 files changed, 115 insertions(+), 64 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e604e71..eabf47f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved). | Target | Service | Runtime | Model | Port | |--------|---------|---------|-------|------| | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 | -| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 on 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; ~10.8 GiB reserved where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770| +| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 and DEPLOYED 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; **11.0 GiB reserved, measured on that deploy** (the preflight formula bills 11.3) where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770| | Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 | Retired endpoints (`frank-npu`, and every prior runtime) are in @@ -407,10 +407,10 @@ MAXIMUM-length conversations DO coexist. The PER-REQUEST ceiling stays `HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context" bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒ -the ledger reserves "about 10.8 GiB", where 200,704 cost 5.5, 262,144 cost 7.2 -and the 2026-10-06 shape's 524,288 measured 14.6; the engine's preflight FORMULA -bills slightly more than the reservation (block scratch + ~115 MiB/slot of O(1) -state) — quote the reservation. +the 2026-10-18 deploy RESERVES **11.0 GiB** (`reserving the KV pool, 401408 +positions (about 11.0 GiB)`; the preflight formula bills 11.3 with block scratch ++ ~115 MiB/slot of O(1) state), where 200,704 cost 5.5, 262,144 cost 7.2 and +the 2026-10-06 shape's 524,288 measured 14.6 — quote the reservation. `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table). @@ -428,10 +428,12 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content + "video". **Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights -62.9 (trunk 62.11 + vision 0.84) + KV ~10.8 + working 5.7 -(`3.8 + 8.4 × MAX_TOK/16384`) ≈ **79.4 GiB in all**, the ~1.0 GiB prompt cache -booked separately by the ledger, + `HALOGEN_HOST_RESERVE_GIB=20` → ~22 GiB -margin over baseline co-tenancy (the pool-equals-ctx boot printed 74.1 GiB with +62.9 (trunk 62.11 + vision 0.84) + KV **11.0** + working 5.7 +(`3.8 + 8.4 × MAX_TOK/16384`) = **79.6 GiB in all** (boot ledger, the +401,408-position deploy of 2026-10-18), the **1.3 GiB** prompt cache (12 entries +× 111 MiB at ctx 200704) booked separately by the ledger, + `HALOGEN_HOST_ +RESERVE_GIB=20` → ~20 GiB margin over baseline co-tenancy (`MemAvailable` read +22 GiB right after the raise; the pool-equals-ctx boot printed 74.1 GiB with ~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why every pool position is a double cost: **RAM the pool takes is RAM that table diff --git a/scripts/frank-dragoon.env b/scripts/frank-dragoon.env index e8444e5..5d8a0cc 100644 --- a/scripts/frank-dragoon.env +++ b/scripts/frank-dragoon.env @@ -68,10 +68,11 @@ # ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the # checkpoint's native window is 262,144 and the engine refuses to go past it # without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position -# (FLAGS.md) ⇒ ~10.8 GiB, where pool == ctx (200,704) cost 5.5, 262,144 cost -# 7.2 and the 2026-10-06 shape's 524,288 measured 14.6. The engine's preflight -# formula bills a little more than the reservation (block scratch + ~115 -# MiB/slot of O(1) state) — quote the RESERVATION. FLAGS.md law: "RAM the pool +# (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the +# KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704) +# cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6. +# The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1) +# state) — quote the RESERVATION. FLAGS.md law: "RAM the pool # takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this # much page cache, so the warm prompt-cache ratio is what to re-measure. # HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP @@ -82,14 +83,16 @@ # Memory ledger against the box's 121.2 GiB unified (the n-gram table is # mmap'd page cache beside the checkpoint, not a pinned charge — the boot # ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"): -# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~10.8 + -# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 79.4 GiB engine-side, with the -# prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked -# separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host -# honest. Against the 121.2 GiB box that leaves ~22 GiB over the reserve for -# baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot -# ledger is the referee — read it after deploy and re-size if the engine -# prints a clamp. +# weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 + +# working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot +# ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB +# prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by +# the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against +# the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable +# read 22 GiB right after the raise) for baseline co-tenancy +# (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the +# referee — read it after deploy and re-size if the engine prints a +# clamp. # HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy # turns the MTP head off for whole requests after low acceptance and # HALVES decode as a result. Frank accepted the head always-on trade diff --git a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md index a8aae63..e7d74e5 100644 --- a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md +++ b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md @@ -202,10 +202,9 @@ boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all answered 200. (Later the same day the pool DOUBLING to 401,408 positions for -slot parity was COMMITTED — `2b12716` — but as of 2026-10-18 NOT yet -redeployed: the live boot still reserves ONE 200,704-position pool (5.5 GiB), -host `frank-dragoon.env` included. When it lands, the reservation is ~10.8 -GiB and the engine-side total ~79.4 GiB; +slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the +boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total +is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working), see [[frank-dragoon-flash-server-v2-native]].) **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/ diff --git a/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md b/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md index 02a9285..66228c9 100644 --- a/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md +++ b/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md @@ -94,9 +94,9 @@ the pool: **two long conversations can no longer share the endpoint** (two of ≤100k can). That is acceptable here because frank-dragoon carries the `slow` (medium-thinking) role and the `security-reviewer` subagent, not the interactive workhorse (that is frank-archon). -Frank lifted exactly that concurrency limit on 2026-10-18 — see *Pool -doubled for slot parity* below; that lift is COMMITTED but not yet deployed, -so the sentences above remain the live truth until the next redeploy. +Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is +**deployed** (section below): the pool is again `slots × ctx`, so two +maximum-length conversations fit the endpoint at once. **Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk, @@ -115,7 +115,7 @@ that long prompts read. `HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk stays pinned. Disk-backing the weights is not a lever on this format. -## Pool doubled for slot parity — committed 2026-10-18, awaiting redeploy +## Pool doubled for slot parity — committed AND deployed 2026-10-18 `HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's call: frank-dragoon's total in-flight context was **half** frank-zealot's — @@ -133,24 +133,28 @@ conversation had to stay under ~100k (commit message; (`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top of the legal band, and exactly frank-zealot's pool number — parity, not approximation. -- Expected reservation **~10.8 GiB** at ~28 KiB/position (preflight formula, - `scripts/frank-dragoon.env:70-72`); the measured points beside it are 5.5 - GiB at 200,704 (boot ledger 2026-10-18), 7.2 GiB reserved / 7.4 GiB billed - at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288 +- Measured reservation **11.0 GiB** — the boot ledger prints `reserving the KV + pool, 401408 positions (about 11.0 GiB)`. The preflight KV-budget line bills + **11.3 GiB** (`~28 KiB/position incl. block scratch + ~115 MiB/slot`) plus + 1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704, + 7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288 (`scripts/frank-dragoon.env:49-52`, `:74-76`). - That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the pool takes is RAM that table loses. -**Deploy state as of 2026-10-18: NOT yet deployed.** The live container -(booted 2026-10-18 01:25) still runs `HALOGEN_KV_POOL_POSITIONS=200704` — -verified in its `flash_serve` `/proc//environ` and in the host-deployed -`frank-dragoon.env` — and the 2026-10-18 boot ledger reserves `the KV pool, -200704 positions (about 5.5 GiB)` with `74.1 GiB in all` -(`/home/dank/logs/frank_dragoon.log`). Until the next -`./scripts/deploy.sh dankclaw --service frank-dragoon`, the boot ledger keeps -printing `2 slots over ONE 200704-position KV pool`; the first boot at the -new pool must show `ONE 401408-position KV pool` and the ~10.8 GiB -reservation — read that line before quoting 401,408 as live anywhere. +**Deployed 2026-10-18** (`./scripts/deploy.sh dankclaw --service +frank-dragoon`, Playwright gate 33/33). The boot ledger confirms the shape: +`reserving the KV pool, 401408 positions (about 11.0 GiB)`, +`2 slots over ONE 401408-position KV pool, each request up to 200704 (it +reserves prompt + max_tokens)`, and the sum +`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB +of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12 +entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram +47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable` +fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid +in page cache. **Not re-measured since the raise:** the warm prompt-cache ratio +(136.7× on the pre-raise boot) — that is the number to watch if a long warm +turn starts costing full prefill again. ## Measured A/B — same battery, both boots (2026-10-17) @@ -307,9 +311,17 @@ restart; `webui.db.bak-20261017b` holds the pre-cutover row). **What the cut cost:** with pool == ctx one maximum-length request fills the pool, so two 200k conversations never coexist (two of ≤100 k do), and the harness window is now 155,648 — the 180 k compaction target of the 262,144 era -is unreachable because the window itself is smaller. (The pool == ctx -concurrency limit is superseded — on redeploy only — by *Pool doubled for slot -parity* above; the window math stands, the pool never moves the ceiling.) +is unreachable because the window itself is smaller. (That pool == ctx +concurrency limit IS superseded: the doubling deployed the next day — see +*Pool doubled for slot parity* above. The window math stands, though: the pool +never moves the per-request ceiling.) **What it bought:** 1.7 GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the 2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry. + +**Third production boot — the `401408` pool, deployed 2026-10-18.** Same ctx, +same working set, same 111 MiB prompt-cache entry; the pool line reads +`reserving the KV pool, 401408 positions (about 11.0 GiB)` and the total +`79.6 GiB in all`. Ledger and the measured cost in +*Pool doubled for slot parity* above; host `MemAvailable` after that boot: +22 GiB. diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index 637ad30..2226059 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -22,14 +22,15 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2 digest serving the NATIVE v2 conversion of the abliterated Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine, MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE -200,704-position KV pool at ctx 200704 — the 2 × ctx = 401,408 raise is -committed 2026-10-18 but NOT yet deployed, vision ON)** and +401,408-position KV pool — 2 × ctx, raised and DEPLOYED 2026-10-18 — at ctx +200704, vision ON)** and **llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml -Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48 -Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled, +Ternary-Bonsai-2-9B at **PQ2_0** — 9B qwen35 hybrid (8 full-attention + 24 +Gated DeltaNet layers + the blk.32 MTP block), THREE 262,144-token +conversations, MTP present in the file but OFF by measurement, q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via -the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom -tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the +the publisher's own Q8_0 mmproj, on the RTX 3090 — and the custom +tensor type 141 + `prism.hadamard.*` metadata mean no other loader can open the file)**. > 2026-08-22: the former `vllmScripts` repository is merged into the @@ -58,11 +59,11 @@ gfx1151 iGPU 114 GiB GTT] E[flash-server container gfx1151, digest-pinned image frank-dragoon, port 7770] F[flash-server v2 native abliterated .hgn + ngram table + vision tower -ONE 200704-position KV pool at ctx 200704 (401408 committed, awaiting deploy), 2 slots, MTP always-on, image input] +ONE 401408-position KV pool at ctx 200704 = 2 slots x ctx, MTP always-on, image input] G[llama.cpp PrismML fork CUDA build frank-zealot, port 7771] - H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj -2 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision] + H[Ternary-Bonsai-2-9B PQ2_0 GGUF + Q8_0 mmproj +3 x 262144 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision] E2 --> E --> F E2 --> G --> H end @@ -94,13 +95,10 @@ frank-zealot, port 7771] `HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**): **one pool behind two slots**, sized so TWO maximum-length requests coexist — raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total - in-flight context, ~10.8 GiB reserved against the card's 262,144/262,144 = - 7.2 GiB. **Deploy state 2026-10-18: the running container and the - host-deployed env still carry the 200,704-position pool** (boot ledger - `reserving the KV pool, 200704 positions (about 5.5 GiB)`; host - `frank-dragoon.env` `HALOGEN_KV_POOL_POSITIONS=200704`) — the doubling lands - on the next redeploy (commit `2b12716`). - longer window: the per-request ceiling stays 200,704, because 262,144 is the + in-flight context and **deployed the same evening**: the boot ledger reserves + `401408 positions (about 11.0 GiB)` with `79.6 GiB in all` + (`scripts/frank-dragoon.env:70-75`). It bought CONCURRENCY, not a longer + window: the per-request ceiling stays 200,704, because 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout 65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5 diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 1c66ecc..d8c0356 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -314,8 +314,9 @@ The LLM inference fleet on the two hosts. Full index: since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2 abliterated** conversion of orcarouter's pinned BF16 (`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a - **401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, ~10.8 GiB — one - pool behind both slots, raised to 2 × ctx on 2026-10-18 so TWO maximum-length + **401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, 11.0 GiB + reserved — one pool behind both slots, raised to 2 × ctx and deployed + 2026-10-18 so TWO maximum-length requests coexist) and the stock peonist n-gram/vision companions **hardlinked** in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP always-on, vision ON, no server-side sampler change diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index f24d044..a827b29 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -2190,3 +2190,39 @@ every restart since the cutover. 200,704). Note the columns are `params`/`meta` JSON — the table has no `context_length`/`description` columns. + +## 2026-10-18 — frank-dragoon: the 401,408-position pool goes live; docs re-keyed to the measured boot + +- `./scripts/deploy.sh dankclaw --service frank-dragoon` — e2e 33/33. The + drop-in check the zealot incident bought ran FIRST: `DropInPaths` empty, so + the repo env governed. Boot ledger: `reserving the KV pool, 401408 positions + (about 11.0 GiB)`, `2 slots over ONE 401408-position KV pool, each request up + to 200704`, and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of + working memory, 79.6 GiB in all`; prompt cache ON (12 entries × 111 MiB at + ctx 200704 = 1.3 GiB), n-gram 47.68 GiB still `Pin::None`, no clamp line. +- The price is exactly the FLAGS.md collision: host `MemAvailable` fell + **26 → 22 GiB**, because every pool position is RAM the n-gram table's page + cache loses. The warm prompt-cache ratio (136.7× measured pre-raise) was NOT + re-measured — recorded in the runbook as the number to watch. +- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the + engine reserves **11.0 GiB** and its preflight bills **11.3**. Re-keyed in + AGENTS.md (topology row, KV-pool law, memory ledger), the `frank-dragoon.env` + lore block, both index pages, the iommu page and the runbook, and every + "committed but not yet deployed" clause was retired with the measurement. +- The same sweep caught the LLM index still describing frank-zealot as + **Bonsai-2-27B at 2 × 200,704** in its intro paragraph and topology diagram — + the 9B swap of the same day had not reached those two spots. Re-keyed: 8 + full-attention + 24 Gated DeltaNet + the `blk.32` MTP block, 3 × 262,144, + tensor type **141**. +- PTQ1_0 provenance corrected: `scripts/download-frank-zealot-bonsai2-ptq1.sh` + IS committed, so the armed-but-never-measured arm that hijacked the zealot + deploy was a **repo-tracked candidate**, not an orphan. The runbook now states + the decision it still owes: measure it in a real window, or record it + declined. +- OpenWebUI: both decoration rows re-keyed in the live DB. The values live + inside JSON columns — `params.context.length` and `meta.description` — not in + fields named for them, and there is no `sqlite3` binary on the host, so + `python3` stdlib + `Connection.backup()` (`webui.db.bak-20261018`). Zealot + 200,704 → **262,144** with the 9B description; dragoon 262,144 → **200,704**, + which had kept the pre-lowering window while its own description said + 200,704.