diff --git a/AGENTS.md b/AGENTS.md index d740025..792288a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1169,7 +1169,7 @@ Retired (rollback targets and history): | radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` | | llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` | | llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` | -| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, 2 slots × 200,192 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` | +| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, TWO slots at ctx 262,144 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` | | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` | | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` | diff --git a/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md b/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md index 23d167c..e265b17 100644 --- a/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md +++ b/serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md @@ -185,3 +185,37 @@ mandatory); unknown sampling params are a 400; the prompt cache is on by default, so a warm 100k follow-up costs seconds. See [[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that carries verbatim, and [[deployment]] for the deploy/health gate. + +## Production boot — the deploy ledger (2026-10-17 22:20, `fcd7630`) + +`./scripts/deploy.sh dankclaw --service frank-dragoon`: healthy after ~12 s, +deploy marker `fcd7630`, e2e gate 33/33. The trial drop-in went to +`~/.AGENTTRASH/20261017-v2abl-trial/` FIRST, so this boot is governed by the +repo env — `systemctl show -p DropInPaths` is empty. + +Ledger lines that settle the shape (engine stdout, not the journal): + +|Line|Figure| +|---|---| +|preflight|`MemAvailable now 105.4 GiB` against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache| +|weights|`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)`| +|KV pool|`reserving the KV pool, 262144 positions (about 7.2 GiB)`; `the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions`; slots `1 -> 2 POOLED, 111.0 MiB each of O(1) state`| +|working memory|5.7 GiB at `HALOGEN_MAX_TOK 8192`| +|total|`75.8 GiB in all` — ~25 GiB of the 121.2 GiB box left for co-tenants| +|n-gram|`47.68 GiB from /models/qwen38-flash-next-ngram.hgn`, `checkpoint: mapped 51.2 GB, NOT pinned (Pin::None)` — disk-backed exactly as designed| +|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, `HALOGEN_CACHE_INPLACE=1` (the KV rows are never copied)| +|spec decode|`MTP depth-2 selectable (drafter 1, speculates while alone)`| +|clamps|none — the requested pool is the delivered pool| + +Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB +(~28 KiB/position **including** block scratch and ~115 MiB/slot of O(1) state) +while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the +RAM ledger and the formula when comparing pool sizes. + +Live probes on the deployed endpoint: `27*59` → `1593` with a warm thinking +channel, tool call `{"city": "Seattle"}`, 512 px solid-red PNG → **Red** (320 +prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s +on the FIRST request after boot, i.e. the warmup artifact; the trial's settled +figure is 50.0. OpenWebUI's `frank-dragoon` row description re-keyed to the +native payload (backup `webui.db.bak-20261017b`; no container restart forced — +forcing it logs every family session out for one cosmetic string). diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index 992e9f3..2c07100 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -98,7 +98,7 @@ frank-zealot, port 7771] stay on disk - [[frank-dragoon-flash-server-v2-swift15]] — **RETIRED 2026-10-06 → 10-17 (rollback target: payload + fetch script on disk)**: the Swift1.5-finetuned - v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots × 200,192 ctx over + v2 `.hgn` (compacted r2, 66,687,678,464 B) at TWO slots at ctx 262,144 over a 524,288-position pool; owns the 400-zone law (the harness chars/4 under-count against the engine ceiling), the MTP-always-on law, and the sampler / prompt-cache / vision contract the native era carries verbatim diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 2dec8db..8cff73b 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -1896,3 +1896,61 @@ every restart since the cutover. hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook header corrected (was still "LIVE", never updated at retirement) and its Rollback table rebuilt. + +## 2026-10-17 — frank-dragoon cutover: native v2 abliterated payload, card's memory-saving pool + +- Ask: move frank-dragoon to `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated` + with the card's recommended settings, save memory, keep the n-gram table + disk-backed. Census first: the endpoint carries only the `slow` + + `security-reviewer` roles (the interactive workhorse is frank-archon), the + table was ALREADY disk-backed (`mapped 51.2 GB, NOT pinned (Pin::None)`), and + `HALOGEN_FLASH_PIN_TRUNK=0` is rejected by the v2 loader — so the only real + lever was the KV pool: 524,288 → the card's 262,144. +- **Filename trap caught before the fetch:** this repo name carried two + different artifacts. The writer-transplant era's file measured + 66,687,678,464 B; the native 2026-10-11 conversion is 66,687,678,432 B — + 32 bytes apart. Byte-exact gate + LFS sha256 `b5bd925f…` in + `scripts/download-frank-dragoon-v2abl.sh`, which also HARDLINKS the peonist + n-gram/vision/tokenizer trio (the release publishes only the main file; a + hardlink is a regular file, so the no-symlink serve gate passes without a + 47.7 GiB copy). +- **Companion risk adjudicated by the engine, not by argument.** The card does + NOT establish the stock peonist companions as matched to a native conversion + and publishes no source-matched pair, so the A/B ran the checkpoint tools on a + stopped service under one knob set (`scripts/census-frank-dragoon.sh`): + `verify` OK, **NIAH 18/18 vs 17/18**, PPL 3.968 vs 3.877 (+2.3 % — the + finetune's edge is gone), KL vs the swift15 logits mean 0.155 / median 0.057 + with top-1 85.1 % = two sound models disagreeing on content, not a corrupt + forward pass. Behavior battery (`scripts/battery-frank-dragoon.py`) on both + boots: refusals 0/5 both, tools/vision identical, needles hit at both depths, + decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 (+12.6 %), thinking + tokens 1,476 vs 784 (the finetune compressed reasoning ×1.9). +- Boot window run through a transient `FRANK_DRAGOON_ENV_FILE` drop-in — never + a scratch port (one iGPU, one server) — with the stop+mask law and a + trap-guarded restore (`scripts/trial-frank-dragoon.sh`). Masking fails on a + relinked unit file, as documented; stop alone was enough. +- Deployed at 22:20 (`fcd7630`, `--service frank-dragoon`): healthy in ~12 s, + e2e 33/33. Production ledger: 62.9 GiB weights locked (0.8 vision) + 7.2 GiB + KV (ONE 262,144-position pool, 6,656 MiB, two slots) + 5.7 GiB working = + **75.8 GiB in all**, no clamp line, n-gram 47.68 GiB read from disk, prompt + cache ON with 12 entries, MTP depth-2. The trial drop-in was moved to + `~/.AGENTTRASH/20261017-v2abl-trial/` before the deploy so the repo env + governs (`DropInPaths=` empty). Live probes: `27*59` → 1593, tool call + `{"city": "Seattle"}`, 512 px red PNG → Red (320 prompt tokens). +- Files moved: env (payload dir/checkpoint/bytes/sha256/pool + the measured + A/B in the header), serve script + deployer (paths, fetch hints), unit + `Description`, seam (`frank_dragoon_unit.bats`: pool law relaxed from + `pool == slots × ctx` to `ctx <= pool <= slots × ctx`, fetch-script name), + new runbook `frank-dragoon-flash-server-v2-native`, swift15 page re-banned + RETIRED, both indexes + AGENTS.md re-keyed, homepage card, `models.yml` + comment (numbers unchanged: window 217,088 / cap 45,056 — the per-REQUEST + ceiling did not move). OpenWebUI `frank-dragoon` row description re-keyed + (backup `webui.db.bak-20261017b`, no restart forced). +- Two stale claims fixed on the way: the retired-era rows said `200,192` ctx + (that ceiling was raised to 262,144 on 2026-10-07, inside the same era), and + the root index still called the halogen-27B page ACTIVE. +- Cost of the swap, stated plainly: two MAXIMUM-length conversations no longer + share the endpoint (one 262,144-position pool behind both slots), PPL +2.3 %, + thinking ×1.9. Bought: 7.4 GiB of RAM back as page cache for the table long + prompts read, +12.6 % cold prefill, retrieval 18/18, the base checkpoint + itself instead of the finetune.