Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3f5ad507c2 | ||
|
|
4607604ef7 | ||
|
|
2def6282b5 | ||
|
|
986bbc9e36 | ||
|
|
f48b45c1fa | ||
|
|
322ebb1658 | ||
|
|
7706e06775 | ||
|
|
f383038437 | ||
|
|
64185d63b9 | ||
|
|
430c42ab18 | ||
|
|
3ccc1e554f | ||
|
|
7a69c31128 | ||
|
|
f96898d4c5 | ||
|
|
3e140cd077 | ||
|
|
c5ac0725f7 | ||
|
|
f44c39812f | ||
|
|
6dcf4a5913 |
No files matched your search
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
|
|
||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||||
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10754**, commit `2459f68b5`; root systemd unit, relinked — the ONLY loader for this payload) | `prism-ml/Ternary-Bonsai-2-27B-gguf` **PQ2_0** (7,206,168,928 B, sha256 `3907dc16…`; `qwen35` hybrid: **64 blocks, 16 full-attention + 48 Gated DeltaNet**, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP head in the file**; `general.file_type 141` = custom tensor type **142** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (629,246,976 B; Qwen3-VL tower, 27 blocks, 768 px, `qwen3vl_merger`) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **q8_0 K + q8_0 V** under flash attention (34 KiB/token over the 16 full layers = 13,328 MiB pool; **f16 KV needs 25,088 MiB and cannot boot this card**), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (no head in the file). Boot 22,664 used / 1,461 free; 76.6 t/s single-stream decode, 691.7 t/s prefill at 166k. Cutover 2026-10-17 from Qwen3.5-9B-Uncensored Q6_K (same shape; payload + master CUDA build on disk as rollback). |
|
||||||
|
|
||||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||||
the [runbook index](#runbook-index). The full swap history lives in
|
the [runbook index](#runbook-index). The full swap history lives in
|
||||||
@@ -429,101 +429,125 @@ Rollback: revert the swap commit and redeploy — the 27B payload
|
|||||||
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
|
||||||
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||||
|
|
||||||
### frank-zealot — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
|
### frank-zealot — llama.cpp (PrismML fork) on the RTX 3090 (Dankclaw, ACTIVE)
|
||||||
|
|
||||||
**llama.cpp master** (0.5.0-dev, commit `5fc4f3c`; CUDA sm_86 build
|
**The PrismML fork build of llama.cpp** (vendor release, `--version` =
|
||||||
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
`0.2.0-dev (build 10754, commit 2459f68b5)`;
|
||||||
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
`/home/dank/GitPackages/llama-prism/llama-server`) as a **root systemd unit**
|
||||||
(`scripts/frank-zealot.service`, `relink_unit`ed into
|
(`scripts/frank-zealot.service`, `relink_unit`ed into
|
||||||
`/etc/systemd/system/`), serving
|
`/etc/systemd/system/`), serving `prism-ml/Ternary-Bonsai-2-27B-gguf` at
|
||||||
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
**PQ2_0** (7,206,168,928 B, sha256 `3907dc16…` = the Hub LFS oid) **plus the
|
||||||
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
publisher's own Q8_0 mmproj** (629,246,976 B), both
|
||||||
LFS oid) **plus the publisher's own BF16 mmproj** (921,704,448 B), both
|
`/home/dank/models/Bonsai2-PQ2_0/`, on port 7771. Cutover 2026-10-17 from the
|
||||||
`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`, on port 7771. Cutover
|
Qwen3.5-9B-Uncensored Q6_K era (same shape).
|
||||||
2026-10-07 (late) from the Gemma4-26B-A4B Q4_K_M 1×200,704 shape (same day).
|
|
||||||
|
|
||||||
**Architecture: qwen35 dense hybrid** — read from the GGUF's own header, not
|
**The runtime is part of the payload identity.** The file is
|
||||||
the model card: 32 layers, `full_attention_interval = 4` ⇒ **8 full-attention
|
`general.file_type 141` (PQ2_0, 2.13 bpw, group 128) with **402 tensors of the
|
||||||
layers (3, 7, 11, …, 31) + 24 Gated DeltaNet linear layers**, 16 attention
|
custom GGML type 142** plus `prism.hadamard.*` metadata (normalized
|
||||||
heads / **4 KV heads** × 256 dim, `ssm` conv_kernel 4 / state_size 128 /
|
Sylvester–Walsh transform, block 1024, 401 named weights, inverse on
|
||||||
group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144, vocab
|
`token_embd`, `gdn_v_grouped`). Stock llama.cpp refuses type 142 at load, and a
|
||||||
248,320. Tensor types are Q6_K × 250 + F32 × 177 — standard quants only, so
|
loader that ignored the Hadamard metadata would dequant wrongly — same lock class
|
||||||
the stock master build loads it (arch `qwen35` is registered in this build's
|
as the ROCmFPX era. Retargeting the payload without the fork ships an endpoint
|
||||||
`src/llama-arch.cpp`). **KV accounting:** the pool scales with the 8 full
|
that cannot load, so the seam pins both paths.
|
||||||
layers only — 8 × 4 × 256 = 8,192 elements/token per cache, so f16 K + f16 V
|
|
||||||
bills **32 KiB/token** = 12.25 GiB at the 401,408 pool; the 24 DeltaNet
|
|
||||||
layers bill a fixed recurrent state per SLOT, not per token, which is exactly
|
|
||||||
why two 200k slots fit where the 26B MoE fit one.
|
|
||||||
|
|
||||||
**The shape (2026-10-07):** `--ctx-size 401408 --parallel 2` = TWO
|
**Architecture: qwen35 hybrid** — read from the GGUF's own header: **64
|
||||||
|
blocks**, `full_attention_interval = 4` ⇒ **16 full-attention + 48 Gated
|
||||||
|
DeltaNet layers**, 24 attention heads / **4 KV heads** × 256 dim, embedding
|
||||||
|
5,120, ffn 17,408, `ssm` conv_kernel 4 / state_size 128 / group_count 16 /
|
||||||
|
time_step_rank 48 / inner_size 6,144, native 262,144, vocab 248,320. **KV
|
||||||
|
accounting:** the pool scales with the 16 full layers — 16 × 4 × 256 = 16,384
|
||||||
|
elements/token per cache, so **q8_0 K + q8_0 V bills 34 KiB/token** = 13,328
|
||||||
|
MiB at the 401,408 pool (ledger-confirmed to the MiB), while **f16 KV needs
|
||||||
|
25,088 MiB and cannot boot on a 24,123 MiB card** (measured `cudaMalloc` OOM).
|
||||||
|
The 48 DeltaNet layers bill a fixed recurrent state per SLOT — ~150 MiB each —
|
||||||
|
so pool arithmetic alone under-prices the shape.
|
||||||
|
|
||||||
|
**The shape (2026-10-17):** `--ctx-size 401408 --parallel 2` = TWO
|
||||||
conversations of 200,704 (Frank's cap; the harness `contextWindow` quotes
|
conversations of 200,704 (Frank's cap; the harness `contextWindow` quotes
|
||||||
the per-slot number), `--flash-attn on`, **f16 K + f16 V** (exact KV),
|
the per-slot number), `--flash-attn on`, **q8_0 K + q8_0 V**, batch 4096 /
|
||||||
batch 4096 / **ubatch 2048** (the measured CUDA optimum: +4.6 % prefill,
|
**ubatch 2048** (the measured CUDA optimum carried from the MiMo/Swift A/Bs on
|
||||||
decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
|
this card), `--log-verbosity 4`, `--mmproj` for vision, **speculative decoding
|
||||||
`--mmproj` for vision, **speculative decoding OFF** (`SPEC_TYPE=none`).
|
OFF** (`SPEC_TYPE=none`). Boot: **22,664 MiB used / 1,461 free**; the
|
||||||
|
projection (21,439) excludes the projector's 873.10 MiB worst-case, which
|
||||||
|
lands AFTER the fit check — read settled `nvidia-smi`, never the projection.
|
||||||
|
Measured: 76.6 t/s single-stream decode (the 9B ran 94.4), 1,258 t/s prefill
|
||||||
|
at 35k, 691.7 t/s at 166k, 109.4 t/s aggregate across two concurrent streams.
|
||||||
|
The **first request after boot is a warmup artifact** (2.6 t/s prefill / 37.7
|
||||||
|
t/s decode measured) — never report it.
|
||||||
|
|
||||||
**Why MTP is off**: it is not a choice — **this Q6_K file carries no
|
**Why MTP is off**: the PQ2_0 export carries no MTP head (upstream ships a
|
||||||
`nextn`/`mtp` tensors** (verified from the tensor-name list), and MTP is a
|
separate `…-CRACK-MTP-GGUF`, +451 MB, for that), and MTP is a property of the
|
||||||
property of the file, not of the flags. The publisher card advertises "MTP
|
file, not of the flags.
|
||||||
support" for the base architecture; the quantized export dropped the head.
|
|
||||||
|
|
||||||
**Measured boots** (nvidia-smi, the referee): the deployed boot is
|
**Vision**: the publisher's own Q8_0 mmproj via `--mmproj` (Qwen3-VL tower: 27
|
||||||
**21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
|
blocks, 768 px, patch 16, merge 2, `qwen3vl_merger`, deepstack). Probe traps
|
||||||
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
|
|
||||||
ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
|
|
||||||
bought back what the projection predicted. The engine ledger confirms the KV
|
|
||||||
law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
|
|
||||||
`V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
|
|
||||||
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
|
|
||||||
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
|
|
||||||
vs 104.8 t/s).
|
|
||||||
|
|
||||||
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
|
|
||||||
natively multimodal; the Q6_K file carries no vision tensors). Probe traps
|
|
||||||
carry over: ≥256 px probe images or the model answers "video", and only the
|
carry over: ≥256 px probe images or the model answers "video", and only the
|
||||||
OpenAI `image_url` content part reaches the model.
|
OpenAI `image_url` content part reaches the model. **New trap:** thinking
|
||||||
|
tokens count against `max_tokens`, so a vision probe at 64 tokens returns
|
||||||
|
EMPTY `content` with `finish_reason: "length"` — budget ≥512 or the endpoint
|
||||||
|
looks blind while it is only mid-thought.
|
||||||
|
|
||||||
|
**Base alignment**: Bonsai-2 is the censored BASE lineage (the 9B it replaced
|
||||||
|
was uncensored). The CRACK re-export on disk
|
||||||
|
(`/home/dank/models/Bonsai2-CRACK/`, header census proves byte-identical
|
||||||
|
geometry) is a one-line `MODEL_FILE` swap if a refusal ever surfaces.
|
||||||
|
|
||||||
**Compaction hub**: `models.yml` routes `compactionModel` for
|
**Compaction hub**: `models.yml` routes `compactionModel` for
|
||||||
`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here — 200,704 is the
|
`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here — 200,704 is the
|
||||||
only window in the fleet that fits a worst-case compaction input (~170k
|
only window in the fleet that fits a worst-case compaction input (~170k
|
||||||
tokens). With `--parallel 2` a compaction request no longer serializes behind
|
tokens). With `--parallel 2` a compaction request no longer serializes behind
|
||||||
a live generation (the Gemma4 era's `--parallel 1` law).
|
a live generation (the Gemma4 era's `--parallel 1` law). Retrieval through a
|
||||||
|
25,960-token prompt was verified exact after the swap.
|
||||||
|
|
||||||
|
**Contract law learned at verbosity 4:** a raw `/completion` request prints
|
||||||
|
`failed to parse reasoning content (… parse_reasoning=true)` whenever the
|
||||||
|
model closes a bare prompt with `` — cosmetic
|
||||||
|
(`probe_request.cpp:789`); the `/v1/chat/completions` path parses
|
||||||
|
`reasoning_content` correctly, which is the only path consumers use. Do not
|
||||||
|
lower `VERBOSITY` to silence it: the KV/buffer ledger is what VRAM forensics
|
||||||
|
read.
|
||||||
|
|
||||||
Runbook:
|
Runbook:
|
||||||
`serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md`.
|
`serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md`.
|
||||||
Rollback: revert the swap commit — the Gemma4 payload
|
Rollback: revert the swap commit — the Qwen3.5-9B Q6_K payload
|
||||||
|
(`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`) with the master CUDA build
|
||||||
|
(`~/GitPackages/llama-master/build-cuda/`), the Gemma4 payload
|
||||||
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
||||||
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
||||||
(`/home/dank/models/Swift15-UncIQ3XXS/`) and the Ornith Q8_0 payload
|
(`/home/dank/models/Swift15-UncIQ3XXS/`) and the Ornith Q8_0 payload
|
||||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||||
|
|
||||||
|
|
||||||
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE)
|
||||||
|
|
||||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||||
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
|
for serve; `libr4d` gfx1201 kernels, plain `podman run`, root
|
||||||
systemd unit, container name = unit name) serving
|
systemd unit, container name = unit name) serving the
|
||||||
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
|
**publisher's container of the official Qwen/Qwen3.8-Flash-Next**:
|
||||||
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
|
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` →
|
||||||
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256
|
||||||
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
|
verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe
|
||||||
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
|
`qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a
|
||||||
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
|
16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation
|
||||||
with the token knob; the RADGRAM2 grams are format-identical across pins and
|
(E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with
|
||||||
cross the boundary by design), **24 whole experts per layer cast bf16** (the
|
per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps),
|
||||||
PROTECTED set, derived from the calibration grams' energy share),
|
**10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head**
|
||||||
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
|
(scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at
|
||||||
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
|
fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the
|
||||||
The plugin's **one-table-one-encoding law** (measured at rad-convert
|
container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K
|
||||||
declare): one expert table takes ONE encoding; only whole bf16 experts are
|
positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %.
|
||||||
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
|
**MODEL_FILE is the only serving argument that moved** (2026-10-15); every
|
||||||
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
|
number of the 2026-10-13 shape below carries verbatim. The plugin's
|
||||||
reference (recorded on the same 1.0.13 pin) is the precision referee;
|
**one-table-one-encoding law** (fleet-measured at rad-convert declare):
|
||||||
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
|
one expert table takes ONE encoding; only whole bf16 experts are legal
|
||||||
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
|
beside it. Rollback: revert the swap commit AND re-download the in-house
|
||||||
Rollback: revert the swap commit — the fp8-native `.rad`
|
w4dn8 `.rad` (KLD-gated Swift1.5 conversion) — it is **no longer on disk**
|
||||||
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
(purged 2026-10-17); the byte-exact copy is published as
|
||||||
|
`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
|
||||||
|
(LFS sha256 `09dd9306…`). The fp8-native `.rad` was deleted 2026-10-15
|
||||||
|
(rebuild: `scripts/download-frank-archon.sh` +
|
||||||
|
`scripts/build-frank-archon-rad.sh`).
|
||||||
|
|
||||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
||||||
`--max-model-len 200000` (the harness contextWindow must quote this),
|
`--max-model-len 200000` (the harness contextWindow must quote this),
|
||||||
@@ -575,7 +599,8 @@ accept 0.7246).
|
|||||||
|
|
||||||
**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**:
|
**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**:
|
||||||
`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so
|
`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so
|
||||||
llama.cpp's exact-K/quantized-V pair (what frank-zealot runs) has NO radiance
|
llama.cpp's exact-K/quantized-V pair (the 9B-era frank-zealot ran it; the
|
||||||
|
Bonsai-2 swap runs q8_0/q8_0) has NO radiance
|
||||||
equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from
|
equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from
|
||||||
the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00
|
the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00
|
||||||
KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA
|
KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA
|
||||||
@@ -630,8 +655,9 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
|||||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||||
|
|
||||||
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`.
|
||||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
Retired Swift1.5 w4dn8 era (payload purged 2026-10-17; rollback = HF re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
||||||
|
Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
||||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
||||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
||||||
@@ -1019,6 +1045,31 @@ linked runbooks.
|
|||||||
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
`Unexpected reasoning effort high` until `high: xhigh` was added on
|
||||||
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
2026-09-16. Halogen accepts the wider set, so the two entries legitimately
|
||||||
differ.
|
differ.
|
||||||
|
- **The vision projector is NOT billed in llama.cpp's own fit check.** The
|
||||||
|
engine prints `[mtmd] estimated worst-case memory usage of mmproj is
|
||||||
|
873.10 MiB` and then fits WITHOUT it: frank-zealot's Bonsai-2 boot projected
|
||||||
|
21,439 MiB and settled at 22,664 (2026-10-17). Price a 3090-class shape from
|
||||||
|
settled `nvidia-smi` after a real boot, never from
|
||||||
|
`model + context + compute` — the projector (and any drafter context) lands
|
||||||
|
after `/health` would already have answered. Same failure class as the
|
||||||
|
DSpark drafter OOM.
|
||||||
|
- **The FIRST request after a boot is a warmup artifact.** Measured on
|
||||||
|
Bonsai-2/3090: 2.6 t/s prefill and 37.7 t/s decode on request #1, then
|
||||||
|
1,258 t/s prefill and 54.7 t/s (per stream, two concurrent) on the next — a
|
||||||
|
3× spread on the same shape. Benchmark the second request or later, or the
|
||||||
|
graph-capture cost reads as a model fault.
|
||||||
|
- **Thinking tokens count against `max_tokens`, so a short-budget vision probe
|
||||||
|
reads as blindness.** The Bonsai-2/`image_url` probe at `max_tokens: 64`
|
||||||
|
returned `content: ""` with `finish_reason: "length"` and half a thought in
|
||||||
|
`reasoning_content` — the image HAD reached the model (322 prompt tokens vs
|
||||||
|
42 for the text alone). At 512 tokens the same image answers "Red". Budget
|
||||||
|
≥512 for any thinking model you probe for vision.
|
||||||
|
- **Quantized KV is not the only dtype law a fork build can hide.** Vendor
|
||||||
|
builds carry their own compile options: frank-zealot's PrismML prebuilt
|
||||||
|
served `--cache-type-v q8_0` under flash attention at the exact 34,816
|
||||||
|
B/token (the `GGML_CUDA_FA_ALL_QUANTS` door), so a build decision must be
|
||||||
|
proven by the KV buffer line in the boot ledger, not by the build flags you
|
||||||
|
remember passing.
|
||||||
|
|
||||||
**Ops**
|
**Ops**
|
||||||
|
|
||||||
@@ -1090,8 +1141,8 @@ Active:
|
|||||||
| Service | Runbook |
|
| Service | Runbook |
|
||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp PrismML fork, Bonsai-2-27B PQ2_0, 2×200,704 q8_0 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md` |
|
||||||
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1099,6 +1150,8 @@ Retired (rollback targets and history):
|
|||||||
|
|
||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
|
| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
|
|||||||
@@ -1,5 +1,16 @@
|
|||||||
# Server App Configs
|
# Server App Configs
|
||||||
|
|
||||||
|
### AI Disclosures:
|
||||||
|
|
||||||
|
- Slop rating: 2/5 - Actively using, actively de-slopping, **repository mostly human owned**, lacks formal process.
|
||||||
|
- Completeness: Unfinished, planning planned, no plan yet
|
||||||
|
- Datacenter model/spend: Less than $20 of Deepseek 4.1 Flash and GLM-5.3-Flash
|
||||||
|
- Local model: Qwen 3.8 Flash Next
|
||||||
|
|
||||||
|
### slop rating of 2 or lower indicates human reading is a worthwhile exercise, and that Frank has read the code and knows the architecture.
|
||||||
|
### slop rating of 2 indicates that while this project is human owned, no human contribution process exists and/or release is agentically driven.
|
||||||
|
|
||||||
|
|
||||||
The single repository for the two homelab servers: the Podman container
|
The single repository for the two homelab servers: the Podman container
|
||||||
stacks **and** the LLM inference fleet. One repo, one checkout per host,
|
stacks **and** the LLM inference fleet. One repo, one checkout per host,
|
||||||
one deploy script.
|
one deploy script.
|
||||||
|
|||||||
@@ -27,8 +27,7 @@ CALIB=/home/dank/corpora/calib
|
|||||||
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||||
LOG=/home/dank/logs/swift15_convert.log
|
LOG=/home/dank/logs/swift15_convert.log
|
||||||
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
|
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
|
||||||
# the format assigns ONE encoding per expert table, so a per-tensor block-fp8
|
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
|
||||||
# down tier cannot be served beside a w4 gate_up tier -- see the recipe header).
|
|
||||||
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
||||||
|
|
||||||
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
||||||
|
|||||||
+31
-26
@@ -947,7 +947,7 @@ deploy_plex() {
|
|||||||
# accepted and then answered with nothing (curl 52, Empty reply) against a
|
# accepted and then answered with nothing (curl 52, Empty reply) against a
|
||||||
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full
|
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full
|
||||||
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered
|
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered
|
||||||
# 200 in 44 ms the whole time. The same requirement is why the Caddyfile dials the
|
# 200 in 44 ms the whole time. The same law is why the Caddyfile dials the
|
||||||
# plex upstream over TLS (tests/caddyfile.bats pins it).
|
# plex upstream over TLS (tests/caddyfile.bats pins it).
|
||||||
# -k: PMS presents its own self-signed cert on 32400.
|
# -k: PMS presents its own self-signed cert on 32400.
|
||||||
#
|
#
|
||||||
@@ -1207,9 +1207,8 @@ deploy_llm_dragoon() {
|
|||||||
# on the FULL container name, which appears in the argv of a manual
|
# on the FULL container name, which appears in the argv of a manual
|
||||||
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
|
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
|
||||||
# this deployer's own command line (`bash deploy-remote.sh
|
# this deployer's own command line (`bash deploy-remote.sh
|
||||||
# frank-dragoon`) — so the sweep can never match the deployer's own argv
|
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
|
||||||
# (`pkill -f` kills the invoker when the pattern appears in the invoker's
|
# pkill law), and it can never match frank-zealot's llama-server.
|
||||||
# own command line), and it can never match frank-zealot's llama-server.
|
|
||||||
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
||||||
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
||||||
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
||||||
@@ -1223,23 +1222,26 @@ deploy_llm_dragoon() {
|
|||||||
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
|
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
|
# deploy_llm_zealot — frank-zealot is llama.cpp (PrismML fork build) on the
|
||||||
# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
# RTX 3090 as a ROOT unit: the runtime is a native binary tree under
|
||||||
# ~/GitPackages/llama-master/build-cuda, so there is no container image and
|
# ~/GitPackages/llama-prism (vendor release build 10754, commit 2459f68b5 —
|
||||||
# no generated unit to avoid. relink_unit points
|
# the ONLY loader that registers the payload's custom tensor type 142 and
|
||||||
|
# reads its prism.hadamard.* metadata), so there is no container image and no
|
||||||
|
# generated unit to avoid. relink_unit points
|
||||||
# /etc/systemd/system/frank-zealot.service at the repo file and boot
|
# /etc/systemd/system/frank-zealot.service at the repo file and boot
|
||||||
# persistence is that unit's [Install] section.
|
# persistence is that unit's [Install] section.
|
||||||
#
|
#
|
||||||
# It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
|
# It is a SEPARATE endpoint on port 7771; frank-dragoon keeps 7770. The 3090 is
|
||||||
# not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
|
# not shared: frank-dragoon's engine runs on the gfx1151 iGPU.
|
||||||
LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
LLM_ZEALOT_MODEL_FILE="${LLM_ZEALOT_MODEL_FILE:-/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf}"
|
||||||
LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
LLM_ZEALOT_RUNTIME="${LLM_ZEALOT_RUNTIME:-/home/dank/GitPackages/llama-prism/llama-server}"
|
||||||
# The projector IS the vision capability here: the model is natively
|
# The projector IS the vision capability here: the Bonsai-2 text file carries
|
||||||
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
# no vision tensors, so the publisher's own mmproj (Qwen3-VL tower, 27 blocks,
|
||||||
# its own BF16 projector in the same repo, so a deploy that starts without
|
# 768 px, qwen3vl_merger, 629,246,976 B) is the whole image path — a deploy
|
||||||
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
# that starts without it silently ships a text-only endpoint. Checked
|
||||||
# frank-dragoon's.
|
# unconditionally, like frank-dragoon's. Its 873.10 MiB worst-case VRAM lands
|
||||||
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
# AFTER the engine's fit check, so it is not inside the q8_0 budget.
|
||||||
|
LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf}"
|
||||||
|
|
||||||
deploy_llm_zealot() {
|
deploy_llm_zealot() {
|
||||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||||
@@ -1247,15 +1249,15 @@ deploy_llm_zealot() {
|
|||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why.
|
||||||
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
|
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
|
||||||
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
|
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
|
||||||
echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
|
echo "frank-zealot: fetch the PrismML fork release build (build 10754," >&2
|
||||||
echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
|
echo "frank-zealot: commit 2459f68b5) into ~/GitPackages/llama-prism first —" >&2
|
||||||
echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
|
echo "frank-zealot: stock llama.cpp refuses the payload's custom tensor type 142." >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
|
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
|
||||||
if [[ ! -f "$f" ]]; then
|
if [[ ! -f "$f" ]]; then
|
||||||
echo "ERROR: frank-zealot payload file absent at $f" >&2
|
echo "ERROR: frank-zealot payload file absent at $f" >&2
|
||||||
echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
|
echo "frank-zealot: run scripts/download-frank-zealot-bonsai2.sh first;" >&2
|
||||||
echo "frank-zealot: refusing to start a unit that cannot load." >&2
|
echo "frank-zealot: refusing to start a unit that cannot load." >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
@@ -1278,11 +1280,14 @@ deploy_llm_zealot() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
||||||
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
# digest-pinned stilldeadcode/radiance image) serving the PUBLISHER's
|
||||||
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
# 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
|
||||||
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
# (qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2) as a
|
||||||
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
||||||
# the image entrypoint; the image lives in dank's rootless podman storage).
|
# the image entrypoint; the image lives in dank's rootless podman storage).
|
||||||
|
# 2026-10-15 payload swap from the in-house w4dn8 (payload on disk =
|
||||||
|
# rollback).
|
||||||
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
||||||
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
||||||
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
||||||
@@ -1291,7 +1296,7 @@ deploy_llm_zealot() {
|
|||||||
# Gates: the image must exist under its digest ref in dank's rootless
|
# Gates: the image must exist under its digest ref in dank's rootless
|
||||||
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
||||||
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
||||||
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
# a truncated fetch is a guaranteed long-timeout boot), and the unit env
|
||||||
# must be complete. Per-file sha256 verification is the fetch script's job
|
# must be complete. Per-file sha256 verification is the fetch script's job
|
||||||
# at download time and rad-convert/rad-info's job at build time, never
|
# at download time and rad-convert/rad-info's job at build time, never
|
||||||
# re-hashed here.
|
# re-hashed here.
|
||||||
@@ -1326,13 +1331,13 @@ deploy_llm_archon() {
|
|||||||
fi
|
fi
|
||||||
if [[ ! -f "$model_dir/$model_file" ]]; then
|
if [[ ! -f "$model_dir/$model_file" ]]; then
|
||||||
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
||||||
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
echo "frank-archon: run scripts/download-frank-archon-iq4r.sh --fetch" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
||||||
if [[ "$mib" -lt "$rad_min" ]]; then
|
if [[ "$mib" -lt "$rad_min" ]]; then
|
||||||
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
||||||
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
|
echo "frank-archon: re-fetch it (scripts/download-frank-archon-iq4r.sh --fetch; delete the staged file first if its sha256 failed)" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
||||||
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
||||||
|
|
||||||
The protection rule (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
||||||
an expert whose share of its layer's down-input energy is at least 3% AND
|
an expert whose share of its layer's down-input energy is at least 3% AND
|
||||||
at least three times its share of the routing is kept bf16; the count must
|
at least three times its share of the routing is kept bf16; the count must
|
||||||
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||||
|
|||||||
Executable
+83
@@ -0,0 +1,83 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# download-frank-archon-iq4r.sh — payload fetch/verify for the frank-archon
|
||||||
|
# swap to the PUBLISHER's 4-bit-codebook container:
|
||||||
|
# StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe — the official
|
||||||
|
# Qwen/Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's own
|
||||||
|
# author into ONE .rad: routed experts as 4-bit codes into a 16-level
|
||||||
|
# non-uniform codebook (Walsh–Hadamard rotation, GPTQ + coordinate-descent,
|
||||||
|
# 10M-token calibration), ten bf16-protected experts per layer, int8 trunk +
|
||||||
|
# lm_head, fp8 HC mixtures, the PLE n-gram table at fp8 fixed scale, the MTP
|
||||||
|
# head, and the bf16 vision tower. File:
|
||||||
|
# qwen3.8-next-flash-fp8-iq4r-moe.rad — 121,969,901,568 B (113.6 GiB,
|
||||||
|
# 116,322 MiB), card KL 0.0789 all-tokens / top-1 92.85 % vs bf16.
|
||||||
|
# Repo is UNGATED — no token needed (never put one on the command line).
|
||||||
|
#
|
||||||
|
# bash scripts/download-frank-archon-iq4r.sh --fetch resume/complete
|
||||||
|
# bash scripts/download-frank-archon-iq4r.sh verify only
|
||||||
|
#
|
||||||
|
# Fetch uses the vendored scripts/hfget.sh (curl+jq only): per-file size
|
||||||
|
# checks against the Hub listing, `-C -` resume, disk-fit gate. The side
|
||||||
|
# files (LICENSE, README, the w4nl recipe) stay in the staging dir as
|
||||||
|
# provenance; the serve gate mounts ONLY /home/dank/models/rad, so the
|
||||||
|
# verified .rad alone moves into it.
|
||||||
|
#
|
||||||
|
# Verify (default) is the AGENTS.md law in two parts: the file lands at the
|
||||||
|
# final path with the Hub's EXACT byte count, and its sha256 equals the
|
||||||
|
# Hub's LFS oid. A truncated transfer is a guaranteed long-timeout boot
|
||||||
|
# without this gate.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
REPO="StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
|
||||||
|
RAD_NAME="qwen3.8-next-flash-fp8-iq4r-moe.rad"
|
||||||
|
STAGE="/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
|
||||||
|
DEST="/home/dank/models/rad/$RAD_NAME"
|
||||||
|
SIZE=121969901568
|
||||||
|
SHA256="0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20"
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
|
mode="verify"
|
||||||
|
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
|
||||||
|
|
||||||
|
if [[ "$mode" == "fetch" ]]; then
|
||||||
|
# HFGET_JOBS=4 is hfget's default (a few streams saturate the link).
|
||||||
|
bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$STAGE" main
|
||||||
|
|
||||||
|
staged="$STAGE/$RAD_NAME"
|
||||||
|
if [[ ! -f "$staged" ]]; then
|
||||||
|
echo "FETCH FAILED: $staged absent after hfget" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
got=$(stat -c %s "$staged")
|
||||||
|
if [[ "$got" != "$SIZE" ]]; then
|
||||||
|
echo "FETCH FAILED: staged .rad is $got B, want $SIZE B" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "staged .rad byte-exact ($SIZE B); hashing (one full read, minutes)"
|
||||||
|
got_sha=$(sha256sum "$staged" | cut -d' ' -f1)
|
||||||
|
if [[ "$got_sha" != "$SHA256" ]]; then
|
||||||
|
echo "FETCH FAILED: sha256 $got_sha, want $SHA256 — DELETE the staged" \
|
||||||
|
"file before re-fetching (curl -C - cannot repair a corrupt body)" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
mkdir -p "$(dirname "$DEST")"
|
||||||
|
mv "$staged" "$DEST"
|
||||||
|
echo "payload placed: $DEST"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "verifying deployed payload: $DEST"
|
||||||
|
if [[ ! -f "$DEST" ]]; then
|
||||||
|
echo " BAD payload absent: $DEST" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
got=$(stat -c %s "$DEST")
|
||||||
|
if [[ "$got" != "$SIZE" ]]; then
|
||||||
|
echo " BAD payload is $got B, want $SIZE B" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "byte-exact ($SIZE B); hashing (one full read, minutes)"
|
||||||
|
got_sha=$(sha256sum "$DEST" | cut -d' ' -f1)
|
||||||
|
if [[ "$got_sha" != "$SHA256" ]]; then
|
||||||
|
echo " BAD sha256 $got_sha, want $SHA256" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "payload verified: $RAD_NAME, size and sha256 match the Hub LFS oid"
|
||||||
@@ -14,7 +14,7 @@
|
|||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) enforces the download contract in three parts: every listed file
|
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size, the shard count matches the weight-map
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
||||||
# only the public API.
|
# only the public API.
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) enforces the download contract in three parts: every listed file
|
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size, the shard count matches the weight-map
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
||||||
# only the public API.
|
# only the public API.
|
||||||
|
|||||||
@@ -4,10 +4,9 @@
|
|||||||
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
||||||
# card pins), and the flat tokenizer the two repos share. All public, no
|
# card pins), and the flat tokenizer the two repos share. All public, no
|
||||||
# token — and a token never rides a command line: argv is readable by every
|
# token — and per the fleet secret law, none would ever touch argv.
|
||||||
# process on the host through /proc, so a key placed there is a key exposed.
|
|
||||||
#
|
#
|
||||||
# The fetch discipline, same as every sibling script: curl -f (an access error
|
# The fetch law, same as every sibling script: curl -f (an access error
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# must not become a "successful" 153-byte artifact), exact byte count from
|
||||||
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an
|
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an
|
||||||
# existing payload without downloading.
|
# existing payload without downloading.
|
||||||
@@ -27,8 +26,7 @@
|
|||||||
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
||||||
# already on disk (tokenizer.json LFS
|
# already on disk (tokenizer.json LFS
|
||||||
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
||||||
# fetched fresh rather than symlinked, so the payload dir is
|
# fetched fresh rather than symlinked, per the no-symlink law.
|
||||||
# self-contained — every gate and every mount sees real files inside it.
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
||||||
|
|||||||
@@ -1,11 +1,10 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
||||||
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
||||||
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and a token never
|
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
|
||||||
# rides a command line: argv is readable by every process on the host
|
# secret law, none would ever touch argv).
|
||||||
# through /proc, so a key placed there is a key exposed).
|
#
|
||||||
#
|
# The fetch law, same as every sibling script: curl -f (an access error
|
||||||
# The fetch discipline, same as every sibling script: curl -f (an access error
|
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# must not become a "successful" 153-byte artifact), exact byte count from
|
||||||
# the model card, LFS sha256 verification, --verify-only to re-check an
|
# the model card, LFS sha256 verification, --verify-only to re-check an
|
||||||
# existing payload without downloading.
|
# existing payload without downloading.
|
||||||
|
|||||||
Executable
+146
@@ -0,0 +1,146 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# download-frank-dragoon-v2abl.sh — fetch frank-dragoon's replacement payload:
|
||||||
|
# the CURRENT native-v2 abliterated checkpoint from
|
||||||
|
# Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated (the 2026-10-11
|
||||||
|
# release: a complete HIP native-v2 conversion of orcarouter's pinned BF16
|
||||||
|
# source, NOT the r1/r2 writer transplant this filename used to carry), and
|
||||||
|
# assemble the companions the engine needs beside it — the REQUIRED n-gram
|
||||||
|
# lookup table, the vision tower and the flat tokenizer. Public files, no
|
||||||
|
# token; per the fleet secret law none would ever touch argv.
|
||||||
|
#
|
||||||
|
# The fetch law, same as every sibling script: curl -f (an access error must
|
||||||
|
# not become a "successful" 153-byte artifact), exact byte count from the
|
||||||
|
# script's manifest, LFS sha256 verification, --verify-only to re-check an
|
||||||
|
# existing payload without downloading.
|
||||||
|
#
|
||||||
|
# COMPANION LAW (read this before shipping the new checkpoint): the current
|
||||||
|
# release publishes ONLY the main file. Its card states plainly that the
|
||||||
|
# interchangeability of the stock peonist companions with this native
|
||||||
|
# conversion "has NOT been established" — the source-derived n-gram would be
|
||||||
|
# 51,200,246,084 B while peonist's stock table is 51,200,246,144 B (60 B
|
||||||
|
# apart, so whole-file identity cannot be assumed), and the source-derived
|
||||||
|
# vision file is the same size as stock but equal size is not proof of equal
|
||||||
|
# content. The six tokenizer assets ARE published with matching blob
|
||||||
|
# identities. No source-matched n-gram/vision pair is published anywhere, so
|
||||||
|
# the only companions available are peonist's — which is also what the
|
||||||
|
# historical r2 recipe pinned, and what the engine's own table-mismatch
|
||||||
|
# check gets to judge at boot. This script therefore hardlinks the verified
|
||||||
|
# peonist trio (a hardlink is a REGULAR file, so the serve gate's
|
||||||
|
# no-symlink law holds and 47.7 GiB is not copied), verifies every byte, and
|
||||||
|
# leaves the verdict to the boot ledger plus the behavior probes.
|
||||||
|
#
|
||||||
|
# File manifest (Quat3rnion @main, verified against the Hub API on
|
||||||
|
# 2026-10-18):
|
||||||
|
# qwen38-flash-next-v2-abliterated.hgn 66,687,678,432 B sha256
|
||||||
|
# b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
||||||
|
# qwen38-flash-next-ngram.hgn 51,200,246,144 B sha256
|
||||||
|
# 3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
||||||
|
# (peonist @5cc17cea; REQUIRED — the engine reads it beside the
|
||||||
|
# checkpoint)
|
||||||
|
# qwen38-flash-next-vision.hgn 897,916,416 B sha256
|
||||||
|
# d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
||||||
|
# (peonist @e053f488; optional on the engine, enabled here for vision)
|
||||||
|
# tokenizer/* @e053f488 — the six assets the two repos share.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-v2-abliterated}"
|
||||||
|
# Where the already-verified peonist companions live (the swift15-era dir).
|
||||||
|
DONOR="${FRANK_DRAGOON_DONOR_DIR:-$HOME/models/halogen-flashnext-v2}"
|
||||||
|
V2ABL="https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated/resolve/main"
|
||||||
|
PEONIST="https://huggingface.co/peonist-ai/halogen-qwen3.8-flash-next/resolve"
|
||||||
|
NGRAM_REV="5cc17cea1a10b502b4a5db41d8d2b1a0f22dba26"
|
||||||
|
TOK_REV="e053f488b120b99ed2525e6ac99f68c51b3b6179"
|
||||||
|
|
||||||
|
FILES=(qwen38-flash-next-v2-abliterated.hgn
|
||||||
|
qwen38-flash-next-ngram.hgn
|
||||||
|
qwen38-flash-next-vision.hgn
|
||||||
|
tokenizer/tokenizer.json
|
||||||
|
tokenizer/chat_template.jinja
|
||||||
|
tokenizer/generation_config.json
|
||||||
|
tokenizer/merges.txt
|
||||||
|
tokenizer/vocab.json
|
||||||
|
tokenizer/tokenizer_config.json)
|
||||||
|
SIZES=(66687678432
|
||||||
|
51200246144
|
||||||
|
897916416
|
||||||
|
12809320
|
||||||
|
8952
|
||||||
|
202
|
||||||
|
3353259
|
||||||
|
6722759
|
||||||
|
17928)
|
||||||
|
SHAS=(b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
|
||||||
|
3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
|
||||||
|
d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
|
||||||
|
0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3
|
||||||
|
- - - - - -)
|
||||||
|
|
||||||
|
# Fetch source per file: repo base + revision (first entry = V2ABL @main).
|
||||||
|
SRCS=("$V2ABL"
|
||||||
|
"$PEONIST/$NGRAM_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV"
|
||||||
|
"$PEONIST/$TOK_REV")
|
||||||
|
|
||||||
|
VERIFY_ONLY=0
|
||||||
|
[ "${1:-}" = "--verify-only" ] && VERIFY_ONLY=1
|
||||||
|
|
||||||
|
mkdir -p "$DEST"
|
||||||
|
|
||||||
|
for i in "${!FILES[@]}"; do
|
||||||
|
f="${FILES[$i]}"; want="${SIZES[$i]}"; sha="${SHAS[$i]}"; base="${SRCS[$i]}"
|
||||||
|
path="$DEST/$f"
|
||||||
|
mkdir -p "$(dirname "$path")"
|
||||||
|
if [ "$VERIFY_ONLY" -eq 0 ] && [ ! -f "$path" ] && [ -f "$DONOR/$f" ]; then
|
||||||
|
# Companion reuse: a hardlink costs no disk and no copy, and stays a
|
||||||
|
# regular file for the serve gate. Only the sha256 below decides.
|
||||||
|
if [ "$(stat -c %s "$DONOR/$f")" = "$want" ]; then
|
||||||
|
ln "$DONOR/$f" "$path"
|
||||||
|
echo "LINK $f from $DONOR (hardlink, $want bytes)"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ "$VERIFY_ONLY" -eq 0 ]; then
|
||||||
|
# A file already at the published size skips the fetch entirely:
|
||||||
|
# resuming it would ask the CDN for a range past EOF (416 -> curl
|
||||||
|
# error under set -e), and re-pulling 62 GiB to prove a point is not
|
||||||
|
# what --verify-only is for. sha256 below still decides.
|
||||||
|
if [ -f "$path" ] && [ "$(stat -c %s "$path")" = "$want" ]; then
|
||||||
|
echo "HAVE $f ($want bytes) — skipping fetch"
|
||||||
|
else
|
||||||
|
echo "FETCH $f -> $path"
|
||||||
|
# HF answers /resolve/ with a 302 to the Xet CDN and curl DROPS
|
||||||
|
# the Range header across redirects — a naive -C - then
|
||||||
|
# "resumes" by pulling the ~1 KB redirect body. Resolve the
|
||||||
|
# effective URL first, then resume against the CDN, which
|
||||||
|
# honors Range.
|
||||||
|
url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$base/$f") \
|
||||||
|
|| { echo "cannot resolve $base/$f" >&2; exit 1; }
|
||||||
|
curl -fL --retry 5 -C - -o "$path" "$url"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
[ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
|
||||||
|
if [ -L "$path" ]; then
|
||||||
|
echo "SYMLINK $path — the serve gate refuses symlinks; replace with a real file" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
got="$(stat -c %s "$path")"
|
||||||
|
[ "$got" = "$want" ] || { echo "SIZE $path: $got != $want" >&2; exit 1; }
|
||||||
|
echo "OK $f ($got bytes)"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "byte counts match; verifying sha256 where the Hub publishes an LFS oid..."
|
||||||
|
for i in "${!FILES[@]}"; do
|
||||||
|
f="${FILES[$i]}"; sha="${SHAS[$i]}"
|
||||||
|
if [ "$sha" = "-" ]; then
|
||||||
|
echo "OK $f (plain git blob, size-checked)"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
got="$(sha256sum "$DEST/$f" | cut -d' ' -f1)"
|
||||||
|
[ "$got" = "$sha" ] || { echo "SHA256 $f: $got != $sha" >&2; exit 1; }
|
||||||
|
echo "OK $f sha256"
|
||||||
|
done
|
||||||
|
echo "DOWNLOAD_DONE $DEST"
|
||||||
Executable
+93
@@ -0,0 +1,93 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Fetch candidate payload for frank-zealot: prism-ml/Ternary-Bonsai-2-27B-gguf
|
||||||
|
# at **PQ2_0** (the ternary g128 pack stored 2 bits/trit + fp16 scale per 128)
|
||||||
|
# plus the publisher's Q8_0 vision projector. Public repo — no auth header,
|
||||||
|
# and never a token on the command line (world-readable in /proc/<pid>/cmdline).
|
||||||
|
#
|
||||||
|
# Why PQ2_0 and not PTQ1_0: measured on the same silicon family (community
|
||||||
|
# 3090 Ti report, PrismML-Eng/Bonsai-demo) PTQ1_0 is ~17 % slower decode and
|
||||||
|
# ~48 % slower prefill than PQ2_0 on Ampere; the model card says the same
|
||||||
|
# (PTQ1_0 wins only on Ada/L4). The pack is consumed packed — 6.71 GiB
|
||||||
|
# resident, never expanded to f16.
|
||||||
|
#
|
||||||
|
# Header facts (HTTP range parse, not the card): arch qwen35, 64 blocks,
|
||||||
|
# full_attention_interval 4 => 16 full-attention + 48 Gated DeltaNet layers,
|
||||||
|
# 4 KV heads x 256 dim, native 262,144, vocab 248,320, and NO nextn/mtp
|
||||||
|
# tensors (block_count is 64 real layers) — so in-file MTP is impossible and
|
||||||
|
# SPEC_TYPE=none is forced by the payload. 402 tensors carry the custom type
|
||||||
|
# id 142 (GGML_TYPE_PQ2_0, group 128): STOCK llama.cpp refuses this file, and
|
||||||
|
# loads the repo's F16 file silently with garbage output — only the
|
||||||
|
# PrismML-Eng/llama.cpp fork applies the prism.hadamard.* activation
|
||||||
|
# transform declared in this file's metadata.
|
||||||
|
#
|
||||||
|
# These ARE LFS files (unlike the HauhauCS payload): the Hub publishes the
|
||||||
|
# oid, and both shas below are the Hub's LFS sha256. The mmproj sha has also
|
||||||
|
# been recomputed locally against the copy in /home/dank/models/Bonsai2-CRACK/.
|
||||||
|
#
|
||||||
|
# NOT pinned by this script: /home/dank/models/Bonsai2-CRACK/
|
||||||
|
# Bonsai-2-27B-PQ2_0-CRACK.gguf (7,206,168,928 B, sha256 5b24ea3eebc3e0bc...,
|
||||||
|
# fetched 2026-10-06 by an undocumented ad-hoc path). Same size as the
|
||||||
|
# official file, different bytes — a dealign/CRACK (uncensored) variant of the
|
||||||
|
# same ternary pack. Provenance unverified; it is a candidate arm, not this
|
||||||
|
# script's payload.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# scripts/download-frank-zealot-bonsai2.sh [DEST_DIR] [--verify-only]
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
DEST=""
|
||||||
|
VERIFY_ONLY=0
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
--verify-only) VERIFY_ONLY=1 ;;
|
||||||
|
*) DEST="$arg" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
DEST="${DEST:-/home/dank/models/Bonsai2-PQ2_0}"
|
||||||
|
|
||||||
|
# repo | hub-name | dest-name | bytes | sha256 (Hub LFS oid)
|
||||||
|
FILES=(
|
||||||
|
"prism-ml/Ternary-Bonsai-2-27B-gguf|Ternary-Bonsai-2-27B-PQ2_0.gguf|Ternary-Bonsai-2-27B-PQ2_0.gguf|7206168928|3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1"
|
||||||
|
"prism-ml/Ternary-Bonsai-2-27B-gguf|Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf|Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf|629246976|6807ede61d570bb86ba34b756a0fa109edc33668604de867c6ea6d8f1d631903"
|
||||||
|
)
|
||||||
|
|
||||||
|
mkdir -p "$DEST"
|
||||||
|
status=0
|
||||||
|
for entry in "${FILES[@]}"; do
|
||||||
|
IFS='|' read -r repo hubname destname size sha <<<"$entry"
|
||||||
|
url="https://huggingface.co/${repo}/resolve/main/${hubname}"
|
||||||
|
f="$DEST/$destname"
|
||||||
|
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]] \
|
||||||
|
&& sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||||
|
echo "OK $destname"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if (( VERIFY_ONLY )); then
|
||||||
|
echo "MISSING $destname"
|
||||||
|
status=1
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
echo "FETCHING $destname (from $repo)"
|
||||||
|
tmp="$f.part"
|
||||||
|
# No resume: `curl -C -` once produced an oversized file on a 31 GiB
|
||||||
|
# payload. -f/--fail-with-body so an HTTP 401/403 body is never a success.
|
||||||
|
curl -fsSL --fail-with-body --retry 3 --retry-delay 5 -o "$tmp" "$url"
|
||||||
|
mv "$tmp" "$f"
|
||||||
|
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
|
||||||
|
echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
|
||||||
|
status=1
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||||
|
echo "SHA FAIL $destname"
|
||||||
|
status=1
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
echo "VERIFIED $destname"
|
||||||
|
done
|
||||||
|
if (( status == 0 )); then
|
||||||
|
echo "DOWNLOAD_DONE $DEST"
|
||||||
|
else
|
||||||
|
echo "DOWNLOAD_FAILED $DEST" >&2
|
||||||
|
fi
|
||||||
|
exit "$status"
|
||||||
@@ -11,8 +11,8 @@ Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
|
|||||||
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
||||||
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
||||||
|
|
||||||
What this instrument shows (75 s window, 4 mixed streams, 2.7M
|
Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed
|
||||||
expert-tensor samples): per-BLOCK read volume is FLAT
|
streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT
|
||||||
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
|
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
|
||||||
routing fires every block every token — there are no "most active layers" at
|
routing fires every block every token — there are no "most active layers" at
|
||||||
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
|
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
|
||||||
|
|||||||
+64
-44
@@ -1,7 +1,15 @@
|
|||||||
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving
|
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
|
||||||
# ukisai/Swift1.5-Qwen3.8-Flash-Next converted to GPTQ w4 (MODEL_FILE below,
|
# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
|
||||||
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
|
# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on
|
||||||
# dankserver :7770.
|
# BOTH R9700s (TP=2), dankserver :7770.
|
||||||
|
# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
|
||||||
|
# of the Swift1.5 finetune to the engine author's published container
|
||||||
|
# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
|
||||||
|
# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
|
||||||
|
# spare). NO argument changed: every number below is the 2026-10-13 shape,
|
||||||
|
# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
|
||||||
|
# disk).
|
||||||
|
|
||||||
#
|
#
|
||||||
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
|
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
|
||||||
# with the per-card budget spent on resident experts: 25,600 MiB of slab +
|
# with the per-card budget spent on resident experts: 25,600 MiB of slab +
|
||||||
@@ -13,9 +21,11 @@
|
|||||||
# +25 % at the full window) while KV beyond the contract buys nothing.
|
# +25 % at the full window) while KV beyond the contract buys nothing.
|
||||||
# Consequence, deliberate: a FOURTH full-height conversation does not fit this
|
# Consequence, deliberate: a FOURTH full-height conversation does not fit this
|
||||||
# pair, and the engine refuses it — it never truncates. See the KV block for the
|
# pair, and the engine refuses it — it never truncates. See the KV block for the
|
||||||
# arithmetic and the measured per-dtype cost. The fp8-native payload this file was
|
# arithmetic and the measured dtype law. The w4dn8 payload is the swap
|
||||||
# written for is the rollback target and keeps its own trio under the
|
# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload
|
||||||
# fp8-native runbook.
|
# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free
|
||||||
|
# the volume for this download — rebuildable via the fetch + build scripts
|
||||||
|
# named in the fp8-native runbook).
|
||||||
#
|
#
|
||||||
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
||||||
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
@@ -30,20 +40,30 @@
|
|||||||
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
||||||
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
||||||
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
||||||
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
|
# THE 4-bit-codebook publisher shape is the DEPLOYED payload since
|
||||||
# --host-pool-mib 12288) remains the documented fallback if the native
|
# 2026-10-15. The card's own run line (--host-pool-mib 12288,
|
||||||
# container proves unbootable.
|
# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's
|
||||||
|
# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
|
||||||
#
|
#
|
||||||
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
|
# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the
|
||||||
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
|
# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's
|
||||||
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
|
# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a
|
||||||
# staging dir whose config.json drops `quantization_config`: the descriptor
|
# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation,
|
||||||
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
|
# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten
|
||||||
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
|
# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC
|
||||||
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL
|
||||||
# containers carry. Per-weight encodings are data, not the descriptor —
|
# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by
|
||||||
# removing the key changes nothing about what the weights hold. Built by
|
# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the
|
||||||
# scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
|
# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the
|
||||||
|
# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
|
||||||
|
# KLD-gated) stays on disk. Behind it, the fp8-native
|
||||||
|
# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
|
||||||
|
# one-time rm, to free the volume for this payload); rebuild =
|
||||||
|
# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
|
||||||
|
# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
|
||||||
|
# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
|
||||||
|
# config.json drops quantization_config; per-weight encodings are data, not
|
||||||
|
# the descriptor).
|
||||||
#
|
#
|
||||||
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
||||||
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
||||||
@@ -59,12 +79,12 @@
|
|||||||
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
||||||
# models.yml's extraBody set is fully inside it. Unknown params are
|
# models.yml's extraBody set is fully inside it. Unknown params are
|
||||||
# refused (400), which is why nothing unverified rides the wire.
|
# refused (400), which is why nothing unverified rides the wire.
|
||||||
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
|
# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe
|
||||||
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
|
# names the mtp.fc_hidden/fc_embedding/draft_head rules; card default
|
||||||
# publisher's throughput peak for this model. Stated explicitly with
|
# depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below —
|
||||||
# --num-speculative-tokens 3.
|
# the depth this shape's decode ladder was measured at.
|
||||||
# - The fused TP all-reduce is declared over max-num-seqs x
|
# - The fused TP all-reduce is declared over max-num-seqs x
|
||||||
# (1 + --num-speculative-tokens) rows; 2 x 3 = 6 rows, under the 25-row
|
# (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row
|
||||||
# exactness bound of the wht6 wire.
|
# exactness bound of the wht6 wire.
|
||||||
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
||||||
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
||||||
@@ -88,7 +108,7 @@ FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd7
|
|||||||
# .rad files; the raw HF download and its staging dir live beside it and
|
# .rad files; the raw HF download and its staging dir live beside it and
|
||||||
# are never mounted.
|
# are never mounted.
|
||||||
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
||||||
FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
|
||||||
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
||||||
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
||||||
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
||||||
@@ -124,12 +144,13 @@ FRANK_ARCHON_TP=2
|
|||||||
FRANK_ARCHON_TP_WIRE=wht6
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
|
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
|
||||||
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
||||||
# block below). A runtime that compiles its kernels specialized to the exact
|
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
||||||
# batch shape can refuse a slot count it has never compiled for; a
|
# tcclaviger vLLM fork's torch.compile shape specialization on
|
||||||
# shape-agnostic engine accepts any count that fits memory. Radiance is the
|
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
||||||
# latter: seqs 2 boots, and measured decode at seqs 2 and 3 is statistically
|
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
||||||
# tied (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys
|
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
||||||
# concurrency, not speed, so a slot costs KV, never decode.
|
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
|
||||||
|
# not speed, so a slot costs KV, never decode.
|
||||||
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
|
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
|
||||||
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
|
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
|
||||||
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
|
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
|
||||||
@@ -241,9 +262,10 @@ FRANK_ARCHON_VRAM_KV_MIB=5120
|
|||||||
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
|
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
|
||||||
# many conversations as it did at bf16 — no re-size needed to park three slots.
|
# many conversations as it did at bf16 — no re-size needed to park three slots.
|
||||||
FRANK_ARCHON_KV_DTYPE=fp8
|
FRANK_ARCHON_KV_DTYPE=fp8
|
||||||
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
# Spec decode: the checkpoint's own MTP head, served at depth 2 — the
|
||||||
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
# publisher's card ships depth 3 as its peak; this fleet's ladder was
|
||||||
# flag states it so the ledger records the intent).
|
# measured at 2 and the swap keeps every argument, so 2 stays (rows
|
||||||
|
# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
|
||||||
FRANK_ARCHON_SPEC_TOKENS=2
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
|
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
|
||||||
@@ -254,10 +276,9 @@ FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
|||||||
# available with the service loaded).
|
# available with the service loaded).
|
||||||
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
||||||
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
# The PLE n-gram table (47.7 GiB) serves from pinned host RAM the cards
|
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
||||||
# read directly, so it cannot be evicted or page-faulted mid-token. `ram`
|
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
|
||||||
# placement REFUSES the start if the memlock limit is too small, and the
|
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
|
||||||
# unit's LimitMEMLOCK=infinity is the matching half of that pairing.
|
|
||||||
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||||
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
|
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
|
||||||
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
@@ -278,9 +299,8 @@ FRANK_ARCHON_TTM_TARGET_MIB=204800
|
|||||||
# multi-flag values as one string; the serve script expands it unquoted on
|
# multi-flag values as one string; the serve script expands it unquoted on
|
||||||
# purpose.
|
# purpose.
|
||||||
FRANK_ARCHON_EXTRA_ARGS=
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a
|
||||||
# guaranteed long-timeout boot instead of a fast failure). The converted
|
# guaranteed long-timeout boot instead of a fast failure). The deployed
|
||||||
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
# publisher container measures 116,322 MiB (121,969,901,568 B); the
|
||||||
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both.
|
||||||
# tail) - floor sits just under the payload.
|
|
||||||
FRANK_ARCHON_RAD_MIN_MIB=110000
|
FRANK_ARCHON_RAD_MIN_MIB=110000
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
||||||
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
# (the publisher's 4-bit-codebook container of the official
|
||||||
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
# Qwen/Qwen3.8-Flash-Next, qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire),
|
||||||
# frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
|
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 (the
|
||||||
# retired unit/env/serve trio and Dockerfile are in that tag).
|
# swap commit is the revert; its .rad stays on disk).
|
||||||
#
|
#
|
||||||
# Same port, same served alias, same container identity as the unit name;
|
# Same port, same served alias, same container identity as the unit name;
|
||||||
# the harness models.yml id is unchanged. The serve script drives
|
# the harness models.yml id is unchanged. The serve script drives
|
||||||
@@ -19,7 +19,7 @@
|
|||||||
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
||||||
# engine still runs out of dank's rootless podman storage.
|
# engine still runs out of dank's rootless podman storage.
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
Description=frank-archon — radiance Qwen3.8-Flash-Next (publisher iq4r) on 2x R9700 TP=2
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
||||||
|
|||||||
@@ -42,10 +42,10 @@
|
|||||||
# form of the engine's default, so the seam test can pin the product.
|
# form of the engine's default, so the seam test can pin the product.
|
||||||
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
||||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||||
# drafter is NOT slot-locked on this engine: some engines silently
|
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||||
# disable drafting the moment a second slot is enabled, this one does not
|
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||||
# (the card's own quickstart ships 2 slots WITH the head), so the second
|
# quickstart ships 2 slots WITH the head), so the second slot is
|
||||||
# slot is concurrent capacity, not a speculation tax.
|
# concurrent capacity, not a speculation tax.
|
||||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
||||||
# card's own quickstart arithmetic does not bill it):
|
# card's own quickstart arithmetic does not bill it):
|
||||||
@@ -88,9 +88,8 @@
|
|||||||
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
||||||
# amdgpu.noretry=0 (check at deploy).
|
# amdgpu.noretry=0 (check at deploy).
|
||||||
|
|
||||||
# Image: digest-pinned (a tag is a mutable pointer — the same tag can
|
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
||||||
# silently deliver different bits — so the digest pins exactly the bits
|
# is what was qualified). 0.16.2 = the version the model card names as its
|
||||||
# that were qualified). 0.16.2 = the version the model card names as its
|
|
||||||
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
||||||
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||||
# Container identity: kept from the 27B era (unit ExecStop names it).
|
# Container identity: kept from the 27B era (unit ExecStop names it).
|
||||||
|
|||||||
+53
-43
@@ -1,59 +1,69 @@
|
|||||||
# frank-zealot — llama.cpp (master, CUDA sm_86) serving
|
# frank-zealot — llama.cpp (PrismML fork build, CUDA) serving
|
||||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
# prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090, :7771.
|
||||||
# :7771.
|
|
||||||
#
|
#
|
||||||
# 2026-10-07 (Frank): swap from Gemma4-26B-A4B QAT Q4_K_M (one 200,704 slot)
|
# 2026-10-17 (Frank): swap from Qwen3.5-9B-Uncensored Q6_K to the Bonsai-2-27B
|
||||||
# to Qwen3.5-9B Q6_K at TWO 200,704-token conversations, EXACT (f16) KV,
|
# ternary checkpoint at the SAME shape Frank asked for — TWO 200,704-token
|
||||||
# MTP disabled (SPEC_TYPE=none), log verbosity 4.
|
# conversations — with q8_0 K + q8_0 V (f16 is arithmetically impossible here,
|
||||||
|
# see the KV law), MTP off (the file carries no nextn/mtp tensors), verbosity 4.
|
||||||
#
|
#
|
||||||
# Architecture — read from the GGUF's own header (HTTP range read, not the
|
# THE RUNTIME IS A LOCK, NOT A PREFERENCE. The file is general.file_type 141
|
||||||
# model card): arch qwen35, 32 layers, dense 9B, full_attention_interval 4
|
# with 402 tensors of the CUSTOM GGML type 142 (pq2_0) plus prism.hadamard.*
|
||||||
# ⇒ 8 full-attention layers (3,7,11,...,31) + 24 Gated DeltaNet linear layers,
|
# metadata (normalized-Sylvester-Walsh transform, block 1024, 401 named
|
||||||
# 16 attention heads / 4 KV heads × 256 dim, ssm conv_kernel 4 / state_size
|
# weights, inverse on token_embd, gdn_v_grouped). Stock llama.cpp refuses type
|
||||||
# 128 / group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144,
|
# 142 at load; only the PrismML fork registers the type AND reads the Hadamard
|
||||||
# vocab 248,320, file_type 18 (Q6_K). Tensor-type histogram: Q6_K×250 +
|
# metadata. The vendor prebuilt build 10754 (commit 2459f68b5) is what every
|
||||||
# F32×177 — standard types only, so the stock master build loads it (no
|
# number in this file was measured on; sha256 12ae5baba033d8cd…
|
||||||
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
LLAMA_BIN=/home/dank/GitPackages/llama-prism/llama-server
|
||||||
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
MODEL_FILE=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf
|
||||||
|
MMPROJ_FILE=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf
|
||||||
|
# The CRACK re-export (dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF, uncensored,
|
||||||
|
# already on disk at /home/dank/models/Bonsai2-CRACK/) is a ONE-LINE swap: the
|
||||||
|
# header census proves byte-identical geometry (851 tensors, same
|
||||||
|
# {pq2_0:402, f32:353, bf16:96} histogram, 64 blocks, vocab 248,320, same
|
||||||
|
# 8,952-char chat template) and the official file's sha256 is the Hub LFS oid.
|
||||||
|
# Bonsai-2 is the BASE (censored) Qwen3.8 family; if a compaction or subagent
|
||||||
|
# request ever comes back with a refusal, point MODEL_FILE at the CRACK file
|
||||||
|
# and restart — the VRAM ledger below does not move.
|
||||||
#
|
#
|
||||||
# KV rate arithmetic: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
|
# Served model id — the harness/family name. The SERVICE, unit, scripts and
|
||||||
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
|
# this env file keep the frank-zealot name; only the alias on the wire is the
|
||||||
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
|
# model id. Consumers that key on the served id: OpenWebUI (model row,
|
||||||
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
|
# access_grant, api_configs model_ids) and the harness models.yml.
|
||||||
# where the 26B MoE fit only one.
|
|
||||||
#
|
|
||||||
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
|
||||||
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
|
||||||
# 2×200,704 f16 K + f16 V measured 21,276 used / 2,849 free at ubatch 1024
|
|
||||||
# and 21,846 / 2,279 at ubatch 2048. This file is 415 MiB lighter in weights,
|
|
||||||
# its BF16 mmproj is 3 MiB heavier ⇒ projected ~21,430 used / ~2,700 free.
|
|
||||||
# The verbosity-4 boot ledger is the referee.
|
|
||||||
#
|
|
||||||
# Ubatch 2048 is the measured CUDA optimum (measured on the MiMo predecessor
|
|
||||||
# at this exact arch + shape: +4.6 % prefill at 8.9k depth, decode flat, cost
|
|
||||||
# 616 MiB). 1024 is the fallback rung if the boot lands under the 2 GiB floor.
|
|
||||||
LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
|
|
||||||
MODEL_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf
|
|
||||||
MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf
|
|
||||||
# Served model id — the harness/family name (Protoss rename completed to the
|
|
||||||
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
|
||||||
# this day). The SERVICE, unit, scripts and this env file keep the
|
|
||||||
# frank-zealot name; only the alias on the wire changed. Consumers that
|
|
||||||
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
|
||||||
# model_ids) and the harness models.yml.
|
|
||||||
ALIAS=frank-zealot
|
ALIAS=frank-zealot
|
||||||
BIND_ADDR=0.0.0.0
|
BIND_ADDR=0.0.0.0
|
||||||
LISTEN_PORT=7771
|
LISTEN_PORT=7771
|
||||||
# CTX_SIZE is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
|
# CTX_SIZE is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
|
||||||
# per conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
|
# per conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
|
||||||
# the harness contextWindow quotes the PER-SLOT number — a longer prompt is
|
# the harness contextWindow quotes the PER-SLOT number — a longer prompt is
|
||||||
# answered with a 400, never truncated.
|
# answered with a 400, never truncated. The model's native window is 262,144;
|
||||||
|
# 2 × 262,144 at q8_0 projects 25,758 MiB and does NOT fit this card (measured
|
||||||
|
# f16 analogue: 41,118). One 262,144 slot fits with 5,995 MiB free but
|
||||||
|
# serializes compaction, so the pair stays.
|
||||||
CTX_SIZE=401408
|
CTX_SIZE=401408
|
||||||
PARALLEL=2
|
PARALLEL=2
|
||||||
BATCH_SIZE=4096
|
BATCH_SIZE=4096
|
||||||
UBATCH_SIZE=2048
|
UBATCH_SIZE=2048
|
||||||
CACHE_TYPE_K=f16
|
# KV law (from the GGUF header, then confirmed by the boot ledger): 16
|
||||||
CACHE_TYPE_V=f16
|
# full-attention layers (full_attention_interval 4 over 64 blocks) × 4 KV heads
|
||||||
|
# × 256 dim = 16,384 elements per token per cache. q8_0 bills 17,408 B per
|
||||||
|
# cache ⇒ 34,816 B/token = 34 KiB, so the 401,408 pool is 13,328.00 MiB — the
|
||||||
|
# ledger said exactly that. f16 would bill 25,088 MiB and the boot DIES:
|
||||||
|
# `cudaMalloc failed: out of memory` on the 25,088.00 MiB request, engine fit
|
||||||
|
# projection 33,198 vs 23,558 free. The 48 DeltaNet layers bill a fixed
|
||||||
|
# recurrent state per SLOT — ~150 MiB each (context 13,627 − KV 13,328 = 299
|
||||||
|
# for two slots), which is why pool arithmetic alone under-prices the shape.
|
||||||
|
CACHE_TYPE_K=q8_0
|
||||||
|
CACHE_TYPE_V=q8_0
|
||||||
|
# Boot ledger (2026-10-17, this exact shape, nvidia-smi settled):
|
||||||
|
# 22,664 MiB used / 1,461 MiB free of 24,123
|
||||||
|
# fit projection 21,439 = 6,539 model + 13,627 context + 1,272 compute
|
||||||
|
# the mtmd projector's 873.10 MiB worst-case is NOT inside that projection —
|
||||||
|
# it lands after the fit check, which is the whole difference between
|
||||||
|
# "2.6 GiB free on paper" and 1.4 GiB free in reality. Watch it.
|
||||||
|
# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV
|
||||||
|
# measured 16,392 used / 7,733 free at the same 2 × 200,704.
|
||||||
|
# 1024 is the ubatch fallback (−616 MiB, measured on the 9B predecessor at
|
||||||
|
# this card/arch; 2048 was the measured CUDA optimum, +4.6 % prefill).
|
||||||
SPEC_TYPE=none
|
SPEC_TYPE=none
|
||||||
NGL=99
|
NGL=99
|
||||||
# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
|
# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
|
Description=llama.cpp (PrismML build 10754) Ternary-Bonsai-2-27B PQ2_0 GGUF (qwen35 hybrid, 2x200704 slots, q8_0 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-zealot)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
|
|||||||
@@ -29,8 +29,7 @@
|
|||||||
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
||||||
# getattr_static()s each one off the instance, breaking on the first miss.
|
# getattr_static()s each one off the instance, breaking on the first miss.
|
||||||
# So BOTH wrappers need all three attributes.
|
# So BOTH wrappers need all three attributes.
|
||||||
# * Method-resolution order forbids the obvious inheritance: adding
|
# * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with
|
||||||
# SupportsEagle3 as a base dies at class creation with
|
|
||||||
# "Cannot create a consistent method resolution order (MRO) for bases ..."
|
# "Cannot create a consistent method resolution order (MRO) for bases ..."
|
||||||
# (GLM's wrappers already carry four/five Protocol-derived bases). The
|
# (GLM's wrappers already carry four/five Protocol-derived bases). The
|
||||||
# attribute route avoids the MRO entirely. Also proven by simulation:
|
# attribute route avoids the MRO entirely. Also proven by simulation:
|
||||||
|
|||||||
@@ -9,8 +9,7 @@ No scratch container, no mask: the service under test is the service.
|
|||||||
Four probes, in increasing cost:
|
Four probes, in increasing cost:
|
||||||
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
||||||
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
||||||
Measurement discipline: generate at least 128 tokens (a short run
|
The bench law applies: >=128 generated tokens, greedy, real text.
|
||||||
measures first-token noise, not the server), greedy, real text.
|
|
||||||
B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as
|
B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as
|
||||||
the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this
|
the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this
|
||||||
host), each carrying a unique marker at its head. All must answer their own
|
host), each carrying a unique marker at its head. All must answer their own
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
||||||
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
||||||
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
||||||
# through GPTQ into the w4nl64a8h table. The format assigns exactly ONE encoding per table
|
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
||||||
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
||||||
# beside it", qwen4exp_fp8 rad_arch_declare — so the original split tier (down block-fp8 +
|
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
||||||
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
||||||
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
||||||
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
||||||
@@ -26,8 +26,8 @@
|
|||||||
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
||||||
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
||||||
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
||||||
# this section from Swift's OWN calibration recording before this recipe is ever run, by the
|
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
||||||
# protection rule -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
||||||
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
||||||
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
||||||
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
||||||
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
# Apache-2.0) serving the PUBLISHER's 4-bit-codebook container of the
|
||||||
# NATIVE block-FP8 precision from a rad-convert container
|
# official Qwen/Qwen3.8-Flash-Next (the engine author's own rad-convert of
|
||||||
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
# the bf16 checkpoint: qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire),
|
||||||
# one card; git tag frank-archon-freetoken-pr587-20261005 is the
|
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 of the
|
||||||
# pre-cutover state).
|
# Swift1.5 finetune (payload on disk = rollback; the swap commit is the
|
||||||
|
# revert); 2026-10-05 radiance cutover from FreeToken (git tag
|
||||||
|
# frank-archon-freetoken-pr587-20261005 is the pre-radiance state).
|
||||||
#
|
#
|
||||||
# The run line below IS the engine argv: the image's entrypoint is the
|
# The run line below IS the engine argv: the image's entrypoint is the
|
||||||
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
||||||
|
|||||||
@@ -70,9 +70,7 @@ fi
|
|||||||
|
|
||||||
# Size-exact gates on all three payload files: a truncated curl exits 0 on
|
# Size-exact gates on all three payload files: a truncated curl exits 0 on
|
||||||
# some failures and a partial .hgn loads far enough to look alive, so the
|
# some failures and a partial .hgn loads far enough to look alive, so the
|
||||||
# byte counts from the Hub tree API are the contract: curl exits 0 on an
|
# byte counts from the Hub tree API are the contract (fleet fetch law).
|
||||||
# HTTP error and writes the error body as the file, so a truncated download
|
|
||||||
# is caught by size here and never boots.
|
|
||||||
ck() {
|
ck() {
|
||||||
# ck <env-name> <path> <expected-bytes>
|
# ck <env-name> <path> <expected-bytes>
|
||||||
if [ ! -f "$2" ]; then
|
if [ ! -f "$2" ]; then
|
||||||
|
|||||||
@@ -1,28 +1,30 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
# Serve prism-ml/Ternary-Bonsai-2-27B (PQ2_0 ternary quant) as frank-zealot on
|
||||||
# frank-zealot on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
# Dankclaw's RTX 3090 via llama.cpp (PrismML vendor build 10754, CUDA sm_86).
|
||||||
#
|
#
|
||||||
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
# 2026-10-17 (Frank): Bonsai-2 PQ2_0 swap at TWO 200,704-token conversations
|
||||||
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
# (--ctx-size 401408 --parallel 2) with q8_0 K + q8_0 V — f16 KV does not fit
|
||||||
# room the 26B MoE Gemma4 shape lacked, so quantized V retires. MTP stays off:
|
# this geometry: the engine's own fit projected 33,198 MiB against 23,558 free
|
||||||
# this GGUF carries no MTP tensors at all (verified from its tensor-name
|
# and cudaMalloc failed on the 25,088 MiB pool. MTP stays off: this GGUF
|
||||||
# list), so there is no in-file head to draft with. Verbosity 4 ships so the
|
# carries no MTP tensors (verified from its tensor-name list). Verbosity 4
|
||||||
# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the journal.
|
# ships so the VRAM-fit forensics (KV layout, buffer sizes, graph capture)
|
||||||
|
# land in the journal.
|
||||||
#
|
#
|
||||||
# Architecture: qwen35 — dense 9B, 32 layers with full_attention_interval 4
|
# Architecture: qwen35 — 27B, 64 blocks with full_attention_interval 4 (16
|
||||||
# (8 full-attention layers at 3,7,...,31 + 24 Gated DeltaNet linear layers),
|
# full-attention layers + 48 Gated DeltaNet linear layers), 24 heads / 4 KV
|
||||||
# 16 heads / 4 KV heads × 256 dim, native 262,144, vocab 248,320. The KV pool
|
# heads × 256 dim, native 262,144, vocab 248,320. The KV pool scales with the
|
||||||
# scales with the 8 full layers: 32 KiB/token at f16 K + f16 V (12.25 GiB at
|
# 16 full layers: 34 KiB/token at q8_0 K + q8_0 V (13,328 MiB at the 401,408
|
||||||
# the 401,408 pool); the DeltaNet layers cost a fixed recurrent state per
|
# pool); the DeltaNet layers cost a fixed recurrent state per SLOT (~150 MiB —
|
||||||
# slot. Feasibility is priced from the MiMo-V2.6-Distill-Qwen-9B boots (same
|
# boot-ledger context minus KV). Boot ledger: 22,664 used / 1,461 free, with
|
||||||
# geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279
|
# the Q8_0 projector's 873 MiB landing AFTER the engine's fit projection.
|
||||||
# at ubatch 2048 — the boot ledger, not arithmetic, is the referee.
|
# The runtime is a hard lock, not a preference: the file carries 402 tensors
|
||||||
# Vision comes from the publisher's own BF16 projector (the model is natively
|
# of custom GGML type 142 plus prism.hadamard.* metadata, which only the
|
||||||
# multimodal), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
|
# vendor build registers — stock llama.cpp refuses it at load.
|
||||||
# the documented text-only opt-out.
|
# Vision comes from the publisher's Q8_0 projector, so --mmproj is what gives
|
||||||
|
# this endpoint eyes; MMPROJ_FILE="" is the documented text-only opt-out.
|
||||||
#
|
#
|
||||||
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
# Sampling: llama.cpp takes it per request (the fleet harness sends the file's
|
||||||
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
# own template defaults from models.yml: thinking temp 1.0 / top_p 0.95 / top_k 20).
|
||||||
#
|
#
|
||||||
# Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
|
# Tunables come from frank-zealot.env unless FRANK_ZEALOT_ENV_FILE overrides.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@@ -37,8 +39,8 @@ if [[ -f "$ENV_FILE" ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# --- defaults (used when the env file does not set them) -------------------
|
# --- defaults (used when the env file does not set them) -------------------
|
||||||
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism/llama-server}"
|
||||||
: "${MODEL_FILE:=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
: "${MODEL_FILE:=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf}"
|
||||||
: "${ALIAS:=frank-zealot}"
|
: "${ALIAS:=frank-zealot}"
|
||||||
: "${BIND_ADDR:=0.0.0.0}"
|
: "${BIND_ADDR:=0.0.0.0}"
|
||||||
: "${LISTEN_PORT:=7771}"
|
: "${LISTEN_PORT:=7771}"
|
||||||
@@ -54,20 +56,20 @@ fi
|
|||||||
# throughput-neutral at a fixed ubatch.
|
# throughput-neutral at a fixed ubatch.
|
||||||
: "${BATCH_SIZE:=4096}"
|
: "${BATCH_SIZE:=4096}"
|
||||||
: "${UBATCH_SIZE:=2048}"
|
: "${UBATCH_SIZE:=2048}"
|
||||||
# Exact (unquantized) KV: f16 K + f16 V, restoring the MiMo-era contract that
|
# Quantized KV: q8_0 K + q8_0 V — the only pair that funds 2 × 200,704 on this
|
||||||
# the 26B Gemma4 shape could not afford. 32 KiB/token at this geometry
|
# card. 34 KiB/token at this geometry (16 full-attention layers × 4 KV heads ×
|
||||||
# (8 full-attention layers × 4 KV heads × 256 dim); 12.25 GiB at the 401,408
|
# 256 dim); 13,328 MiB at the 401,408 pool. f16 measured OOM; q4_0 is the
|
||||||
# pool. Explicit either way so the contract is visible in the argv, and
|
# fallback rung (16,392 used / 7,733 free). Explicit either way so the contract
|
||||||
# --flash-attn stays required for any quantized-V fallback (q8_0 = llama.cpp's
|
# is visible in the argv, and --flash-attn stays required for quantized V
|
||||||
# FP8 class; no true fp8 KV dtype exists in this build).
|
# (no true fp8 KV dtype exists in this build).
|
||||||
: "${CACHE_TYPE_K:=f16}"
|
: "${CACHE_TYPE_K:=q8_0}"
|
||||||
: "${CACHE_TYPE_V:=f16}"
|
: "${CACHE_TYPE_V:=q8_0}"
|
||||||
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
|
# Speculative decoding is OFF: this PQ2_0 file carries NO nextn/mtp tensors, so
|
||||||
# there is no in-file head — this kind of drafter exists only as tensors
|
# there is no in-file head (the law: MTP is a property of the file, not of the
|
||||||
# inside the checkpoint file, and no flag can supply one the file lacks.
|
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
|
||||||
# SPEC_TYPE=none strips BOTH speculative flags (they move as a set);
|
# an MTP-bearing CRACK re-export exists upstream (ManniX-ITA, +451 MB) if
|
||||||
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
# decode ever needs it — that is a new payload with its own VRAM bill, never a
|
||||||
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
# flag flip on this file.
|
||||||
: "${SPEC_TYPE:=none}"
|
: "${SPEC_TYPE:=none}"
|
||||||
: "${SPEC_DRAFT_N_MAX:=3}"
|
: "${SPEC_DRAFT_N_MAX:=3}"
|
||||||
: "${SPEC_DRAFT_FILE:=}"
|
: "${SPEC_DRAFT_FILE:=}"
|
||||||
@@ -78,7 +80,7 @@ fi
|
|||||||
: "${VERBOSITY:=3}"
|
: "${VERBOSITY:=3}"
|
||||||
# Vision projector. No colon on purpose: an explicitly empty value means
|
# Vision projector. No colon on purpose: an explicitly empty value means
|
||||||
# "text-only, on purpose" and must NOT be replaced by the default path.
|
# "text-only, on purpose" and must NOT be replaced by the default path.
|
||||||
: "${MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
: "${MMPROJ_FILE=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf}"
|
||||||
|
|
||||||
# Fail loudly rather than half-loading: a missing file is a download that
|
# Fail loudly rather than half-loading: a missing file is a download that
|
||||||
# has not finished, and Restart=always would turn it into a crash loop that
|
# has not finished, and Restart=always would turn it into a crash loop that
|
||||||
|
|||||||
@@ -298,7 +298,7 @@ def main():
|
|||||||
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
||||||
if not scale_law:
|
if not scale_law:
|
||||||
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
||||||
f"down {ddt}{dsh} scale {dsd}{dssh} (128x128 block scales)")
|
f"down {ddt}{dsh} scale {dsd}{dssh} (scale law 128x128)")
|
||||||
return 2
|
return 2
|
||||||
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
||||||
"gs": tuple(gssh), "ds": tuple(dssh)}
|
"gs": tuple(gssh), "ds": tuple(dssh)}
|
||||||
|
|||||||
@@ -43,8 +43,10 @@ As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`**.
|
|||||||
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
||||||
|
|
||||||
The `model` table carries exactly one decoration row, `frank-zealot`
|
The `model` table carries exactly one decoration row, `frank-zealot`
|
||||||
(context length 200,704 = the per-slot window; description: Qwen3.5-9B
|
(context length 200,704 = the per-slot window; description since 2026-10-17:
|
||||||
Q6_K + BF16 vision mmproj).
|
Ternary-Bonsai-2-27B PQ2_0 + Q8_0 vision mmproj — the row's `description` is
|
||||||
|
live DB state, so a payload swap must re-key it in the database too, not only
|
||||||
|
in the repo).
|
||||||
|
|
||||||
A third gate sits *after* the table, and it is the one that bites family
|
A third gate sits *after* the table, and it is the one that bites family
|
||||||
accounts (2026-10-07): a model row is only visible to a non-admin user when
|
accounts (2026-10-07): a model row is only visible to a non-admin user when
|
||||||
|
|||||||
@@ -452,7 +452,8 @@ access is decided entirely by GROUP MEMBERSHIP consumed downstream —
|
|||||||
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
|
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
|
||||||
every human account).
|
every human account).
|
||||||
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
|
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
|
||||||
debo, freya — kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
|
debo, freya, mitch, eric — mitch (pk 22) and eric (pk 23) created
|
||||||
|
2026-10-15, kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
|
||||||
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
|
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
|
||||||
`<user>@franksdank.cloud` + display name +
|
`<user>@franksdank.cloud` + display name +
|
||||||
`is_active=True` + the password set via `set_password`, membership in
|
`is_active=True` + the password set via `set_password`, membership in
|
||||||
|
|||||||
@@ -0,0 +1,154 @@
|
|||||||
|
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
|
||||||
|
|
||||||
|
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||||||
|
|
||||||
|
The in-house w4dn8 conversion of the Swift1.5 finetune
|
||||||
|
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
|
||||||
|
author's published container**:
|
||||||
|
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
|
||||||
|
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
|
||||||
|
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
|
||||||
|
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
|
||||||
|
**every serving argument carried verbatim** from the 2026-10-13 shape.
|
||||||
|
Rollback: re-download the w4dn8 `.rad` from the HF mirror + point
|
||||||
|
`MODEL_FILE` at it (the local copy was purged 2026-10-17, see Rollback).
|
||||||
|
|
||||||
|
## The payload
|
||||||
|
|
||||||
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
|
||||||
|
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
|
||||||
|
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
|
||||||
|
(hfget resume + size + sha256; staging dir
|
||||||
|
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
|
||||||
|
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
|
||||||
|
|
||||||
|
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
|
||||||
|
| --- | --- |
|
||||||
|
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
|
||||||
|
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
|
||||||
|
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
|
||||||
|
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
|
||||||
|
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
|
||||||
|
| Vision | bf16 vision tower, in-container |
|
||||||
|
|
||||||
|
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
|
||||||
|
only frontier model at base alignment; the abliterated family lives on
|
||||||
|
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
|
||||||
|
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
|
||||||
|
agreement 92.85 %.
|
||||||
|
|
||||||
|
## Why this swap
|
||||||
|
|
||||||
|
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
|
||||||
|
refinement is the engine author's own quality ceiling for this
|
||||||
|
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
|
||||||
|
reproducible, where our w4dn8 gate measured against our own reference.
|
||||||
|
- **Formal parity**: the served file is the one the engine author supports
|
||||||
|
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
|
||||||
|
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
|
||||||
|
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
|
||||||
|
shape needed no translation).
|
||||||
|
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
|
||||||
|
endpoint (it returns to the official base alignment); the 4-slot/200k
|
||||||
|
economics and all shape laws carry over unchanged.
|
||||||
|
|
||||||
|
## Shape (unchanged from 2026-10-13)
|
||||||
|
|
||||||
|
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
|
||||||
|
same budget:
|
||||||
|
|
||||||
|
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
|
||||||
|
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
|
||||||
|
- `--placement expert_tiered --host-pool-mib 44928` + explicit
|
||||||
|
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
|
||||||
|
30,969 MiB ceiling; pool = contract + 1.3 %)
|
||||||
|
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
|
||||||
|
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
|
||||||
|
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
|
||||||
|
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
|
||||||
|
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
|
||||||
|
the KV dtype law — both are engine properties, payload-independent)
|
||||||
|
|
||||||
|
The publisher's card run line (`--host-pool-mib 12288`,
|
||||||
|
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
|
||||||
|
shape on stock 32 GB cards; the fleet ships its own measured shape against
|
||||||
|
the same file.
|
||||||
|
|
||||||
|
## Cutover record — 2026-10-15
|
||||||
|
|
||||||
|
1. `rm` (Frank's one-time authorization): the raw
|
||||||
|
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
|
||||||
|
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
|
||||||
|
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
|
||||||
|
payload is **no longer on disk**: rebuild =
|
||||||
|
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
|
||||||
|
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
|
||||||
|
`iq4r-fetch` (detached-job protocol; log
|
||||||
|
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
|
||||||
|
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
|
||||||
|
detached (the swap restarts the very endpoint the operator session runs
|
||||||
|
on; the harness timeout catches the session on return).
|
||||||
|
4. **Boot ledger (2026-10-15, first boot, pin 1.0.13 — the w4nl codebook
|
||||||
|
loaded, no pin bump)**: `kv_attn` fp8 page **26.00 KiB** at **607,740
|
||||||
|
tokens** (151,935 blocks = 3.77 GiB) — byte-identical to the w4dn8 boot,
|
||||||
|
the sizing law carries; `kv_qsa_bkey` 3.25 KiB at the same token count;
|
||||||
|
`kv_gdn_state` 54 MiB × 6 slots, `kv_gdn_conv` 1.76 MiB × 3,
|
||||||
|
`kv_qsa_tail` 19.5 KiB × 6; pool **5.00 GiB**, prefix checkpoints 8 ×
|
||||||
|
56 MiB; the sizing line reads `kv cache 5.00 GiB … backs 3 sequences all
|
||||||
|
at 200000 tokens at once, which is the worst case`; samplers print **9
|
||||||
|
rows** per rank (3 × (1+2), inside the wht6 25-row bound);
|
||||||
|
`speculation: 'mtp' drafts 2 token(s) a step in 2 serial pass(es)` +
|
||||||
|
prompt-lookup fallback; slab **17,610 (rank0) / 17,658 (rank1) slots**
|
||||||
|
(64 shadow) — above the w4dn8 record of 17,548 (ten bf16-protected
|
||||||
|
experts per layer replace twenty-four, so more of the 4-bit table
|
||||||
|
resides), pinned pool 7,532/7,484 slots; KV lending **4.22 GiB**
|
||||||
|
(3,571/3,568 mover slots); arenas 896.02/988.17 MiB charged before the
|
||||||
|
pair split; prefix cache RECOVERED 900,176 blocks + 126 checkpoint slots
|
||||||
|
from the 64 GiB disk tier (same arch + tokenizer as w4dn8 — the tier
|
||||||
|
survived the payload swap); froggeric template 28,234 B; vision encoder
|
||||||
|
live (≤ 4,194,304 px, 16,384 patches/pass). Health gate passed in ~100 s
|
||||||
|
(radiance binds the port after the full load).
|
||||||
|
5. **Vision probe (2026-10-15)**: 300 px solid-red PNG via `image_url` —
|
||||||
|
reasoning stream: "a solid fill of a deep, saturated red hue". Content
|
||||||
|
came back EMPTY at `max_tokens 64`: the thinking budget consumed all 64
|
||||||
|
(`reasoning_tokens: 64`, finish `length`) — fleet law, probe thinking
|
||||||
|
models with ≥ a few hundred tokens.
|
||||||
|
6. **e2e gate**: deploy exited DEPLOY INCOMPLETE — 30/33 Playwright green,
|
||||||
|
3 red (`12-crawl`, `13-sso` search SSO, `16-marreta`) on DANKCLAW
|
||||||
|
surfaces (vhosts intermittently returned `ERR_CONNECTION_RESET` from the
|
||||||
|
workstation); frank-archon's own health gate passed. The swap touched no
|
||||||
|
dankclaw runtime file (only homepage services.json text). Census of the
|
||||||
|
Caddy/auth stack pending; see `log.md`.
|
||||||
|
7. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
|
||||||
|
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
|
||||||
|
these are the **w4dn8** numbers, the record to beat, not this file's),
|
||||||
|
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
|
||||||
|
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
|
||||||
|
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||||||
|
(116,416 MiB), KLD-gated, same shape — **no longer on disk**. The 2026-10-17
|
||||||
|
disk purge (Frank's approval, trash-staged then emptied: 763 GiB free after)
|
||||||
|
removed it; the byte-exact copy lives on the Hub as
|
||||||
|
`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
|
||||||
|
(`qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`)
|
||||||
|
and re-fetches through `scripts/download-frank-archon-iq4r.sh`'s
|
||||||
|
hfget/verify pattern retargeted at that repo. The same purge deleted the
|
||||||
|
`radiance-state` kvcache tiers, `~/.radiance-cache-w4a8-*`,
|
||||||
|
`~/.cache/frank-primary`, and `/opt/vllm/moremodels` (GLM-4.7 GGUF) — all
|
||||||
|
derived or retired. The fp8-native era is rebuild-only (see the cutover
|
||||||
|
record). The w4dn8 runbook keeps every shape law, the KV dtype table, the
|
||||||
|
slab ladder, and the tuning levers:
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]].
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
|
||||||
|
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
|
||||||
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
|
||||||
|
(rebuild-only now; owns the descriptor-selection law)
|
||||||
|
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
|
||||||
|
- [[deployment]] — the service-scoped deploy this cutover rides
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# frank-archon — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2
|
# frank-archon — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2
|
||||||
|
|
||||||
**Status: LIVE — deployed 2026-10-08 (main `3ca64e1`, e2e 33/33).** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
**Status: RETIRED — superseded 2026-10-15 by the publisher iq4r payload.** · Lived 2026-10-08 → 10-15 (main `3ca64e1`, e2e 33/33). · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||||||
|
|
||||||
The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert)
|
The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert)
|
||||||
is replaced by a **GPTQ-calibrated 4-bit container** of
|
is replaced by a **GPTQ-calibrated 4-bit container** of
|
||||||
@@ -9,10 +9,10 @@ finetune) converted with the recipe
|
|||||||
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine
|
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine
|
||||||
**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as
|
**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as
|
||||||
the FP8-native era; 1.2.0 was used ONLY for the calibration serve).
|
the FP8-native era; 1.2.0 was used ONLY for the calibration serve).
|
||||||
Same port, alias, container name, systemd unit. Rollback: revert the
|
Same port, alias, container name, systemd unit, and every serving argument
|
||||||
`MODEL_FILE` line in `scripts/frank-archon.env` + redeploy —
|
carried into the successor: [[frank-archon-radiance-qwen38-iq4r]]. The
|
||||||
`/home/dank/models/rad/qwen4exp-fp8-native.rad` stays on disk (trash
|
`.rad` itself was purged from disk 2026-10-17; the byte-exact payload
|
||||||
policy; 173 GiB, purge when the window closes).
|
lives on the Hub — see Rollback.
|
||||||
|
|
||||||
## Why this swap
|
## Why this swap
|
||||||
|
|
||||||
@@ -594,15 +594,25 @@ probe; `reasoning_effort` fold; Prometheus target green; `tests/run.sh`
|
|||||||
|
|
||||||
| Layer | Path |
|
| Layer | Path |
|
||||||
|---|---|
|
|---|---|
|
||||||
| FP8-native radiance (immediate) | revert `MODEL_FILE` in `scripts/frank-archon.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk |
|
| Publisher iq4r (successor) | [[frank-archon-radiance-qwen38-iq4r]] — the LIVE payload since 2026-10-15 |
|
||||||
|
| FP8-native radiance | rebuild-only — the `.rad` (173.0 GiB) was `rm`'d 2026-10-15: `scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh` |
|
||||||
| FreeToken PR #587 | git tag `frank-archon-freetoken-pr587-20261005` |
|
| FreeToken PR #587 | git tag `frank-archon-freetoken-pr587-20261005` |
|
||||||
| tcclaviger vLLM shape G | git tag `frank-archon-vllm-shapeG-20261003` |
|
| tcclaviger vLLM shape G | git tag `frank-archon-vllm-shapeG-20261003` |
|
||||||
|
|
||||||
Campaign artifacts on disk: BF16 checkpoint (335 GiB), `kld-ref` (24
|
**This era's own payload** (`qwen4exp-swift15-w4dn8-i8-hc8m.rad`,
|
||||||
GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, logs
|
116,416 MiB) was purged in the 2026-10-17 disk cleanup (trash-staged, then
|
||||||
`/home/dank/logs/swift15_*.log`. The BF16 `.rad` container is deleted
|
emptied with Frank's approval). The byte-exact container is published as
|
||||||
before the final convert (disk gate) — the raw checkpoint is the
|
`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
|
||||||
re-convert source if the window ever reopens.
|
(`qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, Hub LFS sha256
|
||||||
|
`09dd93064365009c508b59012adadab6a36791a50fa319f76b31bf359c01e9ff`) —
|
||||||
|
re-download restores the exact payload; a full re-convert is no longer
|
||||||
|
possible without re-fetching a BF16 source (see next paragraph).
|
||||||
|
|
||||||
|
Campaign artifacts: the raw BF16 checkpoint was `rm`'d 2026-10-15 (Frank's
|
||||||
|
one-time authorization), so re-convert needs a fresh BF16 source; `kld-ref`
|
||||||
|
(24 GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, and the logs
|
||||||
|
`/home/dank/logs/swift15_*.log` remain. The BF16 `.rad` container was
|
||||||
|
deleted before the final convert (disk gate).
|
||||||
|
|
||||||
## Links
|
## Links
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,190 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (ACTIVE)"
|
||||||
|
description: "The ACTIVE frank-zealot runbook (since 2026-10-17) — the PrismML fork build of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape."
|
||||||
|
tags:
|
||||||
|
- llm
|
||||||
|
- frank-zealot
|
||||||
|
- dankclaw
|
||||||
|
- llama-cpp
|
||||||
|
- gguf
|
||||||
|
- rtx3090
|
||||||
|
timestamp: 2026-10-17
|
||||||
|
---
|
||||||
|
|
||||||
|
# frank-zealot — llama.cpp (PrismML) Bonsai-2-27B PQ2_0, 2 × 200,704, q8_0 KV
|
||||||
|
|
||||||
|
Live since 2026-10-17, replacing the
|
||||||
|
[[frank-zealot-llama-qwen35-9b-q6k-2x200704]] era (Qwen3.5-9B Q6_K, exact f16
|
||||||
|
KV, same 2 × 200,704 shape). The trade: the ternary 27B costs ~3.4× the weights
|
||||||
|
of the 9B and buys a 3× larger, stronger base at the SAME window contract —
|
||||||
|
paid for by quantizing KV (f16 KV is arithmetically impossible for this
|
||||||
|
payload on this card, measured below).
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB (24,123 MiB visible to the engine) |
|
||||||
|
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS`), `/v1` OpenAI surface |
|
||||||
|
| Runtime | **PrismML fork build** `/home/dank/GitPackages/llama-prism/llama-server` — vendor release, `--version` = `0.2.0-dev (build 10754, commit 2459f68b5)`, sha256 `12ae5baba033d8cd…`. NOT a preference: see *Why the fork* |
|
||||||
|
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
|
||||||
|
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
|
||||||
|
| Weights | `/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf` — 7,206,168,928 B, sha256 `3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1` |
|
||||||
|
| Projector | `Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf` — 629,246,976 B (full sha256 pinned in the fetch script), Qwen3-VL tower; **873.10 MiB worst-case VRAM, loaded AFTER the engine's fit check** |
|
||||||
|
| Fetch | [`scripts/download-frank-zealot-bonsai2.sh`](../../scripts/download-frank-zealot-bonsai2.sh) — `-f`, byte-count + sha256 verified, idempotent, `--verify-only`, no token on the command line |
|
||||||
|
| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine) |
|
||||||
|
|
||||||
|
## Why the fork — the format is the lock
|
||||||
|
|
||||||
|
Read from the file's own metadata, not the model card:
|
||||||
|
|
||||||
|
|Key|Value|
|
||||||
|
|---|---|
|
||||||
|
|`general.architecture`|`qwen35` (hybrid: Gated DeltaNet + softmax attention)|
|
||||||
|
|`qwen35.block_count`|64, `full_attention_interval` 4 ⇒ **16 full-attention + 48 DeltaNet layers**|
|
||||||
|
|Attention|24 heads / **4 KV heads** × key 256 / value 256; `rope.dimension_sections` [11,11,10,0], `freq_base` 1e7, partial rope `dimension_count` 64|
|
||||||
|
|Widths|embedding 5,120, feed-forward 17,408|
|
||||||
|
|`qwen35.ssm.*`|conv_kernel 4, state_size 128, group_count 16, **time_step_rank 48**, **inner_size 6,144**|
|
||||||
|
|Window / vocab|`context_length` 262,144; vocab 248,320 (gpt2 BPE, pre `qwen35`), eos 248046|
|
||||||
|
|`general.file_type`|**141 = PQ2_0 — 2.13 bpw (group 128)**|
|
||||||
|
|Tensor histogram|851 tensors: **`pq2_0` ×402** (custom GGML type **142**), f32 ×353, bf16 ×96|
|
||||||
|
|`prism.hadamard.*`|transform `normalized-sylvester-walsh-hadamard`, block_size 1024, axis `input-last-dimension`, sign_mode explicit, **401 named weights**, sign widths [5120, 6144, 17408], 28,672 sign values, inverse applied on `token_embd.weight`, `gdn_v_grouped` true|
|
||||||
|
|Sampler metadata|`general.sampling.temp` 1.0 / `top_p` 0.95 / `top_k` 20|
|
||||||
|
|Chat template|8,952 chars; reads `enable_thinking`, **`reasoning_effort` ∈ {`xhigh` (default), `medium`, `low` — anything else is `raise_exception`**, `preserve_thinking`, `tools`/`tool_call`, image + video content parts|
|
||||||
|
|
||||||
|
Stock llama.cpp cannot load this file: type 142 is not in its enum (`gguf-py`'s
|
||||||
|
stock `GGMLQuantizationType` raises `ValueError: 142 is not a valid…` on the
|
||||||
|
same read), and even a loader that parsed it would ignore the
|
||||||
|
`prism.hadamard.*` dequant transform the weights depend on. Same class of lock
|
||||||
|
as the ROCmFPX custom-type era. The vendor build 10754 already carries the
|
||||||
|
September upstream fixes (AVX-512 load segfault, and a `GGML_CUDA_FA_ALL_QUANTS`
|
||||||
|
-capable CUDA backend — proven by q8_0 V serving under flash attention at the
|
||||||
|
correct 34,816 B/token, below).
|
||||||
|
|
||||||
|
## VRAM census (2026-10-17, four arms, production stopped)
|
||||||
|
|
||||||
|
The engine's own `common_memory_breakdown_print` ledger + settled `nvidia-smi`,
|
||||||
|
all arms `--n-gpu-layers 99 --flash-attn on --batch-size 4096 --ubatch-size 2048`,
|
||||||
|
mmproj attached:
|
||||||
|
|
||||||
|
|Shape|KV|projected `model + context + compute`|settled used / free|verdict|
|
||||||
|
|---|---|---|---|---|
|
||||||
|
|**2 × 200,704 (pool 401,408)**|**q8_0 / q8_0**|21,439 = 6,539 + 13,627 + 1,272|**22,664 / 1,461**|**DEPLOYED**|
|
||||||
|
|2 × 200,704|f16 / f16|33,198 = 6,539 + 25,387 + 1,272|—|`cudaMalloc failed: out of memory` on the **25,088.00 MiB** KV request|
|
||||||
|
|1 × 262,144|q8_0 / q8_0|16,905 = 6,539 + 8,853 + 1,512|18,130 / 5,995|fits, but one slot serializes compaction|
|
||||||
|
|2 × 262,144|q8_0 / q8_0|≈25,758 (arithmetic, same terms)|—|does not fit (f16 analogue measured 41,118)|
|
||||||
|
|2 × 200,704|q4_0 / q4_0|15,167 = 6,539 + 7,355 + 1,272|16,392 / 7,733|the roomy fallback rung|
|
||||||
|
|
||||||
|
**KV rate law (confirmed to the MiB):** 16 full layers × 4 KV heads × 256 dim =
|
||||||
|
16,384 elements/token/cache; q8_0 bills 17,408 B per cache ⇒ **34,816 B/token**,
|
||||||
|
so the 401,408 pool = **13,328.00 MiB** — the ledger printed exactly that
|
||||||
|
(`CUDA0 KV buffer size = 13328.00 MiB`). f16 doubles it to 25,088 MiB, which is
|
||||||
|
the wall. The 48 DeltaNet layers bill a fixed recurrent state per **slot**:
|
||||||
|
context 13,627 − KV 13,328 = **299 MiB for two slots (~150 MiB each)** — pool
|
||||||
|
arithmetic alone under-prices the shape by that much.
|
||||||
|
|
||||||
|
**The projector is outside the fit check.** `[mtmd] estimated worst-case memory
|
||||||
|
usage of mmproj is 873.10 MiB` is printed before fitting but not billed in it:
|
||||||
|
settled used (22,664) − projection (21,439) ≈ 1.2 GiB is that projector plus the
|
||||||
|
CUDA context. This is the difference between "2.6 GiB free on paper" and the
|
||||||
|
real **1.4 GiB** — the same late-allocation failure class that killed the
|
||||||
|
DSpark drafter boot in the llama.cpp eras. Any resize must re-read the settled
|
||||||
|
`nvidia-smi`, never the projection.
|
||||||
|
|
||||||
|
## Measured rates (greedy, `Server-Timing`-free `/completion` `timings`)
|
||||||
|
|
||||||
|
|Probe|Result|
|
||||||
|
|---|---|
|
||||||
|
|Short, warm, single stream (128 tok)|**76.6 t/s** decode (the 9B Q6_K predecessor ran 94.4 — the ternary 27B is ~19 % slower, weight bandwidth + Hadamard dequant)|
|
||||||
|
|First request after boot|2.6 t/s prefill / 37.7 t/s decode — **warmup artifact; never report the first request** (measured twice on this payload)|
|
||||||
|
|Prefill at ~35k depth|**1,258.4 t/s** (27.8 s)|
|
||||||
|
|Prefill at 166,027-token depth|**691.7 t/s** (240.0 s), zero errors, slot intact|
|
||||||
|
|Two concurrent streams, 256 tok each|**54.7 + 54.7 = 109.4 t/s aggregate** — `--parallel 2` really batches|
|
||||||
|
|Long-context retrieval (25,960-token prompt, needle at position 180 of 300 passages)|exact: *"Frank keeps the fleet in git, only through it."*|
|
||||||
|
|
||||||
|
Production boot (the 19:44 cutover deploy, same binary/flags/file): **75.8 t/s**
|
||||||
|
single-stream decode at a short prompt — within 1 % of the trial arm, so the
|
||||||
|
ladder above carries to the shipped shape.
|
||||||
|
|
||||||
|
## Behavior contract (probes, not assumptions)
|
||||||
|
|
||||||
|
- **Reasoning** lands in `reasoning_content` (`--reasoning-format deepseek`) ✓.
|
||||||
|
- **Tool calls**: a `get_weather` request returns `finish_reason: tool_calls`
|
||||||
|
with `arguments: {"city":"Seattle"}` — valid JSON, no loop, no HTTP 500. The
|
||||||
|
fork's `KNOWN_ISSUES.md` warning about malformed/looping tool calls did not
|
||||||
|
reproduce on build 10754 at this shape.
|
||||||
|
- **Vision**: a 512 px solid-red PNG answers **"Red"** — but ONLY with a
|
||||||
|
thinking budget ≥ ~512 tokens. At `max_tokens: 64` the request returns
|
||||||
|
`finish_reason: "length"` with EMPTY `content` and half a thought in
|
||||||
|
`reasoning_content`: thinking tokens count against the budget, so a
|
||||||
|
short-budget vision probe reads as "the model is blind". Images do reach the
|
||||||
|
model (a 512 px image bills ~280 prompt tokens: 322 total vs 42 for the text
|
||||||
|
alone). Keep the ≥256 px probe rule from the earlier eras.
|
||||||
|
- **Log noise law**: `/completion` (text context) at `VERBOSITY=4` prints
|
||||||
|
`failed to parse reasoning content (response_format json_schema is currently
|
||||||
|
only supported with parse_reasoning=true), content will be stored in
|
||||||
|
'content' field` (`probe_request.cpp:789`) whenever the model closes a raw
|
||||||
|
prompt with ``. It is **cosmetic** — the full chain of thought
|
||||||
|
is in `content`. The OpenAI `/v1/chat/completions` path parses it into
|
||||||
|
`reasoning_content` correctly, which is the path the harness and every
|
||||||
|
consumer uses. Do not diagnose this as a fault, and do not drop verbosity
|
||||||
|
below 4 — the KV/buffer ledger in the journal is what VRAM forensics read.
|
||||||
|
- **Long raw-prompt completions can come back empty**: at ~26k+ tokens a bare
|
||||||
|
`/completion` request generated 0–1 tokens with no error (the model emits an
|
||||||
|
end-of-turn marker immediately, and a text context has no role frame to keep
|
||||||
|
it talking). Chat traffic is unaffected; nothing in the harness posts raw
|
||||||
|
`/completion` at depth.
|
||||||
|
|
||||||
|
## Ops notes
|
||||||
|
|
||||||
|
- **Trial-window law (cost one boot window on 2026-10-17):** a scratch env file
|
||||||
|
must use the serve script's OWN variable names — `CACHE_TYPE_K`/`CACHE_TYPE_V`,
|
||||||
|
`BIND_ADDR`, `NGL`. An invented `KV_CACHE_TYPE` is silently swallowed by the
|
||||||
|
script's `: "${CACHE_TYPE_K:=f16}"` default and the arm boots **f16**, which
|
||||||
|
for this payload is an OOM that looks like a model problem.
|
||||||
|
- **Base alignment:** Bonsai-2 is the BASE (censored) Qwen3.8-27B lineage, so a
|
||||||
|
refusal is possible on a compaction or subagent request. The CRACK re-export
|
||||||
|
(`dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF`, already on disk at
|
||||||
|
`/home/dank/models/Bonsai2-CRACK/Bonsai-2-27B-PQ2_0-CRACK.gguf`, sha256
|
||||||
|
`5b24ea3e…`) is a **one-line `MODEL_FILE` swap**: the header census proves
|
||||||
|
byte-identical geometry — 851 tensors, the same `{142:402, f32:353, bf16:96}`
|
||||||
|
histogram, 64 blocks, vocab 248,320, the same 8,952-char template. The VRAM
|
||||||
|
ledger does not move. If a refusal ever surfaces, that is the fix; there is
|
||||||
|
also an MTP-bearing CRACK re-export upstream
|
||||||
|
(`ManniX-ITA/…-CRACK-MTP-GGUF`, 7,657,489,920 B, +451 MB head) if decode ever
|
||||||
|
needs the drafter — untested here.
|
||||||
|
- **Compaction hub unchanged:** all three endpoints still route
|
||||||
|
`compactionModel` here; 200,704 per slot is the fleet's only window that fits
|
||||||
|
a worst-case compaction input, and `--parallel 2` keeps compaction off the
|
||||||
|
critical path of a live generation.
|
||||||
|
- **Harness side** (`~/.omp/agent/models.yml`): `contextWindow: 200704` and
|
||||||
|
`maxTokens: 16384` unchanged; `extraBody` now quotes the payload's own
|
||||||
|
1.0 / 0.95 / 20 with `preserve_thinking: true`. The `reasoningEffortMap` fold
|
||||||
|
became **load-bearing** (the template raises on any level outside
|
||||||
|
`xhigh|medium|low`) — an unmapped level is a hard 500 on the role's first
|
||||||
|
request, naming Jinja rather than the model.
|
||||||
|
- **Rollback:** revert the swap commit. The Qwen3.5-9B Q6_K payload, its mmproj
|
||||||
|
and the master CUDA build all remain on disk
|
||||||
|
(`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`,
|
||||||
|
`~/GitPackages/llama-master/build-cuda/`), and the retired era keeps its own
|
||||||
|
page: [[frank-zealot-llama-qwen35-9b-q6k-2x200704]].
|
||||||
|
- **Deployed 2026-10-17 19:44** (`deploy.sh dankclaw --service frank-zealot
|
||||||
|
--service homepage`): boot ledger 22,664 used / 1,461 free, engine fit
|
||||||
|
projected 21,439 and left 2,119 ≥ 1,872 MiB, KV 13,328.00 MiB at
|
||||||
|
`n_ctx_seq = 200,704`, `n_slots = 2`, projector loaded; live probes green
|
||||||
|
(`/v1/models` serves `frank-zealot`, exact `pong` + `reasoning_content`,
|
||||||
|
`tool_calls` `{"city":"Seattle"}`, 512 px PNG → **Red**). e2e gate 33/33.
|
||||||
|
- **Picker row is live state:** the OpenWebUI `model` row's
|
||||||
|
`meta.description` named the 9B, so it was re-keyed in the database too
|
||||||
|
(backup `webui.db.bak-20261017a`). The row was written WITHOUT restarting
|
||||||
|
`open-webui` — the cached list refreshes on its next restart, and forcing one
|
||||||
|
would have logged every family session out for a cosmetic string. See
|
||||||
|
[[OpenWebUI]] for the picker trio.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — the replaced era (rollback target)
|
||||||
|
- [[frank-dragoon-flash-server-v2-swift15]] — the abliterated sibling on the iGPU
|
||||||
|
- [[frank-archon-radiance-qwen38-iq4r]] — the censored-base flagship this payload shares base alignment with
|
||||||
|
- [[deployment]] — deploy flow and incidents
|
||||||
@@ -3,25 +3,30 @@
|
|||||||
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
||||||
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
||||||
driving `podman run` directly, root systemd unit, :7770):
|
driving `podman run` directly, root systemd unit, :7770):
|
||||||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
|
the **publisher's** 4-bit-codebook container of the official
|
||||||
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
|
`Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
|
||||||
|
MTP head and vision tower in-container), on BOTH R9700s
|
||||||
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
||||||
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
||||||
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
||||||
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
||||||
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
|
froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on
|
||||||
and frank-archon-vllm-shapeG-20261003),
|
disk); tags frank-archon-freetoken-pr587-20261005 and
|
||||||
|
frank-archon-vllm-shapeG-20261003),
|
||||||
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||||
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
||||||
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||||
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
|
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
|
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
|
||||||
ctx, vision ON)** and
|
ctx, vision ON)** and
|
||||||
**llama.cpp CUDA (frank-zealot, port 7771: HauhauCS Gemma4-26B-A4B QAT
|
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||||
Uncensored Q4_K_M GGUF — 26B MoE hybrid (25 sliding-window + 5 full-attention
|
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||||
layers), ONE 200,704-token conversation, MTP disabled,
|
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||||
q8_0 K + q8_0 V (FP8 class), vision via the publisher's own BF16
|
q8_0 K + q8_0 V (f16 KV does not fit the RTX 3090 at this window), vision via
|
||||||
mmproj, on the RTX 3090)**.
|
the publisher's own Q8_0 Qwen3-VL mmproj, on the RTX 3090 — and the custom
|
||||||
|
tensor type 142 + `prism.hadamard.*` metadata mean no other loader can open the
|
||||||
|
file)**.
|
||||||
|
|
||||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||||
@@ -37,10 +42,10 @@ graph LR
|
|||||||
gfx1201, 32 GiB each]
|
gfx1201, 32 GiB each]
|
||||||
B2[radiance StillDeadcode, digest-pinned image
|
B2[radiance StillDeadcode, digest-pinned image
|
||||||
podman run, frank-archon, port 7770]
|
podman run, frank-archon, port 7770]
|
||||||
C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad
|
C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad
|
||||||
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
|
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
|
||||||
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
||||||
4 x 200000, froggeric v22.5]
|
3 x 200000, froggeric v22.5]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||||
@@ -103,13 +108,25 @@ frank-zealot, port 7771]
|
|||||||
(`slow`/`advisor` → frank-archon) carried forward. Superseded by the
|
(`slow`/`advisor` → frank-archon) carried forward. Superseded by the
|
||||||
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
|
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
|
||||||
engine moved with the model
|
engine moved with the model
|
||||||
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — **ACTIVE since 2026-10-07**
|
- [[frank-zealot-llama-prism-bonsai2-pq20]] — **ACTIVE since 2026-10-17**
|
||||||
(root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
|
(root systemd unit, port 7771): the **PrismML fork** build of llama.cpp
|
||||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090
|
(vendor 10754, commit `2459f68b5` — the only loader for the payload's custom
|
||||||
— TWO 200,704-token conversations (`--ctx-size 401408 --parallel 2`), exact
|
tensor type 142 + `prism.hadamard.*` transform) serving
|
||||||
f16 K + f16 V, vision via the publisher's BF16 mmproj. MTP is OFF because
|
prism-ml/Ternary-Bonsai-2-27B-gguf **PQ2_0** (2.13 bpw) on the RTX 3090 —
|
||||||
the file has no MTP tensors at all. Priced from the MiMo Q6_K boots (same
|
TWO 200,704-token conversations (`--ctx-size 401408 --parallel 2`), **q8_0 K
|
||||||
qwen35 geometry, same card, same shape). Supersedes the Gemma4 era (below).
|
+ q8_0 V** (34 KiB/token over the 16 full layers = 13,328 MiB; the f16 pair
|
||||||
|
needs 25,088 MiB and dies), settled 22,664 used / 1,461 free, vision via the
|
||||||
|
publisher's Q8_0 mmproj (+873 MiB, charged AFTER the engine's fit check), MTP
|
||||||
|
OFF (no MTP tensors). 76.6 t/s single-stream decode, 691.7 t/s prefill at
|
||||||
|
166k, 109.4 t/s aggregate at two streams. Supersedes the Qwen3.5-9B era below.
|
||||||
|
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — **RETIRED 2026-10-07 →
|
||||||
|
10-17** (payload + master CUDA build on disk as rollback): llama.cpp master
|
||||||
|
(CUDA sm_86) serving HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive
|
||||||
|
**Q6_K** on the RTX 3090 — TWO 200,704-token conversations
|
||||||
|
(`--ctx-size 401408 --parallel 2`), exact f16 K + f16 V, vision via the
|
||||||
|
publisher's BF16 mmproj. MTP is OFF because the file has no MTP tensors at
|
||||||
|
all. Priced from the MiMo Q6_K boots (same qwen35 geometry, same card, same
|
||||||
|
shape). Supersedes the Gemma4 era (below).
|
||||||
- [[frank-zealot-llama-gemma4-q4km-200704]] — **RETIRED 2026-10-07** (same
|
- [[frank-zealot-llama-gemma4-q4km-200704]] — **RETIRED 2026-10-07** (same
|
||||||
day; superseded by qwen35-9b-q6k; payload on disk as rollback): llama.cpp
|
day; superseded by qwen35-9b-q6k; payload on disk as rollback): llama.cpp
|
||||||
master (CUDA sm_86) serving HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-
|
master (CUDA sm_86) serving HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-
|
||||||
@@ -153,7 +170,22 @@ frank-zealot, port 7771]
|
|||||||
target; payload on disk; owns the measured boot-pair table)**: the same
|
target; payload on disk; owns the measured boot-pair table)**: the same
|
||||||
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
||||||
104.8 tok/s, 3,571 MiB true free
|
104.8 tok/s, 3,571 MiB true free
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**:
|
||||||
|
the engine author's published container of the OFFICIAL (base-alignment)
|
||||||
|
Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
|
||||||
|
codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
|
||||||
|
sweeps, 10M-token calibration), ten bf16-protected experts per layer,
|
||||||
|
int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
|
||||||
|
0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
|
||||||
|
that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
|
||||||
|
3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
|
||||||
|
Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
|
||||||
|
fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
|
||||||
|
dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
|
||||||
|
rebuild via the fetch + build scripts named in the fp8-native runbook)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
|
||||||
|
(superseded by the publisher iq4r swap; payload on disk as the rollback)**:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
||||||
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
||||||
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
||||||
|
|||||||
@@ -1,199 +0,0 @@
|
|||||||
---
|
|
||||||
type: reference
|
|
||||||
title: LLM Usage Attribution
|
|
||||||
description: Research catalog of every viable way to answer "who is using the fleet LLMs" — engine-native identity, gateways, auth proxies, tracing platforms, network/host attribution — with the measured fleet baseline (2026-10-13) and a phased adoption plan.
|
|
||||||
tags: [llm, observability, auth, gateway, litellm, tracking, identity]
|
|
||||||
timestamp: 2026-10-14T00:00:00-07:00
|
|
||||||
---
|
|
||||||
|
|
||||||
# LLM Usage Attribution
|
|
||||||
|
|
||||||
Deep-research catalog (2026-10-13/14): solutions for tracking WHO uses the
|
|
||||||
fleet endpoints (`frank-archon`, `frank-dragoon`, `frank-zealot`). Full web
|
|
||||||
citations; live-fleet measurements marked **(measured)**.
|
|
||||||
|
|
||||||
## Baseline: what the fleet answers today
|
|
||||||
|
|
||||||
- All three endpoints are `auth: none`, plain HTTP, LAN-only
|
|
||||||
(`models.yml` providers; no vhost on 7770/7771).
|
|
||||||
- **(measured)** Live `/metrics` label census: `frank-archon` exports
|
|
||||||
`vllm:*`+`radiance:*` with labels only `{model_name, engine, le,
|
|
||||||
finished_reason, position, class}` — zero identity labels.
|
|
||||||
`frank-dragoon` exports `halogen:*`+`llamacpp:*`.
|
|
||||||
`frank-zealot` answers **501** — `serve-frank-zealot.sh` never passes
|
|
||||||
`--metrics`, so the Prometheus job is structurally never-green
|
|
||||||
(`LlmEndpointDown` fires). Fix: add `--metrics` to the ARGS array.
|
|
||||||
- **(measured)** frank-zealot verbosity-4 journal lines carry slot/task/timing
|
|
||||||
only — no client IP, no user. Alloy ships journal + podman ctr.log to Loki,
|
|
||||||
so what lands there has no identity either.
|
|
||||||
- Caddy has no `log` directive anywhere → no per-request access logs on the
|
|
||||||
OpenWebUI vhost.
|
|
||||||
- Net: fleet-level token telemetry exists per ENDPOINT; per-CONSUMER
|
|
||||||
identity exists nowhere.
|
|
||||||
|
|
||||||
## The identity layers (pick the layer, get its resolution)
|
|
||||||
|
|
||||||
| # | Layer | Identity | Answers |
|
|
||||||
|---|---|---|---|
|
|
||||||
| L0 | DHCP/ARP | MAC, hostname, lease | which device was on the LAN |
|
|
||||||
| L1 | DNS (Pi-hole v6) | device → queries | which device resolved/used a name |
|
|
||||||
| L2 | L3/L4 (nftables/conntrack, JA3/JA4) | IP, TLS client lib | bytes per client, client software |
|
|
||||||
| L3 | Transport (API keys, mTLS, SPIFFE, Tailscale) | key/cert/node | which credential connected |
|
|
||||||
| L4 | App gateway (virtual keys → user/team/end_user) | named consumer | tokens/spend/cost per who |
|
|
||||||
| L5 | Trace/content (Langfuse, Helicone, halogen records) | user_id, session, prompt text | what they asked, per session |
|
|
||||||
| L6 | Client host (auditd, eBPF, netdata) | pid/uid/comm per connect | which process on which host |
|
|
||||||
|
|
||||||
Keys/headers/`user` fields are CLIENT-DECLARED — they group traffic, they
|
|
||||||
prove nothing about a human (halogen's own docs warn this verbatim:
|
|
||||||
"they group requests, and they prove nothing about who sent them").
|
|
||||||
|
|
||||||
## Catalog
|
|
||||||
|
|
||||||
### A. Engine-native (per-consumer identity inside the inference server)
|
|
||||||
|
|
||||||
| Engine | AuthN | Identity in logs | Per-identity metrics | OTel | Notes |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| llama.cpp | `--api-key` (multi, comma) / `--api-key-file` **(measured in the deployed build's `--help`)** | request `user` field drives slot assignment/eviction; `--log-requests` dumps bodies | `--metrics` exports `llamacpp:*`, NO key/user label | none upstream | [server README](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) |
|
|
||||||
| vLLM | `--api-key` (multi) | request log; no user persistence | `vllm:*`; NO `api_key` label (cardinality, issue #53926) | native (`--otlp-traces-endpoint`) | [metrics design](https://github.com/vllm-project/vllm/blob/main/docs/design/metrics.md) |
|
|
||||||
| radiance | no public auth docs **[unverified]** | — | **(measured)** no identity labels on live /metrics | none found | closed deployment surface; treat authN as absent until probed |
|
|
||||||
| halogen-flash-server | `HALOGEN_API_KEY` / `HALOGEN_API_KEY_FILE` (multi, comma) | **`HALOGEN_RECORD_DIR` request records**: `turns/*.jsonl` per request — `client_family` (User-Agent), tokens, timings, finish, tool calls, plus HASHES of `user` / `prompt_cache_key` / `safety_identifier` / `metadata.user_id` (`.salt`-keyed); `units/` content-addressed text; caps `HALOGEN_RECORD_GIB`=2, `HALOGEN_RECORD_DAYS`=90; `HALOGEN_RECORD_TEXT=none` for metadata-only; `stats`/`prune` tools in-container | `halogen:*`+`llamacpp:*`, no per-key label | none | [configuration.md](https://github.com/peonist-ai/halogen-flash-server/blob/main/docs/configuration.md) — verbatim-verified 2026-10-13. Best native ledger of the fleet. |
|
|
||||||
| SGLang | `--api-key` (single) | `--log-requests` | `sglang:*`, no key label | native (`--enable-trace`) | for any future swap |
|
|
||||||
| Ollama | single key | request log | `ollama:` no key label | none | not in fleet |
|
|
||||||
|
|
||||||
Wire fields usable from clients today (all client-declared): OpenAI
|
|
||||||
`user`, `safety_identifier`, `prompt_cache_key`, Messages
|
|
||||||
`metadata.user_id`; LiteLLM `end_user_id`; gateway header conventions
|
|
||||||
(`X-Forwarded-User`, `Helicone-User-Id`, `cf-aig-metadata`).
|
|
||||||
|
|
||||||
### B. LLM gateways (front door with virtual keys — the only layer with
|
|
||||||
per-consumer TOKEN counts)
|
|
||||||
|
|
||||||
| Solution | Identity | Usage lands | License | Footprint |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| **LiteLLM Proxy** | virtual keys → `user_id`/`team_id`/`end_user`; budgets, rate limits | Postgres spend ledger; `/metrics` labels `api_key_alias,user,team,end_user,model,client_ip,user_agent` (`litellm_*_tokens_metric`, spend, budget gauges); OTel/Langfuse callbacks | MIT core (SSO/RBAC/audit-log enterprise) | ~150–300 MiB + Postgres |
|
|
||||||
| Kong AI Gateway | Consumer + key-auth/JWT/OIDC/mTLS; AI consumer groups | Postgres; `kong_*` per-consumer; file-log/zipkin/otel | OSS core; token-aware AI plugins enterprise | 0.5–1 GiB |
|
|
||||||
| Apache APISIX | Consumer + key-auth/JWT | `apisix_llm_prompt_tokens`/`completion_tokens` with `consumer` label; http-logger/kafka | Apache-2.0 | 200–400 MiB |
|
|
||||||
| Bifrost (maximhq) | virtual keys `sk-bf-*`/`x-bf-vk`, budgets | Prometheus/OTel, config store (SQLite/Postgres) | Apache-2.0, Go | ~150–300 MiB |
|
|
||||||
| Portkey OSS gateway | virtual keys | logs/OTel | MIT | 200–400 MiB |
|
|
||||||
| Higress AI GW | keys (+OIDC ent) | `higress_*`, ai-logging | Apache-2.0 CE | Istio-based, 0.5 GiB+ |
|
|
||||||
| Envoy AI GW / Gateway API Inference Extension / LLM Gatekeeper | delegated auth | Envoy metrics/access logs | Apache-2.0 | K8s-native |
|
|
||||||
| Gloo / KrakenD / Gravitee AI gateways | keys, OIDC | metrics/logs | OSS core, AI features enterprise | heavy |
|
|
||||||
| Cloudflare AI Gateway | Access JWT `cf.user_id`, `cf-aig-metadata` | Cloudflare-managed analytics/logs | SaaS (exotic for LAN; Tunnel possible) | edge |
|
|
||||||
|
|
||||||
([LiteLLM virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys),
|
|
||||||
[prometheus](https://docs.litellm.ai/docs/proxy/prometheus),
|
|
||||||
[APISIX ai-proxy](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/),
|
|
||||||
[Bifrost](https://github.com/maximhq/bifrost),
|
|
||||||
[Kong AI](https://developer.konghq.com/index/ai-gateway/))
|
|
||||||
|
|
||||||
### C. Auth proxies / identity infrastructure (identity without token counts)
|
|
||||||
|
|
||||||
- **Ory Oathkeeper** (MIT, ~50–100 MiB): per-consumer api-key/JWT/
|
|
||||||
client-credentials → header mutation (`X-User: {{ .Subject }}`) +
|
|
||||||
decision logs.
|
|
||||||
- **oauth2-proxy** (Apache-2.0): OIDC (Authentik) incl. client_credentials,
|
|
||||||
`--pass-user-headers` → `X-Forwarded-User`; ~100 MiB.
|
|
||||||
- **Authentik itself**: proxy-provider custom headers (already injects
|
|
||||||
`X-Authentik-Username`/`-Groups`, spoof-stripped — the /whoami seam proves
|
|
||||||
the pattern on this host); event log + webhook forwarding.
|
|
||||||
- **Caddy `tls client_auth`** (mTLS per client cert) + access-log fields —
|
|
||||||
Caddy already fronts both hosts.
|
|
||||||
- **Smallstep step-ca**: short-lived per-device certs, ACME-style.
|
|
||||||
- **SPIFFE/SPIRE**: per-workload SVIDs (k8s-flavored; heavy for quadlets).
|
|
||||||
- **Tailscale / NetBird**: per-node identity, ACL + network-flow logs,
|
|
||||||
headscale for self-hosted control. Best for off-LAN laptops.
|
|
||||||
- **ApiAuth** (nginx auth_request): the minimal per-consumer apikey sidecar.
|
|
||||||
|
|
||||||
### D. LLM observability / tracing (prompt-level per-user audit)
|
|
||||||
|
|
||||||
| Platform | Identity | Content | Self-host | Shape |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| **Langfuse** | `user_id`,`session_id`, tags | full input/output, masking hook | ✅ ClickHouse, Apache-2.0 | fed via LiteLLM callback / OTLP / SDK |
|
|
||||||
| **Helicone** | api-key→user, `Helicone-User-Id` | full, tiered redaction | ✅ compose (ClickHouse+PG+Redis+Jawn), Apache-2.0 | PROXY — can be the front door itself |
|
|
||||||
| Arize Phoenix | `user.id`,`session.id` | opt-in `gen_ai.*` | ✅, Apache-2.0 | OTLP collector |
|
|
||||||
| OpenLIT | `user.id`,`session.id` | 3-tier capture | ✅, Apache-2.0 | OTLP+ClickHouse |
|
|
||||||
| SigNoz LLM | OTel attrs | opt-in | ✅, Apache-2.0 | OTLP |
|
|
||||||
| OpenLLMetry/Traceloop | assoc props | opt-in | ✅ Helm | OTel |
|
|
||||||
| Lunary / W&B Weave / LangSmith | user/session | full | enterprise-gated | — |
|
|
||||||
|
|
||||||
OTel GenAI semconv (Development-stability): `gen_ai.operation.name`,
|
|
||||||
`gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`/
|
|
||||||
`output_tokens`, `gen_ai.conversation.id`, content attrs opt-in; `user.id`
|
|
||||||
from the registry. vLLM and SGLang emit OTLP natively; llama.cpp only via
|
|
||||||
external SDK — **the fleet's engines are OTel-silent at request level**,
|
|
||||||
which is why the gateway callback (LiteLLM→Langfuse) is the practical feed.
|
|
||||||
|
|
||||||
### E. Network/host passive attribution (no client cooperation)
|
|
||||||
|
|
||||||
- **Pi-hole v6** (already on Dankclaw): query log API per device,
|
|
||||||
`pihole6_exporter` + query-logger→Loki (GPLv3 exporter).
|
|
||||||
- **Caddy access logs**: add a `log` directive (JSON) → Alloy→Loki; with the
|
|
||||||
auth chain it carries `X-Authentik-Username` for browser users.
|
|
||||||
- **OpenWebUI admin**: per-user daily token stats, events/logs, analytics API
|
|
||||||
(`/api/v1/analytics/*`) — per-USER identity exists here already (MIT).
|
|
||||||
- **auditd** connect-syscall rules (uid/pid/comm per connect to 7770/7771),
|
|
||||||
ship via audisp→Loki.
|
|
||||||
- **eBPF**: Tetragon (process+socket events, policies), Coroot, DeepFlow,
|
|
||||||
bpftrace `tracepoint:syscalls:sys_enter_connect` one-liners. K8s-shaped
|
|
||||||
for single hosts, but agents.plugin-style tools work.
|
|
||||||
- **nftables named counters** per consumer IP (bytes; reload resets).
|
|
||||||
- **netdata apps.plugin**: per-process network bytes, auto-discovered.
|
|
||||||
- **JA3/JA4 TLS fingerprinting**: client library only, TLS-only; the LAN is
|
|
||||||
plaintext HTTP today → use the HTTP **User-Agent census** instead (free
|
|
||||||
from any access log).
|
|
||||||
- **DHCP/ARP census** (leases + `ip neigh`): device roster.
|
|
||||||
|
|
||||||
## Phased plan for this fleet
|
|
||||||
|
|
||||||
**Phase 0 — repo-only, no new services**
|
|
||||||
1. frank-zealot: add `--metrics` (fixes the measured 501 + the
|
|
||||||
structurally-red Prometheus job + the firing `LlmEndpointDown`).
|
|
||||||
2. frank-dragoon: set `HALOGEN_API_KEY_FILE` (multi-key per consumer) +
|
|
||||||
mount `HALOGEN_RECORD_DIR` (start `HALOGEN_RECORD_TEXT=none`; flip to
|
|
||||||
`all` if prompt audit is wanted) + Alloy ship of `turns/`.
|
|
||||||
3. frank-zealot: `--api-key-file` (multi-key, verified in the deployed
|
|
||||||
build); radiance: probe for `--api-key` support (contract facts are
|
|
||||||
silent — add to `fleet-probes`).
|
|
||||||
4. models.yml: per-consumer bearer keys (`auth: none` → key) — the provider
|
|
||||||
schema supports apiKey (pi-era conversion note).
|
|
||||||
5. Caddy: `log` on `ai.franksdank.cloud` (OpenWebUI browser census, JSON →
|
|
||||||
Loki).
|
|
||||||
Limits: keys gate access, and only the engines' request records (halogen)
|
|
||||||
see them per-request; llama.cpp/radiance/vLLM metrics stay identity-blind.
|
|
||||||
|
|
||||||
**Phase 1 — LiteLLM Proxy quadlet on Dankclaw = the front door.**
|
|
||||||
One virtual key per consumer (omp-<workstation> ×N, openwebui, hermes,
|
|
||||||
agents, bench). Per-key/user/team Prometheus labels into the existing
|
|
||||||
Grafana+Alertmanager; Postgres spend ledger; budgets + rate limits per
|
|
||||||
key. This is the ONLY layer that yields tokens-per-who without engine
|
|
||||||
cooperation. (Bifrost is the lighter Apache-2.0 Go alternative; Helicone
|
|
||||||
the proxy-native-observability alternative.)
|
|
||||||
|
|
||||||
**Phase 2 — prompt-level audit.** Langfuse self-hosted fed by the LiteLLM
|
|
||||||
callback: per-user traces, sessions, cost; masking + retention knobs.
|
|
||||||
|
|
||||||
**Phase 3 — ambitious identity infra.** step-ca + Caddy mTLS per device;
|
|
||||||
Tailscale (+flow logs) for off-LAN; auditd/eBPF process attribution on the
|
|
||||||
agent hosts; Pi-hole query logger; nft counters per consumer IP. Browser
|
|
||||||
users stay owned by OpenWebUI analytics + its per-user API keys; a pipeline
|
|
||||||
can forward `X-Forwarded-User` so the gateway sees them too.
|
|
||||||
|
|
||||||
## Cautions
|
|
||||||
|
|
||||||
- Client-declared identity (keys, `user`, headers) groups traffic; it does
|
|
||||||
not authenticate humans. Authenticated identity = Authentik (browser) /
|
|
||||||
mTLS (device) / Tailscale (node).
|
|
||||||
- Plain-HTTP LAN = bearer keys sniffable by any tenant; treat keys as
|
|
||||||
identity labels on the LAN, not secrets, until TLS/mTLS at the door.
|
|
||||||
- Prompt-content logging stores a plaintext copy of everything — set
|
|
||||||
retention caps (halogen GIB/days; Langfuse masking; Loki retention) and
|
|
||||||
gitignore the record dirs.
|
|
||||||
- Household-scale cardinality is safe for per-key Prometheus labels;
|
|
||||||
the engines' refusal of key labels is a multi-tenant concern, not ours.
|
|
||||||
|
|
||||||
## Related
|
|
||||||
- [[Fleet Audit and Posture]] — monitoring gaps census
|
|
||||||
- [Observability](../Infrastructure/Observability.md) — the Loki/Prometheus
|
|
||||||
pipes every sink above uses
|
|
||||||
- [LLM index](../LLM/index.md) — endpoint runbooks (env trios the Phase-0
|
|
||||||
edits touch)
|
|
||||||
@@ -23,9 +23,6 @@ Procedures for operating the fleet. The LLM-specific pages live under
|
|||||||
posture, whole-fleet audit, bulk redaction, stash audits
|
posture, whole-fleet audit, bulk redaction, stash audits
|
||||||
- [[SSH and Remote Access]] — passwordless bootstrap without sshpass, fish
|
- [[SSH and Remote Access]] — passwordless bootstrap without sshpass, fish
|
||||||
login shells, GUI-hang diagnosis over ssh
|
login shells, GUI-hang diagnosis over ssh
|
||||||
- [[LLM Usage Attribution]] — who-is-using-the-fleet-LLMs research catalog:
|
|
||||||
engine-native identity, gateways, tracing, network/host attribution, and
|
|
||||||
the phased plan
|
|
||||||
|
|
||||||
## Related
|
## Related
|
||||||
- [Observability Provisioning](../Infrastructure/Observability%20Provisioning.md)
|
- [Observability Provisioning](../Infrastructure/Observability%20Provisioning.md)
|
||||||
|
|||||||
@@ -134,9 +134,6 @@ skill-cluster. See [log.md](log.md) for the wave plan and results.
|
|||||||
posture, whole-fleet audit, bulk redaction
|
posture, whole-fleet audit, bulk redaction
|
||||||
- [[SSH and Remote Access]] — passwordless bootstrap, fish login shells,
|
- [[SSH and Remote Access]] — passwordless bootstrap, fish login shells,
|
||||||
GUI-hang diagnosis over ssh
|
GUI-hang diagnosis over ssh
|
||||||
- [[LLM Usage Attribution]] — who-is-using-the-fleet-LLMs research catalog:
|
|
||||||
engine-native identity, gateways, tracing, network/host attribution,
|
|
||||||
phased plan
|
|
||||||
|
|
||||||
**Architecture:**
|
**Architecture:**
|
||||||
|
|
||||||
@@ -185,12 +182,20 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
### Deployment and Topology
|
### Deployment and Topology
|
||||||
|
|
||||||
- [[deployment]] — unified deploy and update process for both servers
|
- [[deployment]] — unified deploy and update process for both servers
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook
|
||||||
|
(2026-10-15)**: the PUBLISHER's container of the official
|
||||||
|
Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
|
||||||
|
swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
|
||||||
|
25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
|
||||||
|
verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
|
||||||
|
disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
||||||
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
||||||
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
||||||
= revert that line (the FP8-native `.rad` stays on disk)
|
= revert that line (the FP8-native `.rad` stays on disk)
|
||||||
- [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon
|
||||||
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
||||||
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
||||||
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
||||||
@@ -383,8 +388,15 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||||
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — the **ACTIVE** frank-zealot
|
- [[frank-zealot-llama-prism-bonsai2-pq20]] — the **ACTIVE** frank-zealot
|
||||||
runbook since 2026-10-07 (late): llama.cpp master (CUDA sm_86) serving
|
runbook since 2026-10-17: the PrismML vendor llama.cpp build 10754 (the only
|
||||||
|
loader for GGML type 142 / `prism.hadamard.*`) serving
|
||||||
|
prism-ml/Ternary-Bonsai-2-27B-gguf **PQ2_0** on the RTX 3090 — TWO
|
||||||
|
200,704-token conversations, q8_0 K + q8_0 V (f16 OOMs at 2×200,704: 34 KiB
|
||||||
|
vs 64 KiB per token over 16 full-attention layers), MTP **off** (no head in
|
||||||
|
the file), vision through the Q8_0 mmproj; boot 22,664 used / 1,461 free
|
||||||
|
- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — RETIRED 2026-10-17 (rollback
|
||||||
|
target, payload on disk): llama.cpp master (CUDA sm_86) serving
|
||||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090 —
|
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090 —
|
||||||
TWO 200,704-token conversations, exact f16 K + f16 V, MTP **impossible**
|
TWO 200,704-token conversations, exact f16 K + f16 V, MTP **impossible**
|
||||||
(the file carries no `nextn`/`mtp` tensors), vision through the publisher's
|
(the file carries no `nextn`/`mtp` tensors), vision through the publisher's
|
||||||
|
|||||||
@@ -1745,3 +1745,154 @@ every restart since the cutover.
|
|||||||
[[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
|
[[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
|
||||||
(the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
|
(the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
|
||||||
|
|
||||||
|
## 2026-10-15 — frank-archon: publisher iq4r payload swap (MODEL_FILE-only)
|
||||||
|
|
||||||
|
- Ask: replace the served model with
|
||||||
|
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` — the engine author's
|
||||||
|
published rad-convert of the OFFICIAL Qwen3.8-Flash-Next (w4nl codebook
|
||||||
|
experts, ten bf16-protected/layer, int8 trunk, PLE/MTP/vision in-container,
|
||||||
|
card KL 0.0789 / top-1 92.85 %) — **no other argument changes**, mindful
|
||||||
|
the operator session runs on the endpoint being swapped.
|
||||||
|
- Disk: Frank authorized one `rm` — the spent Swift1.5 BF16 source (336 GiB)
|
||||||
|
and the two-era-old `qwen4exp-fp8-native.rad` (173 GiB) left
|
||||||
|
`/home/dank/models`; 150 GB → 658 GB free. fp8-native is now rebuild-only
|
||||||
|
(fetch + build scripts); rollback for THIS swap is the w4dn8 `.rad`
|
||||||
|
(on disk) by reverting the commit.
|
||||||
|
- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes
|
||||||
|
121,969,901,568 + Hub LFS sha256, `--verify-only` default) as transient
|
||||||
|
unit `iq4r-fetch`; verified, staged sidecar files kept as provenance.
|
||||||
|
- Shape carried VERBATIM (25600/5120 pair, 44928 pool, fp8 KV 3 × 200,000,
|
||||||
|
MTP 2, wht6, MNBT 2048, froggeric v22.5). Tests re-keyed
|
||||||
|
(`frank_archon_unit.bats` pins the new payload id + sha256, keeps the
|
||||||
|
w4dn8 rollback provenance), homepage card + trio lore + both indexes
|
||||||
|
updated; new runbook [[frank-archon-radiance-qwen38-iq4r]]. Commits
|
||||||
|
`6dcf4a5` (swap) + `f44c398` + `c5ac072` (docs), full bats green.
|
||||||
|
- Cutover self-severed by design (detached `deploy.sh` survived the session
|
||||||
|
death when frank-archon restarted); engine healthy in ~100 s. First-boot
|
||||||
|
ledger: w4nl loads on pin 1.0.13 (no pin bump); kv_attn 26.00 KiB fp8 @
|
||||||
|
607,740 tokens (identical geometry); slab 17,610/17,658 slots per rank
|
||||||
|
(> w4dn8's 17,548 — fewer bf16-protected experts pack more 4-bit table);
|
||||||
|
sampler 9 rows; MTP 2/step; prefix tier RECOVERED 900,176 blocks across
|
||||||
|
the swap (arch+tokenizer unchanged). Vision probe green (reasoning read
|
||||||
|
the solid-red PNG; the 64-token budget went all to thinking — fleet law).
|
||||||
|
- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw rebooted
|
||||||
|
01:41 and the gluetun boot-race dependency-failed `searxng`, `marreta`,
|
||||||
|
and `marreta-browser` (jobs consumed while gluetun's start job retried;
|
||||||
|
`Restart=always` never armed because the services never ran). Cascade
|
||||||
|
restarted; skill codified (`gluetun-dependent-boot-race-recovery`).
|
||||||
|
Gate rerunning against the repaired fleet; frank-archon's own health gate
|
||||||
|
passed every deploy. Open: decode/prefill ladders vs the w4dn8 record,
|
||||||
|
probe leg D — scheduled probe window.
|
||||||
|
|
||||||
|
## 2026-10-15 — authentik: two more human accounts (Dankclaw)
|
||||||
|
- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, no
|
||||||
|
repo objects.** Each `<user>@franksdank.cloud` in the human-user template
|
||||||
|
shape (`is_active=True`, display name, password via `set_password` on stdin
|
||||||
|
only — never argv, so nothing lands in `ps`), groups `Non-Admin Users` +
|
||||||
|
`jellyfin_users` + `openwebui_users`; no admin group granted. Verified
|
||||||
|
through the exact PasswordStage code path (`authenticate()` accepted each
|
||||||
|
password and rejected a wrong one), `app_entitlements(open-webui)` →
|
||||||
|
`Open WebUI Users` for both (identical to the ridley precedent), and
|
||||||
|
`is_superuser=False` / `path=users`. Zero `proxysession` rows for either —
|
||||||
|
groups were set before any login, so no claim purge was due. Downstream:
|
||||||
|
Jellyfin visibility rides the LDAP outpost cache (expect lag); OpenWebUI
|
||||||
|
mints their accounts as role `pending` on first login.
|
||||||
|
- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user template
|
||||||
|
census extended with pk 22–23.
|
||||||
|
|
||||||
|
## 2026-10-17 — frank-zealot: Bonsai-2-27B PQ2_0 trial + cutover (2 × 200,704, q8_0 KV)
|
||||||
|
|
||||||
|
- Ask: evaluate `prism-ml/Ternary-Bonsai-2-27B-gguf` (PQ2_0) as frank-zealot's
|
||||||
|
payload, and specifically measure **2 × 200,704 with q8_0 K + q8_0 V** on the
|
||||||
|
RTX 3090. Fetch script `scripts/download-frank-zealot-bonsai2.sh` (exact
|
||||||
|
bytes + sha256, `--verify-only`) landed first; payload verified at
|
||||||
|
`/home/dank/models/Bonsai2-PQ2_0/` (7,206,168,928 B + 629,246,976 B mmproj).
|
||||||
|
- Runtime door: the file is `file_type 141` with 402 tensors of custom GGML
|
||||||
|
type **142** plus `prism.hadamard.*` metadata — stock llama.cpp refuses it,
|
||||||
|
so the PrismML vendor build 10754 (`2459f68b5`) is forced, not chosen. A
|
||||||
|
source build with `GGML_CUDA_FA_ALL_QUANTS=ON` was started and then
|
||||||
|
cancelled as unnecessary: the vendor build already serves q8_0 V under flash
|
||||||
|
attention at the exact 34,816 B/token (ledger-confirmed).
|
||||||
|
- Trial-window law cost one boot: the scratch env invented `KV_CACHE_TYPE`
|
||||||
|
where the serve script reads `CACHE_TYPE_K`/`CACHE_TYPE_V`, so the script's
|
||||||
|
`:=f16` default silently won and every arm booted f16. That failed window is
|
||||||
|
now the measured f16 wall: `cudaMalloc` OOM on a 25,088.00 MiB KV request,
|
||||||
|
engine fit 33,198 vs 23,558 free.
|
||||||
|
- Four-arm census (production stopped, trap-guarded restore, `bg_12`):
|
||||||
|
**2 × 200,704 q8_0 = 22,664 used / 1,461 free (DEPLOYED)**; 1 × 262,144
|
||||||
|
q8_0 = 18,130 / 5,995 (fits, but serializes compaction); 2 × 262,144 q8_0
|
||||||
|
≈ 25,758 projected (no fit); 2 × 200,704 q4_0 = 16,392 / 7,733 (fallback
|
||||||
|
rung). Recurrent DeltaNet state bills ~150 MiB per SLOT (context 13,627 −
|
||||||
|
KV 13,328 for two slots), which pool arithmetic alone misses.
|
||||||
|
- Behavior probes green: exact-instruction chat, `reasoning_content` present,
|
||||||
|
`tool_calls` with valid JSON arguments (no fork `KNOWN_ISSUES` loop/500),
|
||||||
|
512 px solid-red PNG answered **"Red"**, needle retrieval exact through a
|
||||||
|
25,960-token prompt, 166,027-token prefill clean in 240.0 s. Rates: 76.6 t/s
|
||||||
|
single-stream (9B ran 94.4), 1,258 t/s at 35k, 691.7 t/s at 166k,
|
||||||
|
**109.4 t/s aggregate** at two concurrent streams.
|
||||||
|
- New fleet laws recorded (AGENTS.md + runbook): the mmproj's 873.10 MiB
|
||||||
|
worst-case is NOT inside the engine's fit projection; the first request after
|
||||||
|
a boot is a warmup artifact (2.6 / 37.7 t/s measured); a 64-token thinking
|
||||||
|
budget makes a thinking model look blind; the Bonsai-2 template reads
|
||||||
|
`reasoning_effort` (xhigh/medium/low, raises otherwise) so the harness fold
|
||||||
|
became load-bearing on this endpoint, and it reads `preserve_thinking`.
|
||||||
|
- Cutover: `scripts/frank-zealot.env` (payload + `CACHE_TYPE_*`=q8_0 + ledger
|
||||||
|
lore), `scripts/deploy-remote.sh` (runtime/model/mmproj gates re-keyed, fork
|
||||||
|
hint), `tests/frank_zealot_unit.bats` (payload pair pinned; KV assertions now
|
||||||
|
env-derived instead of an f16 literal), harness `models.yml` (extraBody 1.0
|
||||||
|
/0.95/20 + `preserve_thinking`, effort-map comment corrected —
|
||||||
|
`contextWindow` 200704 and `maxTokens` 16384 unchanged), new runbook
|
||||||
|
[[frank-zealot-llama-prism-bonsai2-pq20]], indexes re-keyed, 9B era retired
|
||||||
|
in place. Full bats green (371).
|
||||||
|
- Base alignment: **Frank chose the OFFICIAL censored payload** — publisher
|
||||||
|
provenance over refusal-freedom. The 9B it replaces was uncensored, so the
|
||||||
|
homepage card and its warning contract were re-keyed with it: zealot's card
|
||||||
|
now carries the base-alignment notice, the shared legal warning belongs to
|
||||||
|
dragoon alone, and the `homepage_config.bats` equality
|
||||||
|
(`zealot.warn == dragoon.warn`) retired with the payload it described.
|
||||||
|
The CRACK re-export (byte-identical geometry per header census) stays on
|
||||||
|
disk as the one-line `MODEL_FILE` fallback if refusals bite.
|
||||||
|
- **Deployed 19:44** (`--service frank-zealot --service homepage`): production
|
||||||
|
boot reproduced the census exactly — 22,664 used / 1,461 free, fit projected
|
||||||
|
21,439 leaving 2,119 ≥ 1,872, KV 13,328.00 MiB at `n_ctx_seq = 200,704`,
|
||||||
|
`n_slots = 2`, projector loaded, alias `frank-zealot` on `/v1/models`. Live
|
||||||
|
probes: exact `pong` + `reasoning_content`, `tool_calls` `{"city":"Seattle"}`,
|
||||||
|
512 px PNG → **Red**, 75.8 t/s decode (1 % off the trial arm). e2e 33/33.
|
||||||
|
- Two live-path stragglers the cutover also had to catch: `serve-frank-zealot.sh`
|
||||||
|
still DEFAULTED to the 9B paths and f16 KV (a defaulted run would have served
|
||||||
|
the retired payload — now re-keyed to the shipped trio), and the OpenWebUI
|
||||||
|
`model` row's `meta.description` is live DB state naming the 9B (re-keyed,
|
||||||
|
backup `webui.db.bak-20261017a`; no `open-webui` restart forced — the cached
|
||||||
|
list picks it up on the next one, and forcing it logs every family session
|
||||||
|
out for one cosmetic string).
|
||||||
|
|
||||||
|
## 2026-10-17 — dankserver disk purge: Swift w4dn8 rollback + orphan caches (docs re-key)
|
||||||
|
|
||||||
|
- Ask: "dump the swift rollback and the GLM GGUF — make sure the model
|
||||||
|
caches and agenttrash are empty as well." Preflight first: `lsof` empty
|
||||||
|
on both payloads, serving trio names only the iq4r path, and a podman
|
||||||
|
layer census proved no deleted path backs an image container (the 121 GB
|
||||||
|
`16b6663d…` layer is the ENGINE image, not a model blob).
|
||||||
|
- Staged to `/home/dank/.AGENTTRASH/20261017-dump-payloads-caches`, then
|
||||||
|
purged with Frank's approval: the Swift1.5 w4dn8 `.rad` (113.6 GiB,
|
||||||
|
rollback target), `GLM-4.7-Flash-UD-Q5_K_XL.gguf` (21 GiB, `/opt/vllm`
|
||||||
|
dir removed — last non-iq4r model on the box), `radiance-state` kvcache
|
||||||
|
tiers (69 GiB + 61 MiB, derived), `~/.radiance-cache-w4a8-*` (11 GiB
|
||||||
|
stale-pin compile cache), `~/.cache/frank-primary` (4.7 GiB stale-name
|
||||||
|
cache). Also emptied every `.AGENTTRASH` on the host (14 dirs: dated
|
||||||
|
batches, quadlet/config holds, both checkouts, `/etc/caddy`). 543 → 763
|
||||||
|
GiB free.
|
||||||
|
- Recovery: the w4dn8 payload is byte-exact on the Hub as
|
||||||
|
`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
|
||||||
|
(122,071,973,888 B, LFS sha256 `09dd9306…`) — rollback is now a
|
||||||
|
re-download, not a local file. Corpora (grams 41 G, kld-ref 24 G,
|
||||||
|
calibration JSONL) deliberately KEPT: re-convert and KLD-gate inputs.
|
||||||
|
- Post-checks: frank-archon active, `/health` ok; iq4r serving payload
|
||||||
|
byte-size intact; live kvcache tier (122 MiB) untouched; no process
|
||||||
|
references a deleted path.
|
||||||
|
- Docs re-keyed: AGENTS.md fleet row + frank-archon section + both runbook
|
||||||
|
index rows; iq4r runbook Rollback rewritten (HF re-fetch procedure — the
|
||||||
|
fetch script's REPO/RAD_NAME/STAGE/DEST/SIZE/SHA256 constants are
|
||||||
|
hardcoded, so retargeting at the mirror is a script edit); w4dn8 runbook
|
||||||
|
header corrected (was still "LIVE", never updated at retirement) and its
|
||||||
|
Rollback table rebuilt.
|
||||||
@@ -90,14 +90,14 @@
|
|||||||
|
|
||||||
<h2>frank-zealot — the fast utility model</h2>
|
<h2>frank-zealot — the fast utility model</h2>
|
||||||
<ul>
|
<ul>
|
||||||
<li><strong>Model:</strong> <a href="https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive" target="_blank" rel="noopener">Qwen3.5-9B-Uncensored</a> (Q6_K) on the RTX 3090 — roughly 95 tokens per second.</li>
|
<li><strong>Model:</strong> <a href="https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf" target="_blank" rel="noopener">Ternary-Bonsai-2-27B</a> (PQ2_0 ternary weights) on the RTX 3090 — roughly 77 tokens per second.</li>
|
||||||
<li><strong>Equivalent:</strong> last year's frontier level (2025).</li>
|
<li><strong>Equivalent:</strong> last year's frontier level (2025).</li>
|
||||||
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
|
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
|
||||||
<li><strong>Inputs:</strong> text, image.</li>
|
<li><strong>Inputs:</strong> text, image.</li>
|
||||||
<li><strong>Best for:</strong> code editor autocomplete, agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture. This is the model Open WebUI offers to family accounts.</li>
|
<li><strong>Best for:</strong> code editor autocomplete, agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture. This is the model Open WebUI offers to family accounts.</li>
|
||||||
<li><strong>Compaction:</strong> if your agent harness supports setting a compaction model, set this one. A compaction job severely slows the bigger models for far longer than it slows this one.</li>
|
<li><strong>Compaction:</strong> if your agent harness supports setting a compaction model, set this one. A compaction job severely slows the bigger models for far longer than it slows this one.</li>
|
||||||
<li><strong>Supervision:</strong> requires tight human supervision.</li>
|
<li><strong>Supervision:</strong> requires tight human supervision.</li>
|
||||||
<li><strong>Warning:</strong> <details class="key-warn"><summary>⚠ uncensored model</summary><div class="warn-body">Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.</div></details></li>
|
<li><strong>Warning:</strong> <details class="key-warn"><summary>⚠ censored base model</summary><div class="warn-body">Base-alignment (censored) model: refusals are possible on sensitive topics, and any model can still be steered into risky output. Uncensored work belongs on frank-dragoon. Take this warning seriously — don’t ruin this server for everybody.</div></details></li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<h2>frank-dragoon — deep reasoning</h2>
|
<h2>frank-dragoon — deep reasoning</h2>
|
||||||
|
|||||||
@@ -295,13 +295,13 @@
|
|||||||
"model": {
|
"model": {
|
||||||
"unit": "zealot",
|
"unit": "zealot",
|
||||||
"role": "Frank's utility model",
|
"role": "Frank's utility model",
|
||||||
"hf": "https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive",
|
"hf": "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf",
|
||||||
"tagline": "HONOR GUIDE ME!",
|
"tagline": "HONOR GUIDE ME!",
|
||||||
"sfx": [
|
"sfx": [
|
||||||
"zealot-honor.ogg"
|
"zealot-honor.ogg"
|
||||||
],
|
],
|
||||||
"equiv": "Equivalent to last year's frontier level (2025).",
|
"equiv": "Equivalent to last year's frontier level (2025).",
|
||||||
"spec": "Qwen3.5-9B-Uncensored (Q6_K) on an NVIDIA RTX 3090 — ~95 tokens/s; 200k-token context, two submissions at a time; sees images.",
|
"spec": "Ternary-Bonsai-2-27B (PQ2_0 ternary weights) on an NVIDIA RTX 3090 — ~77 tokens/s; 200k-token context, two submissions at a time; sees images.",
|
||||||
"inputs": [
|
"inputs": [
|
||||||
"text",
|
"text",
|
||||||
"image"
|
"image"
|
||||||
@@ -314,7 +314,7 @@
|
|||||||
]
|
]
|
||||||
},
|
},
|
||||||
"supervision": "Requires tight human supervision.",
|
"supervision": "Requires tight human supervision.",
|
||||||
"warn": "Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.",
|
"warn": "Base-alignment (censored) model: refusals are possible on sensitive topics, and any model can still be steered into risky output. Take this warning seriously — don’t ruin this server for everybody.",
|
||||||
"compaction": "If your agent harness lets you set a compaction model, set this one: a compaction job on the bigger models blocks them for far longer."
|
"compaction": "If your agent harness lets you set a compaction model, set this one: a compaction job on the bigger models blocks them for far longer."
|
||||||
},
|
},
|
||||||
"url": "",
|
"url": "",
|
||||||
@@ -366,13 +366,13 @@
|
|||||||
"model": {
|
"model": {
|
||||||
"unit": "archon",
|
"unit": "archon",
|
||||||
"role": "Frank's strongest and fastest model runtime",
|
"role": "Frank's strongest and fastest model runtime",
|
||||||
"hf": "https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700",
|
"hf": "https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe",
|
||||||
"tagline": "...Power Overwhelming...",
|
"tagline": "...Power Overwhelming...",
|
||||||
"sfx": [
|
"sfx": [
|
||||||
"archon-power.ogg"
|
"archon-power.ogg"
|
||||||
],
|
],
|
||||||
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
|
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
|
||||||
"spec": "Our own build: a GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next, quantized and tuned in-house (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
|
"spec": "The engine author's published quant: 4-bit codebook experts + int8 trunk of Qwen3.8-Flash-Next (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
|
||||||
"inputs": [
|
"inputs": [
|
||||||
"text",
|
"text",
|
||||||
"image"
|
"image"
|
||||||
|
|||||||
@@ -3,8 +3,10 @@
|
|||||||
#
|
#
|
||||||
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
|
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
|
||||||
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving
|
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving
|
||||||
# the Swift1.5 Qwen3.8-Flash-Next GPTQ-w4 .rad built by
|
# the publisher's 4-bit-codebook .rad of the official Qwen3.8-Flash-Next
|
||||||
# scripts/convert-swift15-w4dn8.sh). It replaces the FreeToken seam of
|
# (fetched by scripts/download-frank-archon-iq4r.sh; the in-house
|
||||||
|
# Swift1.5 w4dn8 .rad of scripts/convert-swift15-w4dn8.sh is the on-disk
|
||||||
|
# rollback). It replaces the FreeToken seam of
|
||||||
# 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
|
# 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
|
||||||
# flags alone do not say: the served window clears the harness worst case,
|
# flags alone do not say: the served window clears the harness worst case,
|
||||||
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
||||||
@@ -14,7 +16,7 @@
|
|||||||
# half of RAM), the served window is what models.yml must
|
# half of RAM), the served window is what models.yml must
|
||||||
# quote, the host-networking contract (no --publish), the image gate on a
|
# quote, the host-networking contract (no --publish), the image gate on a
|
||||||
# name@sha256 ref, the .rad payload gate with its size floor, the
|
# name@sha256 ref, the .rad payload gate with its size floor, the
|
||||||
# chat-template override mount, the self-healing requirement (the run returns before
|
# chat-template override mount, the self-healing law (the run returns before
|
||||||
# the engine is up, so the serve script must wait on the container for
|
# the engine is up, so the serve script must wait on the container for
|
||||||
# Restart=always to mean anything), the deployer's false-green MainPID check,
|
# Restart=always to mean anything), the deployer's false-green MainPID check,
|
||||||
# the vendored hfget fetch path (no token for this ungated repo), and the
|
# the vendored hfget fetch path (no token for this ungated repo), and the
|
||||||
@@ -28,7 +30,7 @@ SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
|
|||||||
TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh"
|
TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh"
|
||||||
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
||||||
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||||
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
|
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-iq4r.sh"
|
||||||
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
||||||
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
||||||
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
||||||
@@ -136,11 +138,10 @@ nosudo_bin() {
|
|||||||
# Concurrency: 3 slots x the served window is the capacity contract, and
|
# Concurrency: 3 slots x the served window is the capacity contract, and
|
||||||
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
|
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
|
||||||
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
||||||
# "seqs 2 cannot boot" is the failure mode of a runtime that compiles
|
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
||||||
# kernels specialized to the batch shape — it refuses a slot count it has
|
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
||||||
# never compiled for. This engine is shape-agnostic: seqs 2 boots, and
|
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
||||||
# measured decode ties seqs 2 and 3 inside the +-3% noise floor — slots
|
# tied inside the +-3% noise floor — slots buy concurrency, not speed.
|
||||||
# buy concurrency, not speed.
|
|
||||||
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
||||||
@@ -148,8 +149,8 @@ nosudo_bin() {
|
|||||||
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
||||||
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
||||||
# KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
|
# KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
|
||||||
# only bf16|fp8 — no per-tensor K/V split), and the PLE n-gram table is
|
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
||||||
# pinned in host RAM so the cards read it without page-faulting mid-token.
|
# in host RAM per Frank's law.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
||||||
@@ -382,18 +383,23 @@ nosudo_bin() {
|
|||||||
grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE"
|
grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the model id matches the payload's build and fetch scripts" {
|
@test "the model id matches the payload's fetch script (and the rollback's build script)" {
|
||||||
# The payload dir/file are the convert script's output layout; the fetch
|
# The deployed payload is the PUBLISHER's .rad: the fetch script names
|
||||||
# script must name the repo the convert stages from. (FP8-native era,
|
# the repo, the file, and the Hub's LFS sha256; nothing is converted on
|
||||||
# the rollback payload, keeps its own trio: build-frank-archon-rad.sh +
|
# the host for it. The in-house w4dn8 rollback keeps its own trio
|
||||||
# download-frank-archon.sh.)
|
# (convert-swift15-w4dn8.sh + download-frank-archon-swift15.sh, and the
|
||||||
|
# fp8-native era keeps build-frank-archon-rad.sh +
|
||||||
|
# download-frank-archon.sh).
|
||||||
grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad$' "$ENV_FILE"
|
||||||
grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
|
grep -q 'StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe' "$DOWNLOAD"
|
||||||
grep -q 'ukisai/Swift1.5-Qwen3.8-Flash-Next' "$DOWNLOAD"
|
grep -q 'qwen3.8-next-flash-fp8-iq4r-moe.rad' "$DOWNLOAD"
|
||||||
|
grep -q '0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20' "$DOWNLOAD"
|
||||||
grep -q 'hfget.sh' "$DOWNLOAD"
|
grep -q 'hfget.sh' "$DOWNLOAD"
|
||||||
# The repo is UNGATED — no token may be required on the fetch path.
|
# The repo is UNGATED — no token may be required on the fetch path.
|
||||||
! grep -q 'HF_TOKEN' "$DOWNLOAD"
|
! grep -q 'HF_TOKEN' "$DOWNLOAD"
|
||||||
|
# Rollback provenance: the w4dn8 build script names its payload.
|
||||||
|
grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "no bare HIP-reserved env var is defined" {
|
@test "no bare HIP-reserved env var is defined" {
|
||||||
|
|||||||
@@ -11,9 +11,8 @@
|
|||||||
# vision file names AND byte counts inside the /models mount, and the
|
# vision file names AND byte counts inside the /models mount, and the
|
||||||
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
|
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
|
||||||
# the container). A serve gate and a deployer gate that disagree is the
|
# the container). A serve gate and a deployer gate that disagree is the
|
||||||
# crash-loop class this file exists to kill: both sides are pinned here,
|
# 2026-10-01 crash-loop; both sides are pinned here, plus the one engine
|
||||||
# plus the one engine rule the card is emphatic about: HALOGEN_SPEC_ADAPT=0 —
|
# law the card is emphatic about: HALOGEN_SPEC_ADAPT=0 — the default
|
||||||
# the default
|
|
||||||
# adaptive policy silently disables the MTP head per request and HALVES
|
# adaptive policy silently disables the MTP head per request and HALVES
|
||||||
# decode.
|
# decode.
|
||||||
#
|
#
|
||||||
@@ -112,10 +111,9 @@ good_env() {
|
|||||||
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
|
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
|
||||||
# The card is emphatic: the default adaptive policy turns the MTP head
|
# The card is emphatic: the default adaptive policy turns the MTP head
|
||||||
# off for whole requests after low acceptance and HALVES decode. The
|
# off for whole requests after low acceptance and HALVES decode. The
|
||||||
# two-slot coupling of the 27B engine — which disabled drafting whenever a
|
# two-slot law does NOT carry over from the 27B engine — on this line
|
||||||
# second slot existed — does NOT carry over: on this line batching and
|
# batching and speculation coexist (the card's quickstart ships 2
|
||||||
# speculation coexist (the card's quickstart ships 2 slots WITH the head),
|
# slots WITH the head), so 2 is concurrent capacity, not a tax.
|
||||||
# so 2 is concurrent capacity, not a tax.
|
|
||||||
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
|
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
|
||||||
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
|
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
|
||||||
# The cap/timeout pair is coupled (FLAGS.md table): a request at the
|
# The cap/timeout pair is coupled (FLAGS.md table): a request at the
|
||||||
@@ -180,9 +178,8 @@ good_env() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "the serve script refuses a checkpoint of the wrong byte count" {
|
@test "the serve script refuses a checkpoint of the wrong byte count" {
|
||||||
# A truncated curl exit-0 artifact must not boot: curl exits 0 on an HTTP
|
# A truncated curl exit-0 artifact must not boot; the Hub's byte
|
||||||
# error and writes the error body as the file, so the Hub's declared byte
|
# count is the contract (fleet fetch law).
|
||||||
# count is the contract that catches it.
|
|
||||||
podman_stub
|
podman_stub
|
||||||
good_env
|
good_env
|
||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
@@ -283,7 +280,7 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
|
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the shipped env keeps the harness fit budget inside the engine ceiling" {
|
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
||||||
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
||||||
# (a longer prompt is a 400 naming the limit, never a silent
|
# (a longer prompt is a 400 naming the limit, never a silent
|
||||||
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
||||||
|
|||||||
@@ -1,13 +1,14 @@
|
|||||||
# Seam: the frank-zealot llama.cpp root unit on dankclaw.
|
# Seam: the frank-zealot llama.cpp root unit on dankclaw.
|
||||||
# frank-zealot is the RTX 3090 endpoint; since 2026-10-07 it serves
|
# frank-zealot is the RTX 3090 endpoint; since 2026-10-17 it serves
|
||||||
# HauhauCS's Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K, qwen35 hybrid:
|
# prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 — the ternary Bonsai-2-27B, a
|
||||||
# 32 layers, 8 full-attention + 24 Gated DeltaNet, no MTP tensors in the file)
|
# qwen35 hybrid with 64 blocks (16 full-attention + 48 Gated DeltaNet, no
|
||||||
# via the CUDA (sm_86) build of llama.cpp master: TWO 200,704-token
|
# nextn/mtp tensors in the file) — via the PrismML fork build of llama.cpp
|
||||||
# conversations (--ctx-size 401408 --parallel 2), exact f16 K + f16 V under
|
# (vendor build 10754), the ONLY loader that registers the payload's custom
|
||||||
# flash attention, the publisher's own BF16 vision projector, and MTP off —
|
# tensor type 142 and honours its prism.hadamard.* metadata: TWO 200,704-token
|
||||||
# the file carries no nextn/mtp tensors, so there is no in-file head to draft
|
# conversations (--ctx-size 401408 --parallel 2), q8_0 K + q8_0 V under flash
|
||||||
# with. Log verbosity 4 ships so the VRAM-fit forensics (KV layout, buffer
|
# attention (f16 KV does not fit this card at this pool — the boot dies on a
|
||||||
# sizes, graph capture) land in the journal.
|
# 25,088 MiB cudaMalloc), the publisher's own Q8_0 vision projector, and MTP
|
||||||
|
# off. Log verbosity 4 ships so the VRAM-fit forensics land in the journal.
|
||||||
# This seam runs the serve script against a stub engine and asserts on the
|
# This seam runs the serve script against a stub engine and asserts on the
|
||||||
# argv it actually execs, so what is pinned is the command the operator gets.
|
# argv it actually execs, so what is pinned is the command the operator gets.
|
||||||
|
|
||||||
@@ -95,10 +96,15 @@ run_stage() {
|
|||||||
grep -q 'FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env' "$UNIT"
|
grep -q 'FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env' "$UNIT"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the shipped env points at the llama.cpp runtime and the Qwen3.5-9B Q6_K GGUF" {
|
@test "the shipped env points at the fork runtime and the Bonsai-2 PQ2_0 pair" {
|
||||||
# The payload path IS the endpoint's identity.
|
# The payload path IS the endpoint's identity, and the runtime is part of
|
||||||
grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server$' "$ENV_FILE"
|
# it here: the file's custom tensor type 142 + prism.hadamard.* metadata
|
||||||
grep -qE '^MODEL_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3\.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K\.gguf$' "$ENV_FILE"
|
# make the PrismML build the only loader, so a retarget that moves the
|
||||||
|
# model but leaves the master build behind ships an endpoint that cannot
|
||||||
|
# load at all.
|
||||||
|
grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-prism/llama-server$' "$ENV_FILE"
|
||||||
|
grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0\.gguf$' "$ENV_FILE"
|
||||||
|
grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-mmproj-Q8_0\.gguf$' "$ENV_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the deployer gates on exactly the payload paths the env file ships" {
|
@test "the deployer gates on exactly the payload paths the env file ships" {
|
||||||
@@ -135,19 +141,23 @@ run_stage() {
|
|||||||
[ $((pool / slots)) -eq 200704 ]
|
[ $((pool / slots)) -eq 200704 ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the KV contract is exact f16 K + f16 V, flash-attn on" {
|
@test "the KV dtype pair follows the env file and flash-attn stays pinned on" {
|
||||||
|
# Asserted against the env rather than a literal so a retune moves one
|
||||||
|
# file, not two. The shipped pair is q8_0/q8_0: this payload bills
|
||||||
|
# 16 full-attention layers × 4 KV heads × 256 dim = 16,384 elements per
|
||||||
|
# token per cache, so f16/f16 would need 25,088 MiB of the 24,123 MiB card
|
||||||
|
# (measured: cudaMalloc OOM, engine fit 33,198 vs 23,558 free) while q8_0
|
||||||
|
# bills 34,816 B/token = 13,328.00 MiB at the 401,408 pool. The boot
|
||||||
|
# ledger, not this file, is the referee. flash-attn is not decoration:
|
||||||
|
# quantized V is only legal with it, and it must stay explicit so a
|
||||||
|
# future quant swap cannot silently fall back.
|
||||||
|
local env_k env_v
|
||||||
|
env_k="$(sed -nE 's/^CACHE_TYPE_K=(.+)$/\1/p' "$ENV_FILE")"
|
||||||
|
env_v="$(sed -nE 's/^CACHE_TYPE_V=(.+)$/\1/p' "$ENV_FILE")"
|
||||||
serve_argv
|
serve_argv
|
||||||
[ "$status" -eq 0 ]
|
[ "$status" -eq 0 ]
|
||||||
# Frank, 2026-10-07: the 9B rung funds unquantized KV (the 26B MoE era
|
[ "$(argv_value --cache-type-k)" = "$env_k" ]
|
||||||
# shipped q8_0 V because the card was full). qwen35 bills the pool on the
|
[ "$(argv_value --cache-type-v)" = "$env_v" ]
|
||||||
# 8 full-attention layers only (8 × 4 KV heads × 256 dim = 8,192
|
|
||||||
# elements/token ⇒ 32 KiB/token at f16/f16, 12.25 GiB at the 401,408
|
|
||||||
# pool); the 24 DeltaNet layers cost a fixed recurrent state per slot.
|
|
||||||
# The boot ledger, not this file, is the referee. flash-attn stays pinned:
|
|
||||||
# any quantized-V fallback (q8_0 = this build's FP8 class) requires it, and
|
|
||||||
# the build carries GGML_CUDA_FA_ALL_QUANTS=ON.
|
|
||||||
[ "$(argv_value --cache-type-k)" = "f16" ]
|
|
||||||
[ "$(argv_value --cache-type-v)" = "f16" ]
|
|
||||||
[ "$(argv_value --flash-attn)" = "on" ]
|
[ "$(argv_value --flash-attn)" = "on" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -105,8 +105,10 @@ for s in fleet:
|
|||||||
assert [s["name"] for s in fleet] == ["frank-zealot", "frank-dragoon", "frank-archon"], \
|
assert [s["name"] for s in fleet] == ["frank-zealot", "frank-dragoon", "frank-archon"], \
|
||||||
"card render order must be zealot, dragoon, archon: " + str([s["name"] for s in fleet])
|
"card render order must be zealot, dragoon, archon: " + str([s["name"] for s in fleet])
|
||||||
w = {s["name"]: s["model"].get("warn", "") for s in fleet}
|
w = {s["name"]: s["model"].get("warn", "") for s in fleet}
|
||||||
assert w["frank-zealot"] == w["frank-dragoon"] != "", "the legal warnings must be the common uncensored text"
|
assert "legal risk" in w["frank-dragoon"].lower() and "everybody" in w["frank-dragoon"].lower(), \
|
||||||
assert "legal risk" in w["frank-zealot"].lower() and "everybody" in w["frank-zealot"].lower()
|
"the abliterated endpoint must carry the common legal warning"
|
||||||
|
assert "censored" in w["frank-zealot"].lower() and "uncensored" not in w["frank-zealot"].lower(), \
|
||||||
|
"zealot ships the censored base since 2026-10-17: its card warning must not claim uncensored output"
|
||||||
archon = next(s for s in fleet if s["name"] == "frank-archon")
|
archon = next(s for s in fleet if s["name"] == "frank-archon")
|
||||||
assert "permission" in archon["model"], "archon card needs the ask-Frank permission callout"
|
assert "permission" in archon["model"], "archon card needs the ask-Frank permission callout"
|
||||||
PY
|
PY
|
||||||
|
|||||||
Reference in new issue
Block a user