Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
71c6b5e758 | ||
|
|
5ced474eb9 | ||
|
|
2992a7ddd6 | ||
|
|
7c77326c33 | ||
|
|
65dec6a80e | ||
|
|
f99b698d25 | ||
|
|
757a90057f | ||
|
|
009e8a885d | ||
|
|
6b677139ad | ||
|
|
184a5c665a | ||
|
|
958b10545e | ||
|
|
c4dca68a03 | ||
|
|
f29dbdf7c7 | ||
|
|
98f11a19c1 | ||
|
|
a30ac9457a | ||
|
|
75695c312a | ||
|
|
c507f91438 | ||
|
|
b175f4afd1 | ||
|
|
971e23a25c | ||
|
|
3ca64e14fc | ||
|
|
f3dfea13ad | ||
|
|
02fa1e22ae | ||
|
|
1aef7418ad | ||
|
|
fa5a444b64 | ||
|
|
90f678582e | ||
|
|
cc0fdd15a1 | ||
|
|
f148ecab4a | ||
|
|
c2f3411342 | ||
|
|
8c5bde7034 | ||
|
|
22a7331100 | ||
|
|
c1e192df00 | ||
|
|
8375567de3 | ||
|
|
79c2e64c2c | ||
|
|
f20da6a6ff | ||
|
|
058ea6cca8 | ||
|
|
3f8531179e | ||
|
|
3c5d77d1fa | ||
|
|
203b0be145 | ||
|
|
74da746c51 | ||
|
|
7066accc25 | ||
|
|
76aaa2543b | ||
|
|
8220030ba1 | ||
|
|
5191c49967 | ||
|
|
f9330fa201 | ||
|
|
4b71c014e9 | ||
|
|
cee90eea52 | ||
|
|
73c37f4301 | ||
|
|
6632d0a2f6 | ||
|
|
caf5e1e0ca | ||
|
|
5278f5be26 | ||
|
|
ec6184de9a | ||
|
|
1d062f06a3 | ||
|
|
b51e078e36 | ||
|
|
68a525740d | ||
|
|
5b7b3ccb90 | ||
|
|
2137080d1c | ||
|
|
f747cb0dab | ||
|
|
db8faa68ff | ||
|
|
cf6c62d44a | ||
|
|
0aafad5f2c | ||
|
|
a3f45ce028 | ||
|
|
45de2fd47f | ||
|
|
fa858f4c10 | ||
|
|
07af1a9366 | ||
|
|
f0d0614ce6 | ||
|
|
9aed9987b6 | ||
|
|
4048dc8630 | ||
|
|
9bc40fd899 | ||
|
|
bb60e8524f | ||
|
|
319dd0f7da | ||
|
|
650dfff5a2 | ||
|
|
de70320cde | ||
|
|
910c54b6ff | ||
|
|
8c887e6fc9 | ||
|
|
49dd2be4cd | ||
|
|
a11f703c49 | ||
|
|
6cdf88f611 | ||
|
|
259fc5505d | ||
|
|
3e46d51df8 | ||
|
|
d062a376b2 | ||
|
|
95219cf4c1 | ||
|
|
fee66187a1 | ||
|
|
0258ae0897 | ||
|
|
9aee00f2b3 |
No files matched your search
@@ -25,9 +25,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
|
||||
| Target | Service | Runtime | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *…
|
||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 |
|
||||
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp** (27B dense qwen3_5 hybrid, 10,442,827,392 B; a per-tensor allocation mix of stock types; the MTP head in the file **drafted** — **spec ON** (draft-mtp depth 3) since 2026-10-06 evening, ngram skipped on the recurrent-rollback law) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **2 × 114,688** (Frank's 112k cap each since the 2026-10-06 late reshape, cut from 131,072 to fund the **ubatch ladder 512 → 1024 → 2048** — **1,005 MiB free at boot, 763 settled after a vision probe** (encoder buffers resident)) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / **ubatch 2048** (the ceiling for this shape — 4096 would not fit); **prefill 1,209 t/s @ 3,760 tokens; ~52–58 t/s recipe decode** (MTP acceptance 34–54 %, temp-1.0 noise; decode is ubatch-invariant). Renamed from `frank-small` 2026-10-06 (87721ab). Supersedes the same-morning IQ3_S-mtp payloads: the two-slot no-spec shape and the one-slot MTP shape (both 2026-10-06; payload on disk).|7771|
|
||||
| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||
|
||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||
the [runbook index](#runbook-index). The full swap history lives in
|
||||
@@ -286,9 +286,17 @@ deletes the unit file from `/etc/systemd/system/`, not just the symlink.
|
||||
The harness reaches the fleet through `.omp/agent/models.yml` in the
|
||||
operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
|
||||
where the harness runs; there is no copy on the servers. Each provider is one
|
||||
HOST:PORT; each model `id` is the served alias: `frank-primary` →
|
||||
`192.168.0.235:7770`, `frank-large` → `192.168.0.143:7770`, `frank-secondary` →
|
||||
`modelRoles`.
|
||||
HOST:PORT; each model `id` is the served alias. **Official model ids are
|
||||
the StarCraft unit names (2026-10-08) and since 2026-10-09 they are also the
|
||||
ids on the wire:** `frank-archon` → `192.168.0.235:7770`
|
||||
(`FRANK_PRIMARY_SERVED_NAME`), `frank-dragoon` → `192.168.0.143:7770`
|
||||
(`HALOGEN_MODEL_ID`), `frank-zealot` → `192.168.0.143:7771` (`ALIAS`). The
|
||||
SERVICES, units, scripts, env files and runbooks keep the old names
|
||||
(`frank-primary`, `frank-large`, `frank-secondary`); only the served alias,
|
||||
the `models.yml`/`config.yml` ids, the homepage cards, the OpenWebUI grants
|
||||
and the Hermes routing were renamed. Verify a rename on `/v1/models`, never
|
||||
on a chat response: radiance echoes the **requested** model name, so a 200
|
||||
proves nothing about the served id (llama.cpp and halogen ignore it).
|
||||
|
||||
Resolution rules: the default model resolves as a `defaultProvider` +
|
||||
`defaultModel` pair (the provider must serve the model); a subagent override
|
||||
@@ -308,11 +316,13 @@ image]`) — `frank-large` returned to vision with the flash-server v2 swap
|
||||
27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
|
||||
are the engine core; `--mm-max-patches` available),
|
||||
and `frank-secondary` through
|
||||
the publisher's own `mmproj-BF16.gguf` passed as `--mmproj` — the Swift
|
||||
GGUF itself carries no vision tensors. `frank-secondary`'s `contextWindow` is
|
||||
**114688** (cut from 131072 on 2026-10-06 late to fund the ubatch ladder to 2048), the
|
||||
per-slot window, not the model's native 262144: the harness must compact
|
||||
before llama.cpp answers a longer prompt with a 400. Note also
|
||||
the publisher's own BF16 mmproj passed as `--mmproj` — the model is natively
|
||||
multimodal but the Q6_K GGUF carries no vision tensors (same arrangement the
|
||||
Gemma4 era used). `frank-secondary`'s `contextWindow` is
|
||||
**200704** — the PER-SLOT window (`--ctx-size 401408 --parallel 2` since the
|
||||
2026-10-07 Qwen3.5-9B swap), not the pool and not the model's native 262144:
|
||||
the harness must compact before llama.cpp answers a longer prompt with a 400.
|
||||
Note also
|
||||
that
|
||||
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
|
||||
parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
|
||||
@@ -321,7 +331,7 @@ There is **no audio**
|
||||
input anywhere in the fleet: that capability left with the FLM NPU
|
||||
endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||
(the MiMo tower is vision-only).
|
||||
(neither the MiMo nor the Qwen3.5 tower carries audio).
|
||||
|
||||
### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
|
||||
|
||||
@@ -365,15 +375,22 @@ endpoint).
|
||||
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
||||
policy disables the MTP head for whole requests after low acceptance and
|
||||
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
||||
for it (accepted). `HALOGEN_CTX=200192` is the PER-SLOT hard ceiling (400
|
||||
naming the limit, never truncation); `models.yml` `contextWindow: 200192`
|
||||
must match, and compaction at 70 % = 140,134 + 57,344 budget = 197,478 fits.
|
||||
for it (accepted). `HALOGEN_CTX=262144` is the PER-SLOT hard ceiling (400
|
||||
naming the limit, never truncation) — RAISED from 200192 on 2026-10-07
|
||||
under the 400-zone law: the harness clamps max_tokens to
|
||||
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
|
||||
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
|
||||
request 43,158 vs room 42,804), so a harness window == engine ceiling
|
||||
made every request over ~146k est a guaranteed 400. `models.yml`
|
||||
`contextWindow: 217088` leaves the 45k gap that absorbs the estimate
|
||||
error (worst-case request ≈ 219k ≤ 262144); compaction fires at 85 % of
|
||||
the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
|
||||
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
||||
law does NOT carry over; batching and speculation coexist (the card's
|
||||
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
||||
`HALOGEN_KV_POOL_POSITIONS=400384` (explicit 2 × ctx; ~28 KiB/position ⇒
|
||||
~10.7 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
harness always sends 57,344); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||
`HALOGEN_KV_POOL_POSITIONS=524288` (explicit 2 × ctx; ~28 KiB/position ⇒
|
||||
14.0 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
|
||||
1.0/0.95/20, the card's thinking recipe) — an absent temperature is no
|
||||
@@ -389,8 +406,8 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
||||
"video".
|
||||
|
||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
||||
weights ~62 + KV ~10.7 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
||||
~2 ≈ 82.7 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~18 GiB
|
||||
weights ~62 + KV 14.0 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
||||
~2 ≈ 86 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~15 GiB
|
||||
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
|
||||
page cache beside the checkpoint — the card's quickstart arithmetic does
|
||||
not bill it as resident, but READ the boot ledger and re-size if the
|
||||
@@ -417,152 +434,131 @@ that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
|
||||
`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
|
||||
`GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
|
||||
(`scripts/frank-secondary.service`, `relink_unit`ed into
|
||||
`/etc/systemd/system/`), serving `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`
|
||||
at **RCO-IQ3_XXS -mtp** (10,442,827,392 B, sha256 `e13d3e74…`) **plus
|
||||
the publisher's own `mmproj-BF16.gguf`** (931,145,888 B, sha256
|
||||
`19acd7fc…`), both `/home/dank/models/Swift15-UncIQ3XXS/`, on port 7771.
|
||||
Cutover 2026-10-05 (evening) on Frank's explicit contract: **128k
|
||||
context cap, KV at FP8, vision on, as many 128k contexts as fit**;
|
||||
reshaped three times on 2026-10-06 — morning to two 131,072 slots
|
||||
no-spec, evening to this UNCERSED RCO-IQ3_XXS payload with MTP back
|
||||
on, late the RENAME to **frank-secondary** (87721ab) at two 114,688
|
||||
slots with the ubatch ladder climbing 512 → 1024 → 2048 (f2575a1,
|
||||
d8da7a9). The
|
||||
same-day payloads it supersedes: the LIVE MiMo-V2.6 Q6_K shape and the
|
||||
Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never
|
||||
deployed (blocked at the docs gate; the docs debt is what this section
|
||||
settles).
|
||||
`/etc/systemd/system/`), serving
|
||||
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive` at **Q6_K**
|
||||
(7,359,259,008 B, sha256 `c0ba7beb…` computed locally — the Hub carries no
|
||||
LFS oid) **plus the publisher's own BF16 mmproj** (921,704,448 B), both
|
||||
`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`, on port 7771. Cutover
|
||||
2026-10-07 (late) from the Gemma4-26B-A4B Q4_K_M 1×200,704 shape (same day).
|
||||
|
||||
**Architecture: qwen3_5 hybrid, bigger** — block_count 65 (64 real layers +
|
||||
the blk.64 MTP head), `full_attention_interval 4` ⇒ **16 full-attention +
|
||||
48 Gated DeltaNet**, native 262,144 window, dense. KV exists only in the 16
|
||||
attention layers (4 KV heads × 256 dim): 64 KiB/token at f16, **34 KiB/token
|
||||
at q8_0**. The DeltaNet state is a fixed per-slot buffer (~150 MiB/slot at
|
||||
48 linear layers × 48 value heads). Header facts were read from the Hub
|
||||
file by HTTP range + struct parse before anything was downloaded.
|
||||
**Architecture: qwen35 dense hybrid** — read from the GGUF's own header, not
|
||||
the model card: 32 layers, `full_attention_interval = 4` ⇒ **8 full-attention
|
||||
layers (3, 7, 11, …, 31) + 24 Gated DeltaNet linear layers**, 16 attention
|
||||
heads / **4 KV heads** × 256 dim, `ssm` conv_kernel 4 / state_size 128 /
|
||||
group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144, vocab
|
||||
248,320. Tensor types are Q6_K × 250 + F32 × 177 — standard quants only, so
|
||||
the stock master build loads it (arch `qwen35` is registered in this build's
|
||||
`src/llama-arch.cpp`). **KV accounting:** the pool scales with the 8 full
|
||||
layers only — 8 × 4 × 256 = 8,192 elements/token per cache, so f16 K + f16 V
|
||||
bills **32 KiB/token** = 12.25 GiB at the 401,408 pool; the 24 DeltaNet
|
||||
layers bill a fixed recurrent state per SLOT, not per token, which is exactly
|
||||
why two 200k slots fit where the 26B MoE fit one.
|
||||
|
||||
**The shape (2026-10-06 midnight; Frank's tight-fit order, MTP back on
|
||||
in the evening, per-slot cut + ubatch climbed to 2048):** `--ctx-size
|
||||
229376 --parallel 2` = **TWO 114,688-token conversations** (Frank's
|
||||
112k cap each; `n_ctx_slot = CTX_SIZE / PARALLEL`; cut from 131,072 to
|
||||
fund the ubatch rungs), `--flash-attn on` pinned, **q8_0 K + q8_0 V**
|
||||
(llama.cpp's FP8 class — no true fp8 KV dtype exists; 8.5 effective
|
||||
bits; supersedes the same-day f16/f16 rule written for the Ornith 9B
|
||||
fit), all layers on the GPU, `--mmproj` for vision, batch 4096 /
|
||||
ubatch 2048. **Speculative decoding is ON** (draft-mtp, depth 3): the
|
||||
-mtp file's head is drafted against the main model's own weights;
|
||||
ngram-mod skipped on purpose — on a qwen35 hybrid ngram disables
|
||||
recurrent-state rollback, so a rejected draft corrupts the DeltaNet
|
||||
state.
|
||||
**The shape (2026-10-07):** `--ctx-size 401408 --parallel 2` = TWO
|
||||
conversations of 200,704 (Frank's cap; the harness `contextWindow` quotes
|
||||
the per-slot number), `--flash-attn on`, **f16 K + f16 V** (exact KV),
|
||||
batch 4096 / **ubatch 2048** (the measured CUDA optimum: +4.6 % prefill,
|
||||
decode flat, 616 MiB — MiMo-era A/B on this card), `--log-verbosity 4`,
|
||||
`--mmproj` for vision, **speculative decoding OFF** (`SPEC_TYPE=none`).
|
||||
|
||||
**Measured at the midnight boot (2026-10-06, 2 × 114,688 / ubatch
|
||||
2048): 23,120 used / 1,005 MiB free, settling 763 MiB after a vision
|
||||
probe** (`nvidia-smi` free; the encoder buffers stay resident on this
|
||||
boot) — the per-slot cut freed ~1.05 GiB of KV and both ubatch rungs
|
||||
spent it back (~0.3 + ~0.75 GiB measured; the 1024 rung had landed
|
||||
22,374/1,751, the 2048 rung cost 746 MiB, ~130 over the 616 the rung
|
||||
measured on the MiMo shape). The MTP draft ctx costs **~2.4 GiB
|
||||
measured** (23,636 − 22,728 + 1,561 at the 262,144 draft window) — ~6×
|
||||
the ~0.4 GiB the one-slot era assumed, because the drafter's KV is
|
||||
built against the target's FULL context, not the 3-token proposal
|
||||
depth; the lighter RCO-IQ3_XXS quant (−1.56 GiB vs IQ3_S-mtp) funds it.
|
||||
**Prefill 1,209 t/s @ 3,760 tokens; decode 52.0–58.2 tok/s (recipe
|
||||
temp 1.0) / 66.9 (temp 0)**, acceptance 34.2–54.4 % — decode is
|
||||
ubatch-invariant, the spread is temp-1.0 acceptance noise.
|
||||
`reasoning_effort` probe unchanged: `xhigh` 200, `high` → 500 Jinja
|
||||
raise (the models.yml fold stays mandatory).
|
||||
**Why MTP is off**: it is not a choice — **this Q6_K file carries no
|
||||
`nextn`/`mtp` tensors** (verified from the tensor-name list), and MTP is a
|
||||
property of the file, not of the flags. The publisher card advertises "MTP
|
||||
support" for the base architecture; the quantized export dropped the head.
|
||||
|
||||
**Vision with the publisher's own projector**: ajgazin's repo SHIPS
|
||||
`mmproj-BF16.gguf` in the same repo (the IQ3_S era's unsloth mmproj was
|
||||
built from the SHARED BASE `Qwen/Qwen3.8-27B`, legal because Swift-1.5's
|
||||
vision stack was hash-proven byte-identical to the base). The probe
|
||||
traps carry over: ≥256 px probe images or the model answers "video";
|
||||
only the OpenAI `image_url` content part reaches the model. **New on
|
||||
this payload: the uncensored model thinks more than the censored
|
||||
Swift 1.5** — a 64-token budget is eaten entirely by
|
||||
`reasoning_content` (empty `content`); 128 tokens clears the vision
|
||||
probe.
|
||||
**Measured boots** (nvidia-smi, the referee): the deployed boot is
|
||||
**21,436 used / 2,689 free** (ubatch 2048) — the pricing boots were the
|
||||
MiMo-V2.6-Distill-Qwen-9B pair at this exact geometry (**21,276 / 2,849** at
|
||||
ubatch 1024, **21,846 / 2,279** at 2048), so the 415 MiB lighter weights
|
||||
bought back what the projection predicted. The engine ledger confirms the KV
|
||||
law to the MiB: `CUDA0 KV buffer size = 12544.00 MiB` (`K (f16): 6272.00`,
|
||||
`V (f16): 6272.00`), compute buffers 1,136 MiB on CUDA0 + 848 MiB host, the
|
||||
mmproj 879 MiB + 248 MiB. Decode is ubatch-invariant (Swift-era 3090 law). The
|
||||
Q6_K rung costs ~10 % decode vs Q5_K_M (weight bandwidth; MiMo-measured 94.4
|
||||
vs 104.8 t/s).
|
||||
|
||||
The RCO-IQ3_XXS tier is a **per-tensor allocation mix** of stock quant
|
||||
types (IQ3_S 97 / BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32
|
||||
/ IQ2_XXS 30 / Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — stock
|
||||
llama.cpp loads it; the tensor-type histogram is the law (see Key
|
||||
Lessons). Publisher KLD vs Swift BF16: mean 0.002631 / 90th 0.042136 /
|
||||
max 34.49 — RCO is the publisher's "more experimental" family, and the
|
||||
payload is the UNCECERSED Swift 1.5 (23/100 refusals vs Swift 1.5's
|
||||
98/100, KL 0.0884).
|
||||
**Vision**: the publisher's own BF16 mmproj via `--mmproj` (the model is
|
||||
natively multimodal; the Q6_K file carries no vision tensors). Probe traps
|
||||
carry over: ≥256 px probe images or the model answers "video", and only the
|
||||
OpenAI `image_url` content part reaches the model.
|
||||
|
||||
**Compaction hub**: `models.yml` routes `compactionModel` for
|
||||
`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here — 200,704 is the
|
||||
only window in the fleet that fits a worst-case compaction input (~170k
|
||||
tokens). With `--parallel 2` a compaction request no longer serializes behind
|
||||
a live generation (the Gemma4 era's `--parallel 1` law).
|
||||
|
||||
Runbook:
|
||||
`serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md`.
|
||||
Rollback: `/home/dank/models/Swift15-IQ3S/` (the IQ3_S-mtp payload live
|
||||
until the 2026-10-06 evening swap — revert da36fa3 for the two-slot
|
||||
no-spec shape) or `/home/dank/models/MiMo-Q6K/` (revert to the
|
||||
2×200,704 f16 shape — [[frank-secondary-llama-mimo-q6k-2x200704]]) or
|
||||
`/home/dank/models/Ornith-Q8/` (the committed-never-deployed Q8_0;
|
||||
revert to 380aafa's tree).
|
||||
`serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md`.
|
||||
Rollback: revert the swap commit — the Gemma4 payload
|
||||
(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
|
||||
(`/home/dank/models/MiMo-Q6K/`), the Swift payload
|
||||
(`/home/dank/models/Swift15-UncIQ3XXS/`) and the Ornith Q8_0 payload
|
||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||
|
||||
### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE)
|
||||
|
||||
### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
||||
|
||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||
`stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…` digest-pinned,
|
||||
287 MiB, `libr4d@0.5.0` kernels prebuilt for gfx1201, engine as the image
|
||||
entrypoint) serving the **official `Qwen/Qwen3.8-Flash-Next-FP8`
|
||||
checkpoint at NATIVE precision** — modelopt block-FP8 throughout, no
|
||||
requantisation, no calibration. Plain `podman run` through
|
||||
`scripts/serve-frank-primary.sh` under the root systemd unit
|
||||
(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the
|
||||
container name is the unit name (`frank-primary`). Cutover LIVE 2026-10-06
|
||||
00:55 (chain built 2026-10-05/06);
|
||||
rollback tag `frank-primary-freetoken-pr587-20261005` (the FreeToken trio
|
||||
+ Dockerfile are in the tag; the image and the orcarouter payload stay on
|
||||
disk until the window closes), older rollback `frank-primary-vllm-shapeG-20261003`.
|
||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
|
||||
systemd unit, container name = unit name) serving
|
||||
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
|
||||
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
|
||||
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
|
||||
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
|
||||
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
|
||||
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
|
||||
with the token knob; the RADGRAM2 grams are format-identical across pins and
|
||||
cross the boundary by design), **24 whole experts per layer cast bf16** (the
|
||||
PROTECTED set, derived from the calibration grams' energy share),
|
||||
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
|
||||
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
|
||||
The plugin's **one-table-one-encoding law** (measured at rad-convert
|
||||
declare): one expert table takes ONE encoding; only whole bf16 experts are
|
||||
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
|
||||
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
|
||||
reference (recorded on the same 1.0.13 pin) is the precision referee;
|
||||
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
|
||||
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
|
||||
Rollback: revert the swap commit — the fp8-native `.rad`
|
||||
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
||||
|
||||
**Payload chain**: official checkpoint → staging dir whose `config.json`
|
||||
drops `quantization_config` → `stack-qwen38-official-fp8.py` (two
|
||||
value-preserving reshapes: experts byte-stacked into the plugin's stacked
|
||||
`gate_up_proj`/`down_proj` layout with `_scale_inv` at the same index — 49
|
||||
stacks × 512 experts, no requantisation; PLE's 128 scaleless fp8 shards
|
||||
decoded to one bf16 shard with the checkpoint's own bf16 scale
|
||||
1.9931793212890625e-04) → `rad-convert` with ONE `--quant` rule — pure
|
||||
copy for every weight, and the rtn rule re-encodes the bf16 PLE at that
|
||||
same fixed scale, an exact round-trip that restores the original codes and
|
||||
lands the plugin's one-element scale plane →
|
||||
`/home/dank/models/rad/qwen4exp-fp8-native.rad` (177,109 MiB measured,
|
||||
floor `FRANK_PRIMARY_RAD_MIN_MIB=177000`). The staging drop is the plugin
|
||||
SELECTION law (spec §2.4): the descriptor is `(arch_id, quant)` matched
|
||||
exactly, no shipped plugin claims `(qwen4exp, "fp8")`, the shipped
|
||||
`qwen4exp_fp8` plugin claims `(qwen4exp, "")` — and the descriptor is only
|
||||
the selection key, per-weight encodings are data, so nothing else is
|
||||
requantised.
|
||||
Raw serve AND unmodified convert both refuse; that is the design. The
|
||||
checkpoint itself is sha256-verified against the Hub (131/131 shards,
|
||||
2026-10-05). Built by `scripts/build-frank-primary-rad.sh` (`--plan-only`
|
||||
first — its declare phase is the engine's own, so a resolving plan boots).
|
||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
||||
`--max-model-len 200000` (the harness contextWindow must quote this),
|
||||
`--placement expert_tiered --host-pool-mib 40960` + **explicit VRAM
|
||||
partition `--vram-weights-mib 26071 --vram-kv-mib 3543`** (the vram-pair
|
||||
replaced `--expert-vs-cache-ratio`; the pair sizes the fp8-KV shape at the
|
||||
deployed boot), `--kv-cache-dtype fp8` (inherited from the fp8-native
|
||||
env), `--num-speculative-tokens 2`,
|
||||
`--max-num-batched-tokens 2048`, prefix cache 4 GiB pinned host + 64 GiB
|
||||
disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric
|
||||
v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`,
|
||||
`--override-chat-template`). Boot ledger: KV pool 3.46 GiB (~480k fp8
|
||||
tokens), two expert pools 57.84 GiB, cards settled 1.41 GiB free.
|
||||
|
||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6` (fused lossy all-reduce,
|
||||
+5.6–6.3 % prefill / ~−1.3 % 8-stream decode at TP2), `--max-num-seqs 2`
|
||||
(Frank's first boot; raise from the ledger + slot occupancy), `--max-model-len
|
||||
200000` (native 262,144; the harness contextWindow must quote this — a longer
|
||||
prompt is refused, never truncated), `--placement expert_tiered
|
||||
--host-pool-mib 40960` (native FP8 experts are 112.8 GiB total, 56.4 GiB
|
||||
per rank slice — the pinned pool is the price of native precision),
|
||||
`--expert-vs-cache-ratio 0.82` (the measured 99 %-hit point; 0.87 collapses
|
||||
to 0 %), `--kv-cache-dtype fp8` (**7.32 KiB/token**; a 200K sequence is
|
||||
1.43 GiB), `--num-speculative-tokens 3` (the checkpoint's own MTP head —
|
||||
3,101 `mtp.*` tensors, probe depth 3), `--max-num-batched-tokens 2048`,
|
||||
prefix cache 4 GiB pinned host + 64 GiB disk at
|
||||
`/home/dank/radiance-state`, and **`--ngram-placement ram`**: the 47.7 GiB
|
||||
PLE n-gram table (51.2 GB = 29 % of all parameters) serves from pinned
|
||||
host RAM. Froggeric v22.5 chat template vendored
|
||||
(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`).
|
||||
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
||||
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
||||
prefix cache, and the rule that a prompt over `--max-model-len` is
|
||||
refused, never truncated. What changed: the fp8-native era's NVMe expert
|
||||
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
|
||||
40 GiB pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
|
||||
on the deployed shape (512-token streamed probe). The campaign also
|
||||
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
||||
accept 0.7246); production shipped the fp8-KV + spec-2 pair at cutover —
|
||||
the KLD gate scored at the fp8-KV shape, so the gate and the boot agree.
|
||||
Disk-tier sizing law: the engine forces ~9.6 GiB of disk-tier weight
|
||||
buffers + arena onto the weights side, so `weights + kv` is capped near
|
||||
29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`.
|
||||
|
||||
**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB pools +
|
||||
47.7 GiB PLE + 4 GiB prefix host ≈ **134 GiB** charged against
|
||||
`RLIMIT_MEMLOCK`. `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned
|
||||
pages (pre-cutover cap 123 GiB — BELOW the plan); the serve script's
|
||||
TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to start until
|
||||
both knobs are raised in the stop window (sysfs write first, kernel
|
||||
cmdline + reboot only if read-only).
|
||||
**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB expert
|
||||
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
||||
the card reads) + 4 GiB prefix host ≈ **132 GiB** charged against
|
||||
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
||||
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
||||
script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to
|
||||
start until both knobs are raised in the stop window (sysfs write first,
|
||||
kernel cmdline + reboot only if read-only).
|
||||
|
||||
**Contract facts** (source-verified): the HTTP listener binds AFTER the
|
||||
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
|
||||
@@ -574,7 +570,8 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`.
|
||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`.
|
||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
|
||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
|
||||
@@ -1021,8 +1018,8 @@ Active:
|
||||
|
||||
| Service | Runbook |
|
||||
|---------|---------|
|
||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×200,192, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
||||
| frank-secondary (llama.cpp CUDA, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
||||
| frank-secondary (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md` |
|
||||
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||
@@ -1031,12 +1028,13 @@ Retired (rollback targets and history):
|
||||
|
||||
| Runtime | Host | Window | Runbook |
|
||||
|---------|------|--------|---------|
|
||||
| halogen 0.1.4 Qwen3.8-27B p1w4d-d2 + DFlash2 (`frank-large`) | Dankclaw | 2026-10-05 → 10-06 (rollback target — payload + fetch script + image digest on disk; owns the prompt-cache pin and FA-tile lessons) | `frank-large-halogen27b-dflash2.md` |
|
||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-secondary`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-secondary-llama-gemma4-q4km-200704.md` |
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
|
||||
| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` |
|
||||
| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||
| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-secondary`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
|
||||
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
|
||||
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
|
||||
|
||||
@@ -337,7 +337,16 @@ http://:8017 {
|
||||
try_files /docs.html
|
||||
file_server
|
||||
}
|
||||
@data path /services.json /status.json /app.js /style.css /deployed.js
|
||||
# /whoami echoes the outpost-injected identity headers so the page
|
||||
# can gate the admin button client-side (the /admin* 403 above stays
|
||||
# the real gate; this only decides what is shown). The outpost strips
|
||||
# client-supplied X-Authentik-* before injecting, so this is spoof-proof.
|
||||
handle /whoami {
|
||||
templates
|
||||
header Content-Type application/json
|
||||
respond "{\"user\":\"{header.X-Authentik-Username}\",\"groups\":\"{header.X-Authentik-Groups}\"}"
|
||||
}
|
||||
@data path /services.json /status.json /app.js /style.css /deployed.js /img/* /sfx/*
|
||||
handle @data {
|
||||
file_server
|
||||
}
|
||||
|
||||
@@ -10,15 +10,15 @@ model:
|
||||
provider: dankclaw-large
|
||||
base_url: http://192.168.0.143:7770/v1
|
||||
api_key: not-needed
|
||||
default: frank-large
|
||||
default: frank-dragoon
|
||||
|
||||
providers:
|
||||
dankclaw-large:
|
||||
api: http://192.168.0.143:7770/v1
|
||||
discover_models: false
|
||||
models:
|
||||
frank-large:
|
||||
context_length: 114688
|
||||
frank-dragoon:
|
||||
context_length: 262144
|
||||
dankserver-small:
|
||||
api: http://192.168.0.235:7770/v1
|
||||
discover_models: false
|
||||
@@ -29,7 +29,7 @@ providers:
|
||||
# Auxiliary routing from the live configuration. Named providers route via
|
||||
# their `api:` URL; `custom` entries route by their explicit `base_url`.
|
||||
auxiliary:
|
||||
vision: {provider: dankclaw-large, model: frank-large}
|
||||
vision: {provider: dankclaw-large, model: frank-dragoon}
|
||||
web_extract: {provider: dankserver-small, model: frank-medium}
|
||||
compression: {provider: dankserver-small, model: frank-medium}
|
||||
skills_hub: {provider: dankserver-small, model: frank-medium}
|
||||
@@ -50,8 +50,8 @@ auxiliary:
|
||||
effort: none
|
||||
memory_query_rewrite: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
|
||||
tts_audio_tags: {provider: custom, model: frank-medium, base_url: http://192.168.0.235:7770/v1}
|
||||
triage_specifier: {provider: dankclaw-large, model: frank-large}
|
||||
kanban_decomposer: {provider: dankclaw-large, model: frank-large}
|
||||
triage_specifier: {provider: dankclaw-large, model: frank-dragoon}
|
||||
kanban_decomposer: {provider: dankclaw-large, model: frank-dragoon}
|
||||
profile_describer: {provider: dankserver-small, model: frank-medium}
|
||||
curator: {provider: dankserver-small, model: frank-medium}
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
#!/usr/bin/env bash
|
||||
# apply-protected-cast.sh — RUNS ON DANKSERVER. Rewrite the recipe's
|
||||
# PROTECTED-BEGIN/END section from Swift's OWN calibration grams: run
|
||||
# derive-protected-experts.py over the calib dir, splice its cast lines into
|
||||
# the marked section of the recipe in place, keep the pre-edit recipe in the
|
||||
# campaign trash. Refuses to run twice (marker text says derived).
|
||||
set -euo pipefail
|
||||
CALIB=${1:-/home/dank/corpora/calib}
|
||||
RECIPE=${2:-/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe}
|
||||
PY=/home/dank/venvs/radcorpus/bin/python
|
||||
DERIVE=/home/dank/serverAppConfigs-swift15/scripts/derive-protected-experts.py
|
||||
TS=$(date +%Y%m%d-%H%M%S)
|
||||
|
||||
grep -q "derive-protected-experts.py rewrote" "$RECIPE" && {
|
||||
echo "recipe already carries derived cast lines; refusing to re-derive" >&2; exit 1; }
|
||||
grep -q "PROTECTED-BEGIN" "$RECIPE" || { echo "no PROTECTED markers in $RECIPE" >&2; exit 1; }
|
||||
|
||||
"$PY" "$DERIVE" "$CALIB" > "/tmp/protected-cast-$TS.txt"
|
||||
N=$(grep -c 'cast dtype=bf16' "/tmp/protected-cast-$TS.txt")
|
||||
[ "$N" -ge 2 ] || { echo "derive produced $N cast lines; refusing" >&2; exit 1; }
|
||||
echo "derived $N protected experts -> /tmp/protected-cast-$TS.txt"
|
||||
|
||||
cp -f "$RECIPE" "/home/dank/.AGENTTRASH/$TS-qwen4exp-swift15-w4dn8-i8-hc8m.recipe"
|
||||
"$PY" - "$RECIPE" "/tmp/protected-cast-$TS.txt" <<'PYEOF'
|
||||
import sys
|
||||
recipe, castfile = sys.argv[1], sys.argv[2]
|
||||
lines = open(recipe).read().splitlines()
|
||||
cast = [l for l in open(castfile).read().splitlines() if 'cast dtype=bf16' in l]
|
||||
out, insec = [], False
|
||||
for l in lines:
|
||||
if 'PROTECTED-BEGIN' in l:
|
||||
insec = True
|
||||
out.append(l)
|
||||
out.extend(cast)
|
||||
elif 'PROTECTED-END' in l:
|
||||
insec = False
|
||||
out.append(l)
|
||||
out.append('# (derive-protected-experts.py rewrote this section from Swift\'s own grams)')
|
||||
elif not insec:
|
||||
out.append(l)
|
||||
open(recipe, 'w').write('\n'.join(out) + '\n')
|
||||
print(f'spliced {len(cast)} cast lines into {recipe}')
|
||||
PYEOF
|
||||
@@ -104,7 +104,7 @@ def stream_completion(url: str, model: str, prompt: str):
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--url", default="http://127.0.0.1:7770")
|
||||
ap.add_argument("--model", default="frank-large")
|
||||
ap.add_argument("--model", default="frank-dragoon")
|
||||
args = ap.parse_args()
|
||||
|
||||
with urllib.request.urlopen(args.url.rstrip("/") + "/v1/models",
|
||||
|
||||
@@ -87,7 +87,8 @@ podman run -d --name "$SCRATCH" \
|
||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
||||
--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \
|
||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
||||
--prefix-cache-dir "/data/kvcache/$SCRATCH" \
|
||||
@@ -145,7 +146,7 @@ while n < TGT:
|
||||
n += len(seg) + 1
|
||||
prompt = " ".join(parts)
|
||||
json.dump(
|
||||
{"model": "frank-primary", "prompt": prompt,
|
||||
{"model": "frank-archon", "prompt": prompt,
|
||||
"max_tokens": 256, "temperature": 0.0, "stream": False,
|
||||
"stop": ["<|im_end|>"]},
|
||||
open(sys.argv[1], "w"),
|
||||
|
||||
@@ -45,7 +45,7 @@ import urllib.error
|
||||
import urllib.request
|
||||
|
||||
BASE = os.environ.get("AR_BASE", "http://127.0.0.1:7770")
|
||||
MODEL = os.environ.get("AR_MODEL", "frank-primary")
|
||||
MODEL = os.environ.get("AR_MODEL", "frank-archon")
|
||||
PROMPT_CEILING = int(os.environ.get("AR_PROMPT_CEILING", "100000"))
|
||||
DEPTHS = (115, 1056, 4096)
|
||||
WARM_TOKENS = 64
|
||||
|
||||
@@ -57,7 +57,7 @@ run_bench() {
|
||||
# Warmup (small request)
|
||||
curl -s -m 60 http://127.0.0.1:$PORT/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"frank-large","messages":[{"role":"user","content":"Hi"}],"max_tokens":16,"stream":false}' > /dev/null 2>&1 || true
|
||||
-d '{"model":"frank-dragoon","messages":[{"role":"user","content":"Hi"}],"max_tokens":16,"stream":false}' > /dev/null 2>&1 || true
|
||||
|
||||
# Build 8K prompt
|
||||
local prompt
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env python3
|
||||
"""bench-probe.py — RUNS ON DANKSERVER. The campaign's decode probe: one
|
||||
/completion request, a prompt assembled from calib.jsonl to ~TARGET tokens,
|
||||
temperature 0, max_tokens 512. Reports the engine's own timings: decode rate,
|
||||
prefill rate, draft accept. Run 3x; the median is the arm's number."""
|
||||
import json, sys, urllib.request
|
||||
|
||||
TARGET = 43600 # tokens
|
||||
BYTES_PER_TOK = 3.8
|
||||
URL = "http://localhost:7770/v1/completions"
|
||||
|
||||
def build_prompt():
|
||||
want = int(TARGET * BYTES_PER_TOK)
|
||||
parts = []
|
||||
n = 0
|
||||
with open("/home/dank/corpora/out/calib.jsonl") as f:
|
||||
for line in f:
|
||||
d = json.loads(line)
|
||||
p = d.get("prompt") or d.get("messages", [{}])[0].get("content", "")
|
||||
parts.append(p)
|
||||
n += len(p.encode())
|
||||
if n >= want:
|
||||
break
|
||||
return "".join(parts)
|
||||
|
||||
def probe(salt):
|
||||
prompt = f"[probe {salt}]\n" + build_prompt()
|
||||
body = json.dumps({
|
||||
"model": "frank-archon",
|
||||
"prompt": prompt, "temperature": 0.0, "max_tokens": 512,
|
||||
}).encode()
|
||||
req = urllib.request.Request(URL, data=body,
|
||||
headers={"Content-Type": "application/json"})
|
||||
with urllib.request.urlopen(req, timeout=1800) as r:
|
||||
d = json.load(r)
|
||||
t = d.get("timings", {})
|
||||
return {
|
||||
"prompt_n": t.get("prompt_n"),
|
||||
"prefill_tps": round(t.get("prompt_per_second", 0), 1),
|
||||
"pred_n": t.get("predicted_n"),
|
||||
"decode_tps": round(t.get("predicted_per_second", 0), 2),
|
||||
"draft_n": t.get("draft_n"),
|
||||
"accept": (round(t["draft_n_accepted"] / t["draft_n"], 4)
|
||||
if t.get("draft_n") else None),
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(json.dumps(probe(sys.argv[1] if len(sys.argv) > 1 else "a")))
|
||||
@@ -39,7 +39,7 @@ def send_prefill_request():
|
||||
with open(PROMPT_FILE, "r") as f:
|
||||
prompt = f.read()
|
||||
body = json.dumps({
|
||||
"model": "frank-large",
|
||||
"model": "frank-dragoon",
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"max_tokens": 1,
|
||||
"stream": False
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env python3
|
||||
"""calib-push.py — drive a calibration corpus through a radiance scratch
|
||||
server's prefill so its tap writes layer Hessians (docs/MOE-W4.md).
|
||||
|
||||
Runs ON DANKSERVER next to the scratch server (the one autoresearch- /
|
||||
serve-style scratch container booted with RADIANCE_CALIB_DIR set).
|
||||
Reads the corpus JSONL produced by make-calib-corpus.py
|
||||
({"prompt": "..."} a line), POSTs each prompt to /v1/completions with
|
||||
max_tokens=1 (prefill-only cost, one token out), at N-way concurrency
|
||||
matching the scratch server's --max-num-seqs budget.
|
||||
|
||||
Corpus rounds repeat until the engine prints the tap's completion line
|
||||
("calibration wrote layer Hessians") in `podman logs <container>` — that
|
||||
is the ground truth that RADIANCE_CALIB_TOKENS was reached. After that
|
||||
line, further prompts are pure waste, so the driver exits.
|
||||
|
||||
Mechanics notes:
|
||||
- requests carry no sampling params that matter (max_tokens=1, greedy).
|
||||
- a per-request timeout of 30 min: at ~1k tok/s prefill a 12k-token
|
||||
prompt is a few seconds; the timeout only catches a wedged engine.
|
||||
- failures print and continue: the tap pools over tokens, a few lost
|
||||
prompts do not bias the Hessians (the corpus is drawn round-robin over
|
||||
source classes, so loss is class-neutral as long as failures are rare;
|
||||
a systematic 4xx is loud in the log and must be fixed, not ignored).
|
||||
|
||||
Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
|
||||
[--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4]
|
||||
"""
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
DONE_MARK = "calibration wrote"
|
||||
|
||||
|
||||
def tap_done(container):
|
||||
"""True if the engine logged the tap's completion line."""
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["podman", "logs", "--tail", "50", container],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
except Exception:
|
||||
return False
|
||||
return DONE_MARK in (out.stdout + out.stderr)
|
||||
|
||||
|
||||
def push_one(url, prompt):
|
||||
body = json.dumps({
|
||||
"model": "frank-archon",
|
||||
"prompt": prompt,
|
||||
"max_tokens": 1,
|
||||
"temperature": 0,
|
||||
"stream": False,
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
url, data=body, headers={"Content-Type": "application/json"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=1800) as r:
|
||||
r.read()
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"FAIL {str(e)[:120]}", flush=True)
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--port", type=int, default=7770)
|
||||
ap.add_argument("--parallel", type=int, default=64)
|
||||
ap.add_argument("--container", default="arbench")
|
||||
ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl")
|
||||
ap.add_argument("--max-rounds", type=int, default=4)
|
||||
a = ap.parse_args()
|
||||
url = f"http://127.0.0.1:{a.port}/v1/completions"
|
||||
|
||||
prompts = []
|
||||
with open(a.corpus) as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
prompts.append(json.loads(line)["prompt"])
|
||||
print(f"corpus: {len(prompts)} prompts", flush=True)
|
||||
|
||||
for rnd in range(1, a.max_rounds + 1):
|
||||
if tap_done(a.container):
|
||||
break
|
||||
t0 = time.time()
|
||||
ok = 0
|
||||
with concurrent.futures.ThreadPoolExecutor(a.parallel) as ex:
|
||||
for good in ex.map(lambda p: push_one(url, p), prompts):
|
||||
ok += 1 if good else 0
|
||||
dt = time.time() - t0
|
||||
print(f"round {rnd}: {ok}/{len(prompts)} ok in {dt:.0f}s "
|
||||
f"({sum(len(p) for p in prompts)/4/dt:.0f} chars/s est)",
|
||||
flush=True)
|
||||
if tap_done(a.container):
|
||||
break
|
||||
else:
|
||||
print("max rounds exhausted, tap not done", flush=True)
|
||||
sys.exit(1)
|
||||
print("TAP_DONE: calibration wrote layer Hessians", flush=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env bash
|
||||
# convert-swift15-w4dn8.sh — RUNS ON DANKSERVER. The final recipe convert of
|
||||
# the Swift 1.5 checkpoint: gate_up experts through GPTQ against the 10M-token
|
||||
# calibration Grams, down projections block-fp8, protected experts bf16,
|
||||
# trunk int8, PLE fixed-scale fp8 (scripts/recipes/
|
||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe).
|
||||
#
|
||||
# The convert engine is 1.0.13 — the SAME pin that will serve the result, so
|
||||
# the .rad crosses no version boundary. Only the Gram files cross versions
|
||||
# (1.2.0 wrote them; the RADGRAM2 magic, gram.r%d.%s.bin naming and the
|
||||
# calib_gram/calib_resolve/calib_stem symbols are byte-identical between the
|
||||
# two libquant.so). rad-convert runs on the host backend (no GPU visible in
|
||||
# the container) — production can stay up through this script.
|
||||
#
|
||||
# Prereqs (campaign order): calibration complete (calib-push printed TAP_DONE),
|
||||
# the protected-expert cast lines rewritten into the recipe by
|
||||
# derive-protected-experts.py, and the 335 GiB bf16 container deleted —
|
||||
# the peak-disk arithmetic needs that space (checkpoint 335.3 + output ~132
|
||||
# GiB on the volume; --reuse is NOT used, see the recipe header).
|
||||
set -euo pipefail
|
||||
|
||||
PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
IMAGE="stilldeadcode/radiance@$PIN_1013"
|
||||
CKPT=/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next
|
||||
RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe
|
||||
CALIB=/home/dank/corpora/calib
|
||||
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||
LOG=/home/dank/logs/swift15_convert.log
|
||||
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
|
||||
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
|
||||
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
||||
|
||||
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
||||
# per-tensor stderr note, which is silently worse than failing here.
|
||||
[ -d "$CKPT" ] || { echo "no checkpoint at $CKPT" >&2; exit 1; }
|
||||
[ -f "$RECIPE" ] || { echo "no recipe at $RECIPE" >&2; exit 1; }
|
||||
GRAMS=$(find "$CALIB" -maxdepth 1 -name 'gram.r*.bin' 2>/dev/null | wc -l)
|
||||
[ "$GRAMS" -ge 48 ] || { echo "calib dir $CALIB holds $GRAMS gram files (<48)" >&2; exit 1; }
|
||||
if [ -e "$OUT" ]; then echo "$OUT already exists; move it to trash first" >&2; exit 1; fi
|
||||
if [ -e /home/dank/models/rad/qwen4exp-swift15-bf16.rad ]; then
|
||||
echo "the bf16 container is still on disk; delete it first (peak disk)" >&2; exit 1
|
||||
fi
|
||||
|
||||
# Gate 2: disk. Output ~132 GiB plus convert staging, 20 GiB of slack.
|
||||
FREE_MIB=$(df --output=avail -m /home/dank/models | tail -1 | tr -d ' ')
|
||||
[ "$FREE_MIB" -ge $((MIN_MIB + 20480)) ] || {
|
||||
echo "only ${FREE_MIB} MiB free on /home/dank/models" >&2; exit 1; }
|
||||
|
||||
# The recipe names $CALIB; rad-convert expands it from the environment
|
||||
# (MOE-W4.md:275). The recipe's cast section must name THIS checkpoint's
|
||||
# protected experts — derive-protected-experts.py rewrote it.
|
||||
echo "convert: $CKPT -> $OUT (gptq calib=$CALIB, $(date -Is))" | tee "$LOG"
|
||||
podman run --rm --name swift15-convert \
|
||||
-v /home/dank/models:/models \
|
||||
-v /home/dank/serverAppConfigs-swift15:/campaign:ro \
|
||||
-v "$CALIB:/data/calib:ro" \
|
||||
-e CALIB=/data/calib \
|
||||
--entrypoint /opt/radiance/bin/rad-convert \
|
||||
"$IMAGE" "/models/${CKPT#/home/dank/models/}" \
|
||||
--recipe /campaign/"${RECIPE#/home/dank/serverAppConfigs-swift15/}" \
|
||||
-o /models/rad/"$(basename "$OUT")" 2>&1 | tee -a "$LOG"
|
||||
rc=${PIPESTATUS[0]}
|
||||
[ "$rc" -eq 0 ] || { echo "rad-convert rc=$rc; log: $LOG" >&2; exit "$rc"; }
|
||||
|
||||
# Gate 3: the artifact. rad-info's recipe echo is the receipt that GPTQ ran.
|
||||
SIZE_MIB=$(( $(stat -c %s "$OUT") / 1048576 ))
|
||||
[ "$SIZE_MIB" -ge "$MIN_MIB" ] || {
|
||||
echo "$OUT is ${SIZE_MIB} MiB < floor ${MIN_MIB}" >&2; exit 1; }
|
||||
podman run --rm --entrypoint /opt/radiance/bin/rad-info \
|
||||
-v /home/dank/models:/models:ro "$IMAGE" /models/rad/"$(basename "$OUT")" \
|
||||
2>&1 | tee -a "$LOG" | grep -c 'gptq' >/dev/null \
|
||||
|| echo "WARNING: rad-info shows no gptq rule in the recorded recipe" >&2
|
||||
echo "CONVERT_DONE ${SIZE_MIB} MiB -> $OUT"
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
"""deep-probe.py — one chat completion at ~43.6k prompt tokens, temp 0, 512
|
||||
generated, streamed; decode = completion_tokens / (total - ttft)."""
|
||||
import json, time, urllib.request
|
||||
|
||||
TARGET_BYTES = int(43600 * 3.8)
|
||||
|
||||
def build():
|
||||
parts, n = [], 0
|
||||
with open("/home/dank/corpora/out/calib.jsonl") as f:
|
||||
for line in f:
|
||||
p = json.loads(line)["prompt"]
|
||||
parts.append(p)
|
||||
n += len(p.encode())
|
||||
if n >= TARGET_BYTES:
|
||||
break
|
||||
return "".join(parts)
|
||||
|
||||
def main(salt):
|
||||
body = json.dumps({
|
||||
"model": "frank-archon",
|
||||
"messages": [{"role": "user", "content": f"[probe {salt}] Continue and summarize this material in English, 400 words.\n\n" + build()}],
|
||||
"temperature": 0.0, "max_tokens": 512,
|
||||
"stream": True, "stream_options": {"include_usage": True},
|
||||
}).encode()
|
||||
req = urllib.request.Request("http://localhost:7770/v1/chat/completions",
|
||||
data=body, headers={"Content-Type": "application/json"})
|
||||
t0 = time.time(); ttft = None; ctok = 0; usage = {}
|
||||
with urllib.request.urlopen(req, timeout=2400) as r:
|
||||
for raw in r:
|
||||
line = raw.decode().strip()
|
||||
if not line.startswith("data:"):
|
||||
continue
|
||||
d = line[5:].strip()
|
||||
if d == "[DONE]":
|
||||
break
|
||||
j = json.loads(d)
|
||||
if j.get("usage"):
|
||||
usage = j["usage"]
|
||||
for ch in j.get("choices", []):
|
||||
delta = (ch.get("delta") or {})
|
||||
piece = delta.get("content") or delta.get("reasoning_content") or ""
|
||||
if piece and ttft is None:
|
||||
ttft = time.time() - t0
|
||||
ctok += 1
|
||||
total = time.time() - t0
|
||||
gen = usage.get("completion_tokens") or ctok
|
||||
dec = gen / (total - (ttft or 0)) if total > (ttft or 0) else 0
|
||||
print(json.dumps({"salt": salt, "prompt_tokens": usage.get("prompt_tokens"),
|
||||
"completion_tokens": gen, "ttft_s": round(ttft or 0, 2),
|
||||
"total_s": round(total, 1), "decode_tps": round(dec, 2)}))
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
main(sys.argv[1] if len(sys.argv) > 1 else "x")
|
||||
@@ -322,6 +322,12 @@ deploy_homepage() {
|
||||
if [[ -d shared/homepage/fonts ]]; then
|
||||
run_privileged cp -r shared/homepage/fonts /srv/homepage/
|
||||
fi
|
||||
if [[ -d shared/homepage/img ]]; then
|
||||
run_privileged cp -r shared/homepage/img /srv/homepage/
|
||||
fi
|
||||
if [[ -d shared/homepage/sfx ]]; then
|
||||
run_privileged cp -r shared/homepage/sfx /srv/homepage/
|
||||
fi
|
||||
local deployed_at
|
||||
deployed_at="$(git log -1 --format=%cI)"
|
||||
printf 'window.DEPLOYED_AT = "%s";\n' "$deployed_at" > /tmp/deployed.js
|
||||
@@ -1225,13 +1231,14 @@ deploy_llm_large() {
|
||||
#
|
||||
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
|
||||
# not shared: frank-large's engine runs on the gfx1151 iGPU.
|
||||
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
|
||||
LLM_SECONDARY_MODEL_FILE="${LLM_SECONDARY_MODEL_FILE:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
||||
LLM_SECONDARY_RUNTIME="${LLM_SECONDARY_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
||||
# The projector IS the vision capability here: the GGUF carries no vision
|
||||
# tensors, and the publisher ships mmproj-BF16.gguf in the same repo, so a
|
||||
# deploy that starts without this file silently ships a text-only
|
||||
# endpoint. Checked unconditionally, like frank-large's.
|
||||
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
|
||||
# The projector IS the vision capability here: the model is natively
|
||||
# multimodal and the Q6_K file carries no vision tensors — the publisher ships
|
||||
# its own BF16 projector in the same repo, so a deploy that starts without
|
||||
# this file silently ships a text-only endpoint. Checked unconditionally, like
|
||||
# frank-large's.
|
||||
LLM_SECONDARY_MMPROJ="${LLM_SECONDARY_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
||||
|
||||
deploy_llm_secondary() {
|
||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env python3
|
||||
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
||||
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
||||
|
||||
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
||||
an expert whose share of its layer's down-input energy is at least 3% AND
|
||||
at least three times its share of the routing is kept bf16; the count must
|
||||
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||
as two equal tables by parity) — an odd half gains its next most energetic
|
||||
non-qualifying expert.
|
||||
|
||||
Data: the calibration dir's per-expert down Grams,
|
||||
gram.r<rank>.blk.L.ffn_down_exps.<e>.bin — 32-byte RADGRAM2 header
|
||||
(magic, k, rank, rows, domain) then k*k f32, named with the checkpoint's
|
||||
GLOBAL expert numbering (arch/common/rad_block_moe_fp8.h: ge =
|
||||
expert_base + e). rows (header) is the expert's routing count; the trace
|
||||
of the Gram is its down-input energy. Rank halves: 0-255, 256-511
|
||||
(contiguous expert_base split at TP=2).
|
||||
|
||||
Output: the cast-rule lines (recipe format) on stdout, a report on stderr.
|
||||
|
||||
Usage: derive-protected-experts.py <calib dir> [--min-share 0.03]
|
||||
[--min-route-mult 3.0]
|
||||
"""
|
||||
import argparse
|
||||
import glob
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
|
||||
NAME_RE = re.compile(r"^gram\.r(\d)\.blk\.(\d+)\.ffn_down_exps\.(\d+)\.bin$")
|
||||
|
||||
|
||||
def read_gram(path):
|
||||
"""(k, rank, rows, trace) from a RADGRAM2 file (full matrix on disk)."""
|
||||
with open(path, "rb") as f:
|
||||
hdr = f.read(32)
|
||||
magic, k, rank, rows, domain = struct.unpack("<8sIIQI", hdr[:28])
|
||||
if magic != b"RADGRAM2":
|
||||
raise SystemExit(f"not a RADGRAM2 file: {path}")
|
||||
n = k * k
|
||||
a = np.frombuffer(f.read(n * 4), dtype=np.float32)
|
||||
if a.size != n:
|
||||
raise SystemExit(f"short gram file: {path}")
|
||||
tr = float(a.reshape(k, k).diagonal().sum())
|
||||
return k, rank, int(rows), tr
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("calib_dir")
|
||||
ap.add_argument("--min-share", type=float, default=0.03)
|
||||
ap.add_argument("--min-route-mult", type=float, default=3.0)
|
||||
a = ap.parse_args()
|
||||
|
||||
# layer -> global expert -> (rank, rows, energy)
|
||||
data = {}
|
||||
for path in sorted(glob.glob(os.path.join(
|
||||
a.calib_dir, "gram.r*.ffn_down_exps.*.bin"))):
|
||||
m = NAME_RE.match(os.path.basename(path))
|
||||
if not m:
|
||||
continue
|
||||
rank, layer, expert = int(m.group(1)), int(m.group(2)), int(m.group(3))
|
||||
k, _, rows, tr = read_gram(path)
|
||||
data.setdefault(layer, {})[expert] = (rank, rows, tr)
|
||||
|
||||
if not data:
|
||||
raise SystemExit("no per-expert down grams found")
|
||||
|
||||
lines, report = [], []
|
||||
total = 0
|
||||
for layer in sorted(data):
|
||||
experts = data[layer]
|
||||
tot_e = sum(v[2] for v in experts.values())
|
||||
tot_r = sum(v[1] for v in experts.values())
|
||||
if tot_e <= 0 or tot_r <= 0:
|
||||
report.append(f"blk.{layer}: empty recording, skipped")
|
||||
continue
|
||||
qual, pool = [], []
|
||||
for e, (rank, rows, tr) in experts.items():
|
||||
es, rs = tr / tot_e, rows / max(tot_r, 1)
|
||||
rec = (e, rank, es, rs)
|
||||
if es >= a.min_share and es >= a.min_route_mult * rs:
|
||||
qual.append(rec)
|
||||
elif es > 0:
|
||||
pool.append(rec)
|
||||
chosen = {r[0]: (r[2], r[3]) for r in qual}
|
||||
for half in (0, 1): # rank halves by the contiguous expert_base split
|
||||
n_half = sum(1 for e in chosen if e // 256 == half)
|
||||
if n_half % 2:
|
||||
pool.sort(key=lambda r: -r[2])
|
||||
for cand in pool:
|
||||
if cand[1] == half and cand[0] not in chosen:
|
||||
chosen[cand[0]] = (cand[2], cand[3])
|
||||
report.append(
|
||||
f"blk.{layer}: half {half} odd -> added e{cand[0]} "
|
||||
f"(energy {cand[2]*100:.2f}%)")
|
||||
break
|
||||
for e in sorted(chosen):
|
||||
es, rs = chosen[e]
|
||||
lines.append(f"blk.{layer}.ffn_*_exps.{e}.weight"
|
||||
f" cast dtype=bf16")
|
||||
report.append(f"blk.{layer} e{e:3d} (rank {e // 256}): "
|
||||
f"energy {es*100:5.2f}%, routing {rs*100:5.3f}%, "
|
||||
f"ratio {es/max(rs,1e-12):6.1f}")
|
||||
total += len(chosen)
|
||||
|
||||
print("\n".join(lines))
|
||||
print("\n".join(report), file=sys.stderr)
|
||||
print(f"total protected: {total}", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,113 @@
|
||||
#!/usr/bin/env bash
|
||||
# download-frank-primary-swift15.sh — payload fetch/verify for the planned
|
||||
# frank-primary swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next
|
||||
# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors
|
||||
# shards + weight-map index + tokenizer + vision preprocessing configs + chat
|
||||
# template + licenses). Key set verified identical to the official
|
||||
# Qwen/Qwen3.8-Flash-Next base (0 added / 0 missing), so the qwen4exp recipes
|
||||
# apply verbatim; the checkpoint ships BF16 with NO quantization_config.
|
||||
# Repo is UNGATED — no token needed (never put one on the command line).
|
||||
#
|
||||
# bash scripts/download-frank-primary-swift15.sh --fetch resume/complete
|
||||
# bash scripts/download-frank-primary-swift15.sh verify only
|
||||
#
|
||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
||||
# present with the listed size, the shard count matches the weight-map
|
||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
||||
# only the public API.
|
||||
set -euo pipefail
|
||||
|
||||
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
||||
DEST="/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
||||
EXPECTED_SHARDS=131
|
||||
|
||||
mode="verify"
|
||||
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
if [[ "$mode" == "fetch" ]]; then
|
||||
# HFGET_JOBS=4 is hfget's default (a few streams saturate the link).
|
||||
bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$DEST" main
|
||||
fi
|
||||
|
||||
echo "verifying local payload: $DEST"
|
||||
fail=0
|
||||
|
||||
INDEX="$DEST/model.safetensors.index.json"
|
||||
if [[ ! -f "$INDEX" ]]; then
|
||||
echo " BAD weight-map index missing: $INDEX" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 - "$REPO" "$DEST" <<'PY' || fail=1
|
||||
import hashlib, json, os, sys, urllib.request
|
||||
|
||||
repo, dest = sys.argv[1], sys.argv[2]
|
||||
|
||||
def listing():
|
||||
"""The tree API, following the Link cursor exactly like hfget.sh."""
|
||||
out, api, pages = [], (
|
||||
f"https://huggingface.co/api/models/{repo}/tree/main"
|
||||
"?recursive=true&expand=true"), 0
|
||||
while api:
|
||||
with urllib.request.urlopen(api) as r:
|
||||
out += json.load(r)
|
||||
api = None
|
||||
link = r.headers.get("Link", "")
|
||||
for part in link.split(","):
|
||||
if 'rel="next"' in part:
|
||||
api = part[part.find("<") + 1:part.find(">")]
|
||||
pages += 1
|
||||
if pages > 200:
|
||||
raise SystemExit("listing did not terminate")
|
||||
return out
|
||||
|
||||
files = [e for e in listing() if e.get("type") == "file"]
|
||||
sizes = {e["path"]: e.get("size", 0) for e in files}
|
||||
lfs = {e["path"]: e["lfs"]["oid"] for e in files if e.get("lfs")}
|
||||
|
||||
bad = 0
|
||||
for path, want in sorted(sizes.items()):
|
||||
p = os.path.join(dest, path)
|
||||
if not os.path.isfile(p):
|
||||
print(f" BAD {path}: missing")
|
||||
bad += 1
|
||||
elif os.path.getsize(p) != want:
|
||||
print(f" BAD {path}: {os.path.getsize(p)} bytes, listing says {want}")
|
||||
bad += 1
|
||||
print(f" {'OK' if bad == 0 else 'FAIL'}: {len(sizes) - bad}/{len(sizes)} files present at listed size")
|
||||
if bad:
|
||||
raise SystemExit(1)
|
||||
|
||||
# Shard set against the weight-map index.
|
||||
wm = json.load(open(os.path.join(dest, "model.safetensors.index.json")))["weight_map"]
|
||||
shards = sorted(set(wm.values()))
|
||||
print(f" index maps {len(wm)} tensors across {len(shards)} shards")
|
||||
if len(shards) != 131:
|
||||
print(f" BAD shard count {len(shards)}, want 131")
|
||||
raise SystemExit(1)
|
||||
|
||||
# Content: every LFS-tracked file hashed against the Hub oid. The safetensors
|
||||
# shards dominate the bytes; tokenizer/config files ride the same check when
|
||||
# the Hub tracks them in LFS (they are plain blobs here, already size-checked).
|
||||
todo = [p for p in shards if p in lfs]
|
||||
for i, path in enumerate(todo):
|
||||
h = hashlib.sha256()
|
||||
with open(os.path.join(dest, path), "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1 << 24), b""):
|
||||
h.update(chunk)
|
||||
if h.hexdigest() != lfs[path]:
|
||||
print(f" BAD {path}: sha256 mismatch (want {lfs[path]})")
|
||||
bad += 1
|
||||
if (i + 1) % 16 == 0:
|
||||
print(f" ...hashed {i + 1}/{len(todo)}")
|
||||
print(f" {'OK' if bad == 0 else 'FAIL'}: {len(todo) - bad}/{len(todo)} shards sha256-match the Hub")
|
||||
if bad:
|
||||
raise SystemExit(1)
|
||||
PY
|
||||
|
||||
[[ "$fail" -eq 0 ]] || exit 1
|
||||
echo "payload verified: 131 shards + index + tokenizer + configs, sizes and sha256 checked"
|
||||
@@ -1,27 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
|
||||
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
|
||||
# **with the MTP head in the file** (the -mtp variant), plus the
|
||||
# PUBLISHER's OWN vision projector (mmproj-BF16.gguf, same repo). Public
|
||||
# repo — no auth header, and never a token on the command line (it is
|
||||
# world-readable in /proc/<pid>/cmdline).
|
||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
||||
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
||||
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
||||
# header, and never a token on the command line (it is world-readable in
|
||||
# /proc/<pid>/cmdline).
|
||||
#
|
||||
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
|
||||
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
|
||||
# success, then byte count and sha256. This repo is on Xet storage, so
|
||||
# the Hub tree/blob APIs expose no LFS oid: the sha256 values below were
|
||||
# computed locally after the first verified fetch (byte counts match the
|
||||
# Hub's declared sizes). Idempotent: complete files are skipped. The
|
||||
# projector is stored under a self-documenting dest name; the URL name
|
||||
# and sha stay pinned in the table below.
|
||||
# success, then byte count and sha256. The Hub exposes no LFS oid for this
|
||||
# repo (Xet storage), so the sha256 values below were computed locally after
|
||||
# the first verified fetch — byte counts match the Hub's declared sizes.
|
||||
# A sha of "-" means "first fetch: size-verified, sha printed for pinning".
|
||||
# Idempotent: complete files are skipped.
|
||||
#
|
||||
# 2026-10-06 (Frank, evening): the payload moved to the UNCELERSED
|
||||
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (1.56 GiB lighter than the
|
||||
# IQ3_S-mtp it replaced; the lighter quant funds the MTP draft ctx plus
|
||||
# the second 131,072 slot). Rollback payload stays on disk:
|
||||
# /home/dank/models/Swift15-IQ3S/ (the IQ3_S-mtp that was live until this
|
||||
# swap); earlier payloads Ornith-Q8/ and MiMo-Q6K/ as well. Their
|
||||
# re-fetch recipes are in git history (predecessors of this script).
|
||||
# 2026-10-07 (Frank): the payload moved to Qwen3.5-9B-Uncensored Q6_K at
|
||||
# 2 x 200,704 (qwen35 hybrid: 32 layers, 8 full-attention + 24 Gated
|
||||
# DeltaNet, no MTP tensors in this file). Rollback payloads stay on disk:
|
||||
# /home/dank/models/Gemma4-26B-A4B/ (Q4_K_M, the 2026-10-07 era being
|
||||
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
||||
@@ -35,15 +32,14 @@ for arg in "$@"; do
|
||||
*) DEST="$arg" ;;
|
||||
esac
|
||||
done
|
||||
DEST="${DEST:-/home/dank/models/Swift15-UncIQ3XXS}"
|
||||
DEST="${DEST:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K}"
|
||||
|
||||
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
|
||||
FILES=(
|
||||
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|RCO/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|10442827392|e13d3e7400b6c7f980ebd30c0edd6be43eb1865b50be8562a9db042706c7d803"
|
||||
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|mmproj-BF16.gguf|mmproj-BF16.gguf|931145888|19acd7fcf4ee504328566eddf0f8e603ef853a205b4830209d2a4908213abdea"
|
||||
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|7359259008|c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986"
|
||||
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|921704448|05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934"
|
||||
)
|
||||
|
||||
mkdir -p "$DEST"
|
||||
status=0
|
||||
for entry in "${FILES[@]}"; do
|
||||
IFS='|' read -r repo hubname destname size sha <<<"$entry"
|
||||
@@ -51,7 +47,9 @@ for entry in "${FILES[@]}"; do
|
||||
f="$DEST/$destname"
|
||||
have=0
|
||||
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
|
||||
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
if [[ "$sha" == "-" ]]; then
|
||||
have=1
|
||||
elif sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
have=1
|
||||
fi
|
||||
fi
|
||||
@@ -73,6 +71,10 @@ for entry in "${FILES[@]}"; do
|
||||
status=1
|
||||
continue
|
||||
fi
|
||||
if [[ "$sha" == "-" ]]; then
|
||||
echo "VERIFIED $destname (size only) sha256=$(sha256sum "$f" | cut -d' ' -f1)"
|
||||
continue
|
||||
fi
|
||||
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
echo "SHA FAIL $destname"
|
||||
status=1
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env bash
|
||||
# fleet-probes-swift15.sh — RUNS ON DANKSERVER. The cutover contract probes.
|
||||
set -uo pipefail
|
||||
B=http://localhost:7770/v1
|
||||
|
||||
echo "== vision (solid red 512px, expect 'Red')"
|
||||
python3 - <<'PY'
|
||||
import base64, json, urllib.request, struct, zlib
|
||||
def png(color):
|
||||
w = h = 512
|
||||
raw = b"".join(b"\x00" + bytes(color) * w for _ in range(h))
|
||||
def chunk(t, d):
|
||||
c = t + d
|
||||
return struct.pack(">I", len(d)) + c + struct.pack(">I", zlib.crc32(c))
|
||||
return (b"\x89PNG\r\n\x1a\n"
|
||||
+ chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0))
|
||||
+ chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b""))
|
||||
b64 = base64.b64encode(png((255, 0, 0))).decode()
|
||||
body = json.dumps({"model": "frank-archon", "max_tokens": 256, "temperature": 0,
|
||||
"messages": [{"role": "user", "content": [
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64," + b64}},
|
||||
{"type": "text", "text": "What color is this image? Answer with one word."}]}]}).encode()
|
||||
r = urllib.request.Request("http://localhost:7770/v1" + "/chat/completions",
|
||||
data=body, headers={"Content-Type": "application/json"})
|
||||
d = json.load(urllib.request.urlopen(r, timeout=300))
|
||||
m = d["choices"][0]["message"]
|
||||
ans = (m.get("content") or m.get("reasoning_content") or "")
|
||||
print("VISION ANSWER:", ans[:80].replace("\n", " "))
|
||||
PY
|
||||
|
||||
echo "== tools (expect tool_calls)"
|
||||
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' -d '{
|
||||
"model":"frank-archon","temperature":0,"max_tokens":128,
|
||||
"messages":[{"role":"user","content":"What is 23 times 47? Use the calculator tool."}],
|
||||
"tools":[{"type":"function","function":{"name":"calculator","description":"evaluate arithmetic",
|
||||
"parameters":{"type":"object","properties":{"expr":{"type":"string"}},"required":["expr"]}}}]}' \
|
||||
| python3 -c 'import json,sys; d=json.load(sys.stdin); m=d["choices"][0]["message"]; print("TOOL_CALLS:", m.get("tool_calls"))'
|
||||
|
||||
echo "== reasoning_effort fold (xhigh/medium/low/high all 200 on froggeric v22.5)"
|
||||
for lvl in xhigh medium low high; do
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" -m 120 $B/chat/completions \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d "{\"model\":\"frank-archon\",\"max_tokens\":8,\"reasoning_effort\":\"$lvl\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}")
|
||||
echo " reasoning_effort=$lvl -> HTTP $code"
|
||||
done
|
||||
|
||||
echo "== reasoning_content channel"
|
||||
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \
|
||||
-d '{"model":"frank-archon","max_tokens":64,"messages":[{"role":"user","content":"Is 97 prime? Think briefly."}]}' \
|
||||
| python3 -c 'import json,sys; m=json.load(sys.stdin)["choices"][0]["message"]; print("has reasoning_content:", bool(m.get("reasoning_content")), "| content:", (m.get("content") or "")[:60].replace(chr(10)," "))'
|
||||
|
||||
echo "== response_format json_object"
|
||||
curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \
|
||||
-d '{"model":"frank-archon","max_tokens":64,"response_format":{"type":"json_object"},"messages":[{"role":"user","content":"Return JSON {\"ok\": true}"}]}' \
|
||||
| python3 -c 'import json,sys; d=json.load(sys.stdin); print("JSON MODE:", (d["choices"][0]["message"].get("content") or "")[:60])'
|
||||
echo "PROBES_DONE"
|
||||
@@ -22,15 +22,25 @@
|
||||
# tool calls verified, and the native 262,144 context.
|
||||
#
|
||||
# SHAPE — the knobs that are NOT free, with the arithmetic:
|
||||
# HALOGEN_CTX=200192 is the PER-SLOT window and the per-conversation hard
|
||||
# HALOGEN_CTX=262144 is the PER-SLOT window and the per-conversation hard
|
||||
# ceiling — a prompt over it is a 400 naming the limit, never a silent
|
||||
# truncation. Harness compaction fires at 70% = 140,134, and 140,134 +
|
||||
# 57,344 (harness maxTokens) = 197,478 <= 200,192 keeps prompt + budget
|
||||
# inside the slot. models.yml contextWindow must quote 200192 or the
|
||||
# harness discovers the wall as a 400.
|
||||
# HALOGEN_KV_POOL_POSITIONS=400384 = KV_SLOTS(2) x CTX — the explicit
|
||||
# truncation. RAISED from 200192 on 2026-10-07: the harness clamps
|
||||
# max_tokens to contextWindow − est − 4096, its chars/4 est UNDER-counts
|
||||
# the rendered prompt by ~3 % on code/agent traffic, and compaction only
|
||||
# fires at 85 % — so with window == engine ceiling, every request over
|
||||
# ~146k est (65-85 % of the window) was a guaranteed 400. Measured
|
||||
# 2026-10-07: est 152,938, server 157,388, request 43,158 vs room 42,804
|
||||
# — over by 354, margin 4,096 < the 4,450 under-count.
|
||||
# Harness contract (models.yml): contextWindow 217088, maxTokens 45056.
|
||||
# Compaction fires at 85 % = 184,524 est ≈ 190k measured (Frank's 180k
|
||||
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
||||
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
||||
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
||||
# 10 % under-count). Compaction rides on frank-secondary: input ≈
|
||||
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
||||
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
||||
# form of the engine's default, so the seam test can pin the product.
|
||||
# KV costs ~28 KiB/position (FLAGS.md) => 400,384 x 28 KiB = 10.7 GiB.
|
||||
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||
@@ -39,11 +49,11 @@
|
||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
||||
# card's own quickstart arithmetic does not bill it):
|
||||
# weights ~62 GiB resident + KV 10.7 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
||||
# = ~82.7 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ)
|
||||
# ~10-15. ~18 GiB margin. The boot ledger is the referee — read it
|
||||
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
||||
# after deploy and re-size if the engine prints a clamp.
|
||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||
# turns the MTP head off for whole requests after low acceptance and
|
||||
@@ -57,8 +67,8 @@
|
||||
# an absent temperature is no longer greedy. models.yml extraBody
|
||||
# carries the same values — consistent, not load-bearing.
|
||||
# * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
|
||||
# max_tokens; the harness always sends 57344. Requests over
|
||||
# HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
|
||||
# max_tokens; the harness sends at most its maxTokens 45056. Requests
|
||||
# over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
|
||||
# queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
|
||||
# worst case needs > 6550 s; 7200 honors it. Do not move one alone.
|
||||
# * max_tokens INCLUDES thinking; a reasoning reply over a small budget
|
||||
@@ -120,9 +130,12 @@ FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config
|
||||
|
||||
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
||||
# stays at the image default, which is what was qualified) ------------------
|
||||
HALOGEN_MODEL_ID=frank-large
|
||||
HALOGEN_CTX=200192
|
||||
HALOGEN_KV_POOL_POSITIONS=400384
|
||||
# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
|
||||
# scripts and this file keep the frank-large name. OpenWebUI api_configs
|
||||
# model_ids for the :7770 backend MUST name this value.
|
||||
HALOGEN_MODEL_ID=frank-dragoon
|
||||
HALOGEN_CTX=262144
|
||||
HALOGEN_KV_POOL_POSITIONS=524288
|
||||
HALOGEN_KV_SLOTS=2
|
||||
HALOGEN_HOST_RESERVE_GIB=20
|
||||
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
||||
|
||||
@@ -50,14 +50,30 @@
|
||||
# publisher's throughput peak for this model. Stated explicitly with
|
||||
# --num-speculative-tokens 3.
|
||||
# - The fused TP all-reduce is declared over max-num-seqs x
|
||||
# (1 + --num-speculative-tokens) rows; 2 x 4 = 8 rows fits either wire.
|
||||
# (1 + --num-speculative-tokens) rows; 3 x 3 = 9 rows fits either wire.
|
||||
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
||||
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
||||
# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
|
||||
# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
|
||||
# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
|
||||
# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
|
||||
# Acceptance on the greedy harness is deterministic (the control
|
||||
# reproduced the recorded 0.755 exactly), so the drop is a real numerics
|
||||
# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
|
||||
# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
|
||||
# The vram-pair translation itself is exact (1.2.0 boot ledger took
|
||||
# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
|
||||
# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
|
||||
# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
|
||||
# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
|
||||
# file already uses the pair, which BOTH pins accept (rollback symmetry).
|
||||
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
# The .rad container FILE (mounted under /models) and the state dir for the
|
||||
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
||||
# .rad files; the raw HF download and its staging dir live beside it and
|
||||
# are never mounted.
|
||||
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad
|
||||
FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad
|
||||
FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||
FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
|
||||
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
||||
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
||||
@@ -70,7 +86,11 @@ FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state
|
||||
# the GPUs or the port.
|
||||
FRANK_PRIMARY_CONTAINER=frank-primary
|
||||
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
|
||||
FRANK_PRIMARY_SERVED_NAME=frank-primary
|
||||
# Served model id — Protoss name (wire rename 2026-10-09). The container,
|
||||
# unit, scripts and this file keep the frank-primary name above; only the
|
||||
# alias on the wire moved. Radiance validates the requested model name, so
|
||||
# every client (harness, bench scripts, probes) MUST send this value.
|
||||
FRANK_PRIMARY_SERVED_NAME=frank-archon
|
||||
FRANK_PRIMARY_PORT=7770
|
||||
# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
|
||||
# repo's root chat_template.jinja, vendored here; the checkpoint's own
|
||||
@@ -106,9 +126,15 @@ FRANK_PRIMARY_PLACEMENT=expert_tiered
|
||||
FRANK_PRIMARY_HOST_POOL_MIB=40960
|
||||
# VRAM left unclaimed per card (publisher default 96 MiB).
|
||||
FRANK_PRIMARY_GPU_HEADROOM_MIB=96
|
||||
# Weights share of claimed VRAM (publisher production value; higher = more
|
||||
# prefix-cache hit rate, less KV headroom).
|
||||
FRANK_PRIMARY_EXPERT_VS_CACHE=0.82
|
||||
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
||||
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
|
||||
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
|
||||
# vram_weights 25.46 GiB, vram_kv 3.46 GiB per card -> 26071 / 3543.
|
||||
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
|
||||
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
|
||||
# boot ledger arbitrates; re-derive after any payload change.
|
||||
FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071
|
||||
FRANK_PRIMARY_VRAM_KV_MIB=3543
|
||||
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
|
||||
# 2 KV heads, 256 head_dim, K+V).
|
||||
FRANK_PRIMARY_KV_DTYPE=fp8
|
||||
@@ -139,4 +165,4 @@ FRANK_PRIMARY_EXTRA_ARGS=
|
||||
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
||||
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
||||
# tail) - floor sits just under the payload.
|
||||
FRANK_PRIMARY_RAD_MIN_MIB=177000
|
||||
FRANK_PRIMARY_RAD_MIN_MIB=110000
|
||||
@@ -1,98 +1,62 @@
|
||||
# frank-secondary — llama.cpp (master, CUDA sm_86) serving
|
||||
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
|
||||
# **with the MTP head in the file** on the RTX 3090, :7771.
|
||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090,
|
||||
# :7771.
|
||||
#
|
||||
# 2026-10-06 (Frank, evening): the payload moved to ajgazin's UNCELERSED
|
||||
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (10,442,827,392 B) — 1.56 GiB
|
||||
# lighter than the IQ3_S-mtp it replaces — with MTP BACK ON (draft-mtp,
|
||||
# depth 3), TWO 131,072 slots, and ubatch 512. The uncensored finetune
|
||||
# changes behavior by design (publisher: 23/100 refusals vs Swift 1.5's
|
||||
# 98/100, KL 0.0884 from Swift 1.5) — Frank's order. The RCO tier is the
|
||||
# same per-tensor ALLOCATION recipe as the IQ3_S file (histogram: iq3_s
|
||||
# 97 / bf16 96 / iq3_xxs 83 / iq2_s 72 / iq4_xs 33 / iq2_xs 32 / iq2_xxs
|
||||
# 30 / q2_k 28 / q4_k 19 / q6_k 8 / iq1_s 4 / iq1_m 4 — all stock types,
|
||||
# stock llama.cpp loads it); publisher KLD: Mean 0.002631, 90 % 0.0421
|
||||
# (kld_results.csv). RCO files are "more experimental" per the README —
|
||||
# the quality floor is the IQ3_S rung, not a promise.
|
||||
# 2026-10-07 (Frank): swap from Gemma4-26B-A4B QAT Q4_K_M (one 200,704 slot)
|
||||
# to Qwen3.5-9B Q6_K at TWO 200,704-token conversations, EXACT (f16) KV,
|
||||
# MTP disabled (SPEC_TYPE=none), log verbosity 4.
|
||||
#
|
||||
# The architecture is unchanged (same model family): qwen35 hybrid —
|
||||
# block_count 65 (64 real layers + the blk.64 MTP head),
|
||||
# full_attention_interval 4 => SIXTEEN full-attention layers, 24 Q heads
|
||||
# / 4 KV heads, k/v length 256, native window 262144, dense,
|
||||
# nextn_predict_layers=1. The MTP head ships in every main GGUF of this
|
||||
# repo (README); --spec-type draft-mtp loads it against the main model's
|
||||
# own weights.
|
||||
# Architecture — read from the GGUF's own header (HTTP range read, not the
|
||||
# model card): arch qwen35, 32 layers, dense 9B, full_attention_interval 4
|
||||
# ⇒ 8 full-attention layers (3,7,11,...,31) + 24 Gated DeltaNet linear layers,
|
||||
# 16 attention heads / 4 KV heads × 256 dim, ssm conv_kernel 4 / state_size
|
||||
# 128 / group_count 16 / time_step_rank 32 / inner_size 4096, native 262,144,
|
||||
# vocab 248,320, file_type 18 (Q6_K). Tensor-type histogram: Q6_K×250 +
|
||||
# F32×177 — standard types only, so the stock master build loads it (no
|
||||
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
||||
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
||||
#
|
||||
# THE KV CONTRACT STAYS q8_0 K + q8_0 V (Frank, 2026-10-05): 34 KiB/token
|
||||
# (16 attn layers x 4 heads x 256 x 1.0625 B, K and V).
|
||||
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
|
||||
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
|
||||
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
|
||||
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
|
||||
# where the 26B MoE fit only one.
|
||||
#
|
||||
# Why TWO slots + MTP fits now (it did not under IQ3_S):
|
||||
# card 24,576 MiB. Weights 9,963 MiB (10,442,827,392 B; IQ3_S was
|
||||
# 11,561). Projector ~1,290 MiB (the PUBLISHER's OWN mmproj-BF16.gguf,
|
||||
# 931,145,888 B, at the measured ~1.45x card law — no substitution).
|
||||
# Compute buffers at ubatch 512 ~1.3 GiB. CUDA ctx ~500 MiB. DeltaNet
|
||||
# state ~150 MiB/slot x 2. MTP draft ctx back: ~0.4 GiB. Non-KV ~13.9
|
||||
# GiB. Two q8_0 slots = 8,704 MiB. Predicted free ~2.6-3.2 GiB — back
|
||||
# over the old 2-3 GiB floor that the IQ3_S two-slot boot violated
|
||||
# (measured there: 1,397 MiB free). MEASURE nvidia-smi free at boot
|
||||
# (quote the free field) and let the runbook carry the number.
|
||||
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
||||
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
||||
# 2×200,704 f16 K + f16 V measured 21,276 used / 2,849 free at ubatch 1024
|
||||
# and 21,846 / 2,279 at ubatch 2048. This file is 415 MiB lighter in weights,
|
||||
# its BF16 mmproj is 3 MiB heavier ⇒ projected ~21,430 used / ~2,700 free.
|
||||
# The verbosity-4 boot ledger is the referee.
|
||||
#
|
||||
# Payload history (rollback trail): /home/dank/models/Swift15-IQ3S/ (the
|
||||
# IQ3_S-mtp that was live until this swap), Ornith-Q8/, MiMo-Q6K/ all
|
||||
# stay on disk; one MODEL_FILE + shape edit back each.
|
||||
# Ubatch 2048 is the measured CUDA optimum (measured on the MiMo predecessor
|
||||
# at this exact arch + shape: +4.6 % prefill at 8.9k depth, decode flat, cost
|
||||
# 616 MiB). 1024 is the fallback rung if the boot lands under the 2 GiB floor.
|
||||
LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
|
||||
MODEL_FILE=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf
|
||||
|
||||
# Vision: the publisher SHIPS the projector — mmproj-BF16.gguf
|
||||
# (931,145,888 B) from the same repo, fetched by
|
||||
# scripts/download-frank-secondary.sh. No substitution needed for this
|
||||
# payload (the unsloth base-tower proof of 2026-10-05 remains the
|
||||
# fallback argument for the IQ3_S rollback payload).
|
||||
# Empty string = text-only on purpose.
|
||||
MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf
|
||||
|
||||
# Fleet endpoint: same port and alias as every frank-secondary before it, so the
|
||||
# harness models.yml id and the homepage probe do not move.
|
||||
ALIAS=frank-secondary
|
||||
MODEL_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf
|
||||
MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf
|
||||
# Served model id — the harness/family name (Protoss rename completed to the
|
||||
# wire 2026-10-09: harness ids + homepage cards 2026-10-08, /v1/models aliases
|
||||
# this day). The SERVICE, unit, scripts and this env file keep the
|
||||
# frank-secondary name; only the alias on the wire changed. Consumers that
|
||||
# key on the served id: OpenWebUI (model row, access_grant, api_configs
|
||||
# model_ids) and the harness models.yml.
|
||||
ALIAS=frank-zealot
|
||||
BIND_ADDR=0.0.0.0
|
||||
LISTEN_PORT=7771
|
||||
|
||||
# TWO conversations at Frank's 112k cap (2026-10-06 evening): --ctx-size is
|
||||
# the POOL llama.cpp divides across slots; 229376 / 2 = 114,688 per
|
||||
# conversation. Lowering from 128k (131,072) frees ~1 GiB of KV, which
|
||||
# funds the ubatch 512 -> 1024 increase (prefill throughput). The
|
||||
# per-slot window is the product contract, so models.yml follows to
|
||||
# 114,688.
|
||||
CTX_SIZE=229376
|
||||
# CTX_SIZE is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
|
||||
# per conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
|
||||
# the harness contextWindow quotes the PER-SLOT number — a longer prompt is
|
||||
# answered with a 400, never truncated.
|
||||
CTX_SIZE=401408
|
||||
PARALLEL=2
|
||||
|
||||
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves);
|
||||
# the 1024 rung was the first A/B back once the per-slot context dropped
|
||||
# to 112k (2026-10-06 evening), and 2048 is the second (Frank, same night)
|
||||
# — the rung measured +4.6 % prefill for 616 MiB on the 3090 (2026-10-05
|
||||
# boot pair), affordable inside the 1,751 MiB the 112k cut freed. The pair
|
||||
# is a measurement, not a law (loud failure at graph capture if it does
|
||||
# not fit).
|
||||
BATCH_SIZE=4096
|
||||
UBATCH_SIZE=2048
|
||||
|
||||
# The KV contract for this swap (Frank, 2026-10-05): q8_0 K + q8_0 V —
|
||||
# llama.cpp's FP8 class. The flags stay explicit so the choice is visible in
|
||||
# the argv either way, and --flash-attn stays pinned on in the serve script:
|
||||
# it is required for quantized V (the build carries
|
||||
# GGML_CUDA_FA_ALL_QUANTS=ON since 2026-09-17, so q8_0 KV is a real path,
|
||||
# not a silent f16 fallback).
|
||||
CACHE_TYPE_K=q8_0
|
||||
CACHE_TYPE_V=q8_0
|
||||
|
||||
# Speculative decoding is ON (Frank, 2026-10-06 evening): draft-mtp at
|
||||
# depth 3 against the head IN the file (the -mtp variant ships the
|
||||
# blk.64.nextn.* tensors). ngram-mod stays skipped — on a qwen35 hybrid
|
||||
# it disables recurrent-state rollback, so a rejected draft would
|
||||
# corrupt the DeltaNet state; MTP needs no such bypass (its draft ctx
|
||||
# re-derives state from the target's own weights). The flags move as a
|
||||
# set: the serve script emits BOTH only when SPEC_TYPE names a drafter.
|
||||
SPEC_TYPE=draft-mtp
|
||||
SPEC_DRAFT_N_MAX=3
|
||||
|
||||
CACHE_TYPE_K=f16
|
||||
CACHE_TYPE_V=f16
|
||||
SPEC_TYPE=none
|
||||
NGL=99
|
||||
# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
|
||||
# and graph capture land in the journal, which is what the VRAM-fit forensics
|
||||
# read. Levels above 4 flood the journal.
|
||||
VERBOSITY=4
|
||||
@@ -1,5 +1,5 @@
|
||||
[Unit]
|
||||
Description=llama.cpp Swift-1.5-Qwen3.8-27B Uncensored RCO IQ3_XXS GGUF (qwen35 hybrid, 2x131072 slots, q8_0 K+V, draft-mtp depth 3, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
|
||||
Description=llama.cpp Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K GGUF (qwen35 hybrid, 2x200704 slots, f16 K+V, MTP off, mmproj vision) on the RTX 3090 (dankclaw, frank-secondary)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# kld-gate-swift15.sh — RUNS ON DANKSERVER. Score the quantized candidate
|
||||
# against the BF16 reference and apply the pre-approved gate:
|
||||
# mean KL <= 0.03 AND top-1 agreement >= 95%.
|
||||
# Same engine (1.0.13) and same boot shape as the reference RECORD
|
||||
# (run-swift15-bf16.sh kld): disk-tier, no speculation, deterministic, fp8 KV.
|
||||
# --kld-ref reads its tokens from the reference (no --kld-corpus here — the
|
||||
# (run-swift15-bf16.sh kld): 1.0.13, tp 2, disk-tier, fp8 KV, no speculation,
|
||||
# deterministic, max-model-len 16384, vram-kv 3072, 4 slots. Two deviations
|
||||
# are forced by the w4 file: MNBT 2048 (the w4 kernels stream from the file
|
||||
# tier ONLY at MNBT<=2048 — at the recording's 32768 the engine demands a
|
||||
# widened form: "DID NOT FIT ... stored form is not the container's"), and
|
||||
# pool 40960 (the recording's 12 GiB pool predates the w4 tables; at 12288 the
|
||||
# gate_up slab cannot be cached). Chunk size is softmax-invariant, so the
|
||||
# MNBT delta is float-order noise riding on the KL floor.
|
||||
set -uo pipefail
|
||||
PIN=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
C=swift15gate
|
||||
LOG=/home/dank/logs/swift15_kldgate.log
|
||||
KLD_REF_DIR=/home/dank/corpora/kld-ref
|
||||
KLD_OUT=/home/dank/radiance-state/kld-gate.json
|
||||
|
||||
[ -d "$KLD_REF_DIR" ] || { echo "missing reference: $KLD_REF_DIR"; exit 2; }
|
||||
: > "$KLD_OUT"
|
||||
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
||||
/home/dank/models/tpl.jinja 2>/dev/null || true
|
||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||
|
||||
podman run --rm --name "$C" \
|
||||
--network host --init \
|
||||
--security-opt seccomp=unconfined \
|
||||
--ulimit memlock=-1:-1 \
|
||||
--device /dev/kfd --device /dev/dri \
|
||||
-v /home/dank/models:/models:ro \
|
||||
-v /home/dank/radiance-state:/data \
|
||||
-v "$KLD_REF_DIR:/data/kld-ref:ro" \
|
||||
"stilldeadcode/radiance@$PIN" \
|
||||
--model /models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad \
|
||||
--tp 2 --placement expert_tiered --weights-disk-tier \
|
||||
--host-pool-mib 40960 --deterministic --gpu-headroom-mib 768 \
|
||||
--num-speculative-tokens 0 \
|
||||
--kv-cache-dtype fp8 --max-model-len 16384 \
|
||||
--ngram-placement ram \
|
||||
--override-chat-template /models/tpl.jinja \
|
||||
--kld-ref /data/kld-ref --kld-out /data/kld-gate.json \
|
||||
--vram-kv-mib 3072 \
|
||||
--max-num-seqs 4 --max-num-batched-tokens 2048 \
|
||||
2>&1 | tee "$LOG"
|
||||
rc=${PIPESTATUS[0]}
|
||||
[ "$rc" -eq 0 ] || { echo "KLD GATE BOOT FAILED rc=$rc; log: $LOG"; exit "$rc"; }
|
||||
|
||||
python3 - "$KLD_OUT" "$LOG" <<'PY'
|
||||
import json, sys, re
|
||||
out, log = sys.argv[1], sys.argv[2]
|
||||
def num(pat):
|
||||
m = re.findall(pat, open(log, errors="replace").read(), re.I)
|
||||
return float(m[-1]) if m else None
|
||||
kl = top1 = None
|
||||
try:
|
||||
d = json.load(open(out))
|
||||
def find(obj, keys):
|
||||
if isinstance(obj, dict):
|
||||
for k, v in obj.items():
|
||||
if any(s in k.lower() for s in keys) and isinstance(v, (int, float)):
|
||||
return float(v)
|
||||
for v in obj.values():
|
||||
r = find(v, keys)
|
||||
if r is not None: return r
|
||||
return None
|
||||
kl = find(d, ("mean_kl", "mean kl", "kl_mean", "mean"))
|
||||
top1 = find(d, ("top1", "top_1", "top-1", "agreement"))
|
||||
if top1 is not None and top1 <= 1.0:
|
||||
top1 *= 100.0
|
||||
except Exception as e:
|
||||
print("report JSON unusable (%s); falling back to log" % e)
|
||||
if kl is None:
|
||||
kl = num(r"mean[_ ]kl[^0-9]*([0-9]*\.?[0-9]+(?:e[-+][0-9]+)?)")
|
||||
if top1 is None:
|
||||
top1 = num(r"top[-_ ]?1[^0-9]*([0-9]*\.?[0-9]+)")
|
||||
if kl is None or top1 is None:
|
||||
print("KLD GATE PARSE FAILED: mean_kl=%s top1=%s — inspect %s / %s" % (kl, top1, out, log))
|
||||
sys.exit(2)
|
||||
print("KLD GATE: mean KL = %.4f (max 0.03) | top-1 = %.2f%% (min 95.0)" % (kl, top1))
|
||||
ok = (kl <= 0.03) and (top1 >= 95.0)
|
||||
print("KLD GATE " + ("PASS" if ok else "FAIL"))
|
||||
sys.exit(0 if ok else 1)
|
||||
PY
|
||||
gate=$?
|
||||
echo "kld-gate complete rc=$gate; log: $LOG report: $KLD_OUT"
|
||||
exit "$gate"
|
||||
@@ -0,0 +1,40 @@
|
||||
#!/bin/bash
|
||||
# kld-then-calib.sh — RUNS ON DANKSERVER. Campaign chain: wait for the KLD
|
||||
# record to finish, verify it, then boot the 1.2.0 calibration server and
|
||||
# push the 10M-token corpus. One GPU, so the phases serialize; prod is
|
||||
# restored by each run script's trap.
|
||||
set -u
|
||||
KLOG=/home/dank/logs/swift15-kld-driver.log
|
||||
CLOG=/home/dank/logs/swift15-calib-driver.log
|
||||
S=/home/dank/serverAppConfigs-swift15/scripts
|
||||
# Phase 1: KLD record (SKIP_KLD=1 when the record is already complete —
|
||||
# kld.json's rows count is the receipt; a mid-run script swap can kill the
|
||||
# wrapper with rc=127 after the engine exited clean).
|
||||
if [ "${SKIP_KLD:-0}" != 1 ]; then
|
||||
while ! grep -q "exited rc=" "$KLOG" 2>/dev/null; do sleep 60; done
|
||||
RC=$(grep -o "exited rc=[0-9]*" "$KLOG" | tail -1)
|
||||
if [ "$RC" != "exited rc=0" ]; then echo "KLD FAILED ($RC); calib NOT started"; exit 1; fi
|
||||
fi
|
||||
N=$(find /home/dank/corpora/kld-ref -type f | wc -l)
|
||||
SZ=$(du -sm /home/dank/corpora/kld-ref | cut -f1)
|
||||
echo "KLD OK files=$N size_mib=$SZ"
|
||||
[ "$N" -ge 5 ] && [ "$SZ" -ge 1000 ] || { echo "kld-ref looks thin; calib NOT started"; exit 1; }
|
||||
grep -q '"rows": 50235' /home/dank/corpora/kld-ref/kld.json || {
|
||||
echo "kld.json rows != 50235; calib NOT started"; exit 1; }
|
||||
|
||||
# Phase 2: calibration serve (1.2.0) + corpus push.
|
||||
nohup bash "$S/run-swift15-bf16.sh" calib > "$CLOG" 2>&1 &
|
||||
CP=$!
|
||||
for _ in $(seq 1 160); do
|
||||
sleep 15
|
||||
curl -sf localhost:7770/health >/dev/null 2>&1 && break
|
||||
done
|
||||
if ! curl -sf localhost:7770/health >/dev/null 2>&1; then
|
||||
echo "CALIB BOOT FAILED; see /home/dank/logs/swift15_bf16.log"; kill "$CP" 2>/dev/null; exit 1
|
||||
fi
|
||||
echo "CALIB UP, pushing corpus"
|
||||
python3 "$S/calib-push.py" --port 7770 --parallel 16 --container swift15ref --max-rounds 6
|
||||
PUSH=$?
|
||||
kill "$CP" 2>/dev/null
|
||||
sleep 10
|
||||
echo "CALIB_CHAIN_DONE push=$PUSH grams=$(find /home/dank/corpora/calib -name 'gram.r*.bin' 2>/dev/null | wc -l)"
|
||||
@@ -0,0 +1,317 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the two Swift1.5 campaign corpora (calibration + KLD).
|
||||
|
||||
calib.jsonl — 10M tokens of agent-shaped RAW text (45% code, 30% prose,
|
||||
25% tool-call JSON), one {"prompt": text} per line, pushed to
|
||||
/v1/completions with max_tokens=1 during the calibration run (the Hessian
|
||||
tap is of the prompt tokens; docs/MOE-W4.md step 2). Code and prose mix
|
||||
the repo checkout with HF dataset parquets (Frank's call 2026-10-06:
|
||||
local material is only ~1.6M unique tokens; MOE-W4 wants 10M):
|
||||
nampdn-ai/tiny-codes (code) and Salesforce/wikitext wikitext-103-raw
|
||||
(prose). Tool calls are synthesized from commands harvested from the
|
||||
repo's own scripts plus a fleet-ops set, with argument-shaped variants.
|
||||
|
||||
kld.jsonl — 1.8M tokens, one {"prompt", "score_from", "source"} per
|
||||
line; each prompt is a multi-turn conversation rendered through the
|
||||
Froggeric v22.5 template with a >= --kld-min-prompt (8192) token target,
|
||||
and the last 256 tokens (the tail of the assistant answer) are the
|
||||
scored span: score_from = n - 256. At 8k-token prompts the 1.8M corpus
|
||||
is ~220 prompts = ~56k scored positions ~= 25 GB of record, matching
|
||||
the published Flash-Next KLD scale (83k positions, MOE-W4.md). The KLD
|
||||
reference run and every candidate run consume the SAME file (TOOLS.md
|
||||
--kld-record / --kld-ref). The KLD corpus is LOCAL material only, so the
|
||||
file-level disjointness rule (fnv1a64(path) % 5 == 0 -> KLD pool) is
|
||||
untouched by the HF additions.
|
||||
|
||||
Deps: tokenizers, jinja2, pyarrow (venv /home/dank/venvs/radcorpus).
|
||||
Usage: make-calib-corpus.py --repo ROOT --tokenizer-json PATH
|
||||
--template PATH --out-dir DIR [--hf-code-parquet F,F]
|
||||
[--hf-prose-parquet F,F] [--calib-tokens 10000000]
|
||||
[--kld-tokens 1800000]
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
|
||||
from jinja2 import Environment, FileSystemLoader
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
CODE_EXT = (".sh", ".py", ".jinja", ".recipe", ".env", ".example",
|
||||
".yaml", ".yml", ".bats", ".service", ".container", ".toml")
|
||||
JSONL_EXTS = (".jsonl", ".json")
|
||||
DOC_DIR = "serverAppConfigs-docs"
|
||||
|
||||
FLEET_CMDS = [
|
||||
"systemctl status frank-primary --no-pager",
|
||||
"journalctl -u frank-primary -n 100 --no-pager",
|
||||
"podman ps --format '{{.Names}} {{.Status}}'",
|
||||
"podman logs --tail 50 frank-primary",
|
||||
"curl -s http://192.168.0.235:7770/health",
|
||||
"curl -s http://192.168.0.235:7770/metrics | grep radiance_",
|
||||
"df -h /home/dank | tail -1",
|
||||
"free -g",
|
||||
"sudo systemctl restart frank-primary",
|
||||
"cd /home/dank/serverAppConfigs && git pull --ff-only",
|
||||
"./scripts/deploy.sh dankserver --service frank-primary",
|
||||
"./tests/run.sh 2>&1 | tail -5",
|
||||
"jq . /home/dank/models/rad/index.json | head -40",
|
||||
"python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"",
|
||||
"ip neigh show 192.168.0.235",
|
||||
"amdgpu_top --device-info 2>/dev/null | head -20",
|
||||
"grep -E 'MemAvailable' /proc/meminfo",
|
||||
"sha256sum model-00001-of-00131.safetensors",
|
||||
"git log --oneline -10",
|
||||
"awk '{s+=$1} END {print s}' shard_sizes.txt",
|
||||
]
|
||||
TOOL_NAMES = ["bash", "read", "grep", "web_search", "edit", "write"]
|
||||
WORKDIRS = ["/home/dank", "/home/dank/serverAppConfigs", "/home/dank/models",
|
||||
"/etc/systemd/system", "/var/log"]
|
||||
DESCS = ["check service state", "tail the engine ledger", "list containers",
|
||||
"query health", "read metrics", "check disk", "check memory",
|
||||
"restart the endpoint", "sync the checkout", "run deployer",
|
||||
"run the test seams", "inspect the index", "probe the peer"]
|
||||
|
||||
|
||||
def fnv(s: str) -> int:
|
||||
h = 0xcbf29ce484222325
|
||||
for b in s.encode():
|
||||
h = ((h ^ b) * 0x100000001b3) & 0xFFFFFFFFFFFFFFFF
|
||||
return h
|
||||
|
||||
|
||||
def harvest_cmds(repo: str, rng: random.Random) -> list[str]:
|
||||
cmds = list(FLEET_CMDS)
|
||||
pat = re.compile(r"^\s*(?:sudo\s+)?([a-z][\w./-]*(?:\s+\S+){1,6})\s*$")
|
||||
for root, dirs, files in os.walk(repo):
|
||||
dirs[:] = [d for d in dirs if not d.startswith(".")]
|
||||
for f in files:
|
||||
if not f.endswith(".sh"):
|
||||
continue
|
||||
try:
|
||||
for line in open(os.path.join(root, f), encoding="utf-8",
|
||||
errors="replace"):
|
||||
m = pat.match(line.rstrip())
|
||||
if m and len(m.group(1)) > 12:
|
||||
cmds.append("ssh dank@192.168.0.235 '"
|
||||
+ m.group(1)[:90] + "'")
|
||||
except OSError:
|
||||
pass
|
||||
rng.shuffle(cmds)
|
||||
return cmds[:4000]
|
||||
|
||||
|
||||
def tool_call(rng: random.Random, cmds: list[str]) -> str:
|
||||
"""One agent turn: 2-4 tool calls with argument-shaped JSON."""
|
||||
parts = []
|
||||
for _ in range(rng.randint(2, 4)):
|
||||
args: dict = {"command": rng.choice(cmds)}
|
||||
if rng.random() < 0.4:
|
||||
args["cwd"] = rng.choice(WORKDIRS)
|
||||
if rng.random() < 0.3:
|
||||
args["timeout"] = rng.choice([10, 30, 60, 300, 600])
|
||||
if rng.random() < 0.5:
|
||||
args["description"] = rng.choice(DESCS)
|
||||
parts.append('<tool_call>{"name": "%s", "arguments": %s}<|im_end|>'
|
||||
% (rng.choice(TOOL_NAMES),
|
||||
json.dumps(args, ensure_ascii=False)))
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def load_parquet_texts(path: str, kind: str, rng: random.Random,
|
||||
cap: int) -> list[str]:
|
||||
import pyarrow.parquet as pq
|
||||
d = pq.read_table(path).to_pydict()
|
||||
n = len(next(iter(d.values())))
|
||||
idx = range(n) if n <= cap else rng.sample(range(n), cap)
|
||||
out = []
|
||||
for i in idx:
|
||||
if kind == "code":
|
||||
if "content" in d:
|
||||
frag = d["content"][i]
|
||||
else:
|
||||
frag = ("// %s\n### Instruction:\n%s\n### Response:\n%s"
|
||||
% (d.get("file_name", [""])[i], d["prompt"][i],
|
||||
d["code"][i] or d["response"][i]))
|
||||
else:
|
||||
frag = d["text"][i]
|
||||
if frag and frag.strip():
|
||||
out.append(frag[:8000])
|
||||
rng.shuffle(out)
|
||||
return out[:cap]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--repo", required=True)
|
||||
ap.add_argument("--tokenizer-json", required=True)
|
||||
ap.add_argument("--template", required=True)
|
||||
ap.add_argument("--out-dir", required=True)
|
||||
ap.add_argument("--calib-tokens", type=int, default=10_000_000)
|
||||
ap.add_argument("--kld-tokens", type=int, default=1_800_000)
|
||||
ap.add_argument("--chunk-tokens", type=int, default=8192)
|
||||
ap.add_argument("--score-tail", type=int, default=256)
|
||||
ap.add_argument("--kld-min-prompt", type=int, default=8192)
|
||||
ap.add_argument("--hf-code-parquet", default="")
|
||||
ap.add_argument("--hf-prose-parquet", default="")
|
||||
ap.add_argument("--seed", type=int, default=20261006)
|
||||
a = ap.parse_args()
|
||||
|
||||
rng = random.Random(a.seed)
|
||||
tokenizer = Tokenizer.from_file(a.tokenizer_json)
|
||||
env = Environment(loader=FileSystemLoader(os.path.dirname(a.template)),
|
||||
trim_blocks=True, lstrip_blocks=True)
|
||||
env.filters["tojson"] = lambda v: json.dumps(v, ensure_ascii=False)
|
||||
env.filters["raise_exception"] = lambda m: (_ for _ in ()).throw(
|
||||
ValueError(m))
|
||||
tname = os.path.basename(a.template)
|
||||
|
||||
code, prose, jsonl = [], [], []
|
||||
for root, dirs, files in os.walk(a.repo):
|
||||
dirs[:] = [d for d in dirs if not d.startswith(".")]
|
||||
for f in sorted(files):
|
||||
p = os.path.relpath(os.path.join(root, f), a.repo)
|
||||
if DOC_DIR in p and f.endswith(".md"):
|
||||
prose.append(p)
|
||||
elif p.startswith(DOC_DIR):
|
||||
pass
|
||||
elif f.endswith(CODE_EXT):
|
||||
code.append(p)
|
||||
elif f.endswith(JSONL_EXTS) and f != "package-lock.json":
|
||||
jsonl.append(p)
|
||||
# file-level split: bucket 0 -> KLD pool, 1..4 -> calibration pool
|
||||
pools: dict[str, tuple[list, list]] = {}
|
||||
for cls, lst in (("code", code), ("prose", prose), ("tools", jsonl)):
|
||||
pools[cls] = ([p for p in lst if fnv(p) % 5 == 0],
|
||||
[p for p in lst if fnv(p) % 5 != 0])
|
||||
|
||||
def text_of(p: str) -> str:
|
||||
path = os.path.join(a.repo, p)
|
||||
raw = open(path, encoding="utf-8", errors="replace").read()
|
||||
if p.endswith(JSONL_EXTS):
|
||||
lines = [json.loads(l)["content"] for l in raw.splitlines()
|
||||
if l.strip() and "content" in l]
|
||||
raw = "\n".join(lines)
|
||||
toks = tokenizer.encode(raw, add_special_tokens=False).ids
|
||||
return tokenizer.decode(toks[:2400], skip_special_tokens=True)
|
||||
|
||||
cmds = harvest_cmds(a.repo, rng)
|
||||
# fragment pools per class, corpus-split first
|
||||
frags: dict[str, dict[str, list[str]]] = {}
|
||||
for cls in pools:
|
||||
kld_files, cal_files = pools[cls]
|
||||
fr = {}
|
||||
src = {"kld": kld_files, "cal": cal_files}
|
||||
if cls == "tools":
|
||||
fr["kld"] = [tool_call(rng, cmds) for _ in range(600)]
|
||||
fr["cal"] = [tool_call(rng, cmds) for _ in range(40000)]
|
||||
else:
|
||||
fr["kld"] = [text_of(p) for p in src["kld"]]
|
||||
fr["cal"] = [text_of(p) for p in src["cal"]]
|
||||
frags[cls] = fr
|
||||
# per-class fragment caps: prose rows are short (~66 tok) so need many;
|
||||
# codeparrot files are long (~2.6k tok) so a small cap suffices
|
||||
hfcap = {"code": 40000, "prose": 60000, "tools": 0}
|
||||
for cls, spec in (("code", a.hf_code_parquet),
|
||||
("prose", a.hf_prose_parquet)):
|
||||
for path in [p for p in spec.split(",") if p]:
|
||||
frags[cls]["cal"].extend(
|
||||
load_parquet_texts(path, cls, rng, cap=hfcap[cls]))
|
||||
for cls in frags:
|
||||
rng.shuffle(frags[cls]["cal"])
|
||||
|
||||
def toks(s: str) -> int:
|
||||
return len(tokenizer.encode(s, add_special_tokens=False).ids)
|
||||
|
||||
def emit_calib() -> tuple[int, list[dict]]:
|
||||
targets = {"code": .45, "prose": .30, "tools": .25}
|
||||
rows: list[dict] = []
|
||||
for cls, frac in targets.items():
|
||||
want = int(a.calib_tokens * frac)
|
||||
pool = frags[cls]["cal"]
|
||||
if not pool:
|
||||
continue
|
||||
got, i, buf, nbuf = 0, 0, [], 0
|
||||
while got < want:
|
||||
frag = pool[i % len(pool)]
|
||||
i += 1
|
||||
n = toks(frag)
|
||||
if nbuf + n > a.chunk_tokens and buf:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
buf, nbuf = [], 0
|
||||
if got >= want:
|
||||
break
|
||||
buf.append(frag)
|
||||
nbuf += n
|
||||
if i > len(pool) * 40:
|
||||
break # pool exhausted with repeats; stop
|
||||
if buf and got < want:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
print(f"calib {cls}: {got} tokens, {i} fragments")
|
||||
rng.shuffle(rows)
|
||||
return len(rows), [{k: v for k, v in r.items() if k == "prompt"}
|
||||
for r in rows]
|
||||
|
||||
def emit_kld() -> tuple[int, list[dict]]:
|
||||
rows, total = [], 0
|
||||
classes = ["code", "prose", "tools"]
|
||||
i = {c: 0 for c in classes}
|
||||
guard = 0
|
||||
while total < a.kld_tokens and guard < 5000:
|
||||
guard += 1
|
||||
msgs, srcs = [], []
|
||||
for t in range(rng.choice([2, 3, 4])):
|
||||
cls = classes[(len(rows) + t) % 3]
|
||||
pool = frags[cls]["kld"]
|
||||
if not pool:
|
||||
continue
|
||||
frag = pool[i[cls] % len(pool)]
|
||||
i[cls] += 2
|
||||
msgs.append({"role": "user",
|
||||
"content": "Summarize and explain this "
|
||||
f"engineering material:\n{frag}"})
|
||||
msgs.append({"role": "assistant",
|
||||
"content": frag[:4000]})
|
||||
srcs.append(cls)
|
||||
rendered = env.get_template(tname).render(
|
||||
messages=msgs, add_generation_prompt=False)
|
||||
n = toks(rendered)
|
||||
if n < a.kld_min_prompt:
|
||||
continue
|
||||
rows.append({"prompt": rendered,
|
||||
"score_from": n - a.score_tail,
|
||||
"source": "+".join(sorted(set(srcs)))})
|
||||
total += n
|
||||
return len(rows), rows
|
||||
|
||||
os.makedirs(a.out_dir, exist_ok=True)
|
||||
nc, rows = emit_calib()
|
||||
with open(os.path.join(a.out_dir, "calib.jsonl"), "w") as f:
|
||||
for r in rows:
|
||||
f.write(json.dumps(r) + "\n")
|
||||
nk, krows = emit_kld()
|
||||
with open(os.path.join(a.out_dir, "kld.jsonl"), "w") as f:
|
||||
for r in krows:
|
||||
f.write(json.dumps(r) + "\n")
|
||||
scored = sum(len(tokenizer.encode(r["prompt"], add_special_tokens=False).ids)
|
||||
- r["score_from"] for r in krows)
|
||||
manifest = {
|
||||
"seed": a.seed, "template": tname,
|
||||
"tokenizer": os.path.basename(a.tokenizer_json),
|
||||
"calib_prompts": nc, "kld_prompts": nk,
|
||||
"kld_scored_positions": scored,
|
||||
"kld_score_tail": a.score_tail,
|
||||
"disjoint_file_pools": "local KLD material only; HF code feeds calib",
|
||||
"calib_mix": {"code": .45, "prose": .30, "tools": .25},
|
||||
"hf_code": a.hf_code_parquet, "hf_prose": a.hf_prose_parquet,
|
||||
}
|
||||
with open(os.path.join(a.out_dir, "manifest.json"), "w") as f:
|
||||
json.dump(manifest, f, indent=1)
|
||||
print(json.dumps(manifest, indent=1))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,105 @@
|
||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
||||
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
||||
# base) as a radiance container for frank-primary: EVERY routed expert -- gate_up AND down --
|
||||
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
||||
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
||||
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
||||
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
||||
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
||||
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
||||
#
|
||||
# Why down is the calibrated half: the calibration tap rides the down projection, so its GPTQ
|
||||
# rounding propagates against real down-input Hessians; gate_up shares the same w4 table
|
||||
# uncalibrated. The published w5dn64 comparison (mean KL 0.0789 -> 0.0717, top-1 92.85% ->
|
||||
# 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier trades part of that
|
||||
# quality back for prefill.
|
||||
#
|
||||
# CALIB=DIR rad-convert <Swift1.5-Qwen3.8-Flash-Next checkpoint> \
|
||||
# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||
#
|
||||
# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of
|
||||
# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16:
|
||||
# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it.
|
||||
# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint +
|
||||
# container + container + output > purge-freed space), and --reuse only saves a re-read.
|
||||
|
||||
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
||||
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
||||
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
||||
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
||||
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
||||
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
||||
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
||||
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
||||
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
||||
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
||||
blk.2.ffn_*_exps.279.weight cast dtype=bf16
|
||||
blk.2.ffn_*_exps.413.weight cast dtype=bf16
|
||||
blk.34.ffn_*_exps.146.weight cast dtype=bf16
|
||||
blk.34.ffn_*_exps.153.weight cast dtype=bf16
|
||||
blk.34.ffn_*_exps.407.weight cast dtype=bf16
|
||||
blk.34.ffn_*_exps.453.weight cast dtype=bf16
|
||||
blk.37.ffn_*_exps.288.weight cast dtype=bf16
|
||||
blk.37.ffn_*_exps.377.weight cast dtype=bf16
|
||||
blk.38.ffn_*_exps.406.weight cast dtype=bf16
|
||||
blk.38.ffn_*_exps.420.weight cast dtype=bf16
|
||||
blk.42.ffn_*_exps.37.weight cast dtype=bf16
|
||||
blk.42.ffn_*_exps.90.weight cast dtype=bf16
|
||||
blk.42.ffn_*_exps.302.weight cast dtype=bf16
|
||||
blk.42.ffn_*_exps.388.weight cast dtype=bf16
|
||||
blk.44.ffn_*_exps.315.weight cast dtype=bf16
|
||||
blk.44.ffn_*_exps.350.weight cast dtype=bf16
|
||||
blk.46.ffn_*_exps.290.weight cast dtype=bf16
|
||||
blk.46.ffn_*_exps.419.weight cast dtype=bf16
|
||||
blk.47.ffn_*_exps.119.weight cast dtype=bf16
|
||||
blk.47.ffn_*_exps.143.weight cast dtype=bf16
|
||||
blk.47.ffn_*_exps.399.weight cast dtype=bf16
|
||||
blk.47.ffn_*_exps.445.weight cast dtype=bf16
|
||||
# PROTECTED-END
|
||||
# (derive-protected-experts.py rewrote this section from Swift's own grams)
|
||||
|
||||
# Every other routed expert, trunk and MTP layer alike: u4 codes into the w4nl table, rotated by
|
||||
# the 128-wide Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least
|
||||
# error as stored, chosen by GPTQ against $CALIB and swept three times by coordinate descent.
|
||||
# The calibration Grams are down-input Hessians, so down is the calibrated half of the table;
|
||||
# gate_up has no recording of its own and quantises uncalibrated (said on stderr, as the base
|
||||
# recipe says). The MTP layer has no recording either. Expert mass: gate_up 150.00 GiB -> ~37.6,
|
||||
# down 75.00 GiB -> ~18.8: ~56 GiB of experts, ~28 GiB a rank at TP=2 -- well inside the pool.
|
||||
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
|
||||
|
||||
# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP
|
||||
# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover
|
||||
# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is
|
||||
# [10240, 320] and takes four scales a row, 80 columns each.
|
||||
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
||||
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
|
||||
|
||||
# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts.
|
||||
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
||||
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
||||
|
||||
# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error.
|
||||
# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and
|
||||
# keeping it as the checkpoint holds it costs 20 MB.
|
||||
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
|
||||
# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs
|
||||
# ~0.6 points of top-1 on its own.
|
||||
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
|
||||
# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from.
|
||||
# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over
|
||||
# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau
|
||||
# before this rule is trusted.
|
||||
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
|
||||
|
||||
# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no
|
||||
# mtp.draft_head.weight (same as the base), kept for shape parity with the family.
|
||||
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
|
||||
@@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env bash
|
||||
# run-swift15-bf16.sh — RUNS ON DANKSERVER. Boot the no-recipe bf16 Swift1.5
|
||||
# container for the two offline campaign phases. The engine owns the GPUs, so
|
||||
# the production unit must be stopped first (this script does it and restores
|
||||
# on exit, autoresearch pattern: stop, never mask — relinked unit).
|
||||
#
|
||||
# kld --kld-record the reference logits from the bf16 model
|
||||
# (runs the corpus and EXITS; engine writes DIR and leaves)
|
||||
# calib serve with the calibration tap; a separate driver
|
||||
# (scripts/calib-push.py) pushes the corpus; the engine writes the
|
||||
# Hessians at RADIANCE_CALIB_TOKENS and the tap switches itself off.
|
||||
# Stop this script after calib-push prints TAP_DONE.
|
||||
#
|
||||
# Shape = the vendor's bf16 calibration lines (GUIDE.md §4.3:528 + MOE-W4.md
|
||||
# "calibrate the checkpoint as it ships"): expert_tiered + weights-disk-tier,
|
||||
# no speculation, deterministic. Large prefill steps because a disk-tier step
|
||||
# costs the same at 1 token as at 32K. The container declared --max-ctx 4096;
|
||||
# --max-model-len 16384 exceeds it on purpose (the KLD prompts run to 12,007
|
||||
# tokens, calib to 8,242) — the boot ledger arbitrates the QSA buffers; if
|
||||
# declare refuses, re-convert with a wider --max-ctx.
|
||||
set -uo pipefail
|
||||
|
||||
MODE="${1:?usage: run-swift15-bf16.sh kld|calib}"
|
||||
PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
PIN_120=sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa
|
||||
RAD=/home/dank/models/rad/qwen4exp-swift15-bf16.rad
|
||||
C=swift15ref
|
||||
LOG=/home/dank/logs/swift15_bf16.log
|
||||
KLD_REF_DIR=/home/dank/corpora/kld-ref
|
||||
CALIB_DIR=/home/dank/corpora/calib
|
||||
KLD_CORPUS=/home/dank/corpora/out/kld.jsonl
|
||||
|
||||
restore() { sudo systemctl start frank-primary >/dev/null 2>&1 || true; }
|
||||
trap restore EXIT
|
||||
sudo systemctl stop frank-primary >/dev/null 2>&1 || true
|
||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||
|
||||
ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
||||
--tp 2 --placement expert_tiered --weights-disk-tier
|
||||
--host-pool-mib 12288 --deterministic --gpu-headroom-mib 768
|
||||
--num-speculative-tokens 0
|
||||
--kv-cache-dtype fp8 --max-model-len 16384
|
||||
--ngram-placement ram
|
||||
--override-chat-template /models/tpl.jinja)
|
||||
|
||||
case "$MODE" in
|
||||
kld)
|
||||
# 1.0.13: the engine that will serve the candidate — the reference is
|
||||
# recorded on the same engine that scores it. The weights slab DERIVES as
|
||||
# card - headroom - kv, and the disk tier refuses under 9.57 GiB (4.21 of
|
||||
# layer buffers + 5.36 fixed), so kv is explicit SMALL: recording is
|
||||
# prefill-only, self-batched at MNBT 32768 = two 16k prompts at a time;
|
||||
# 4 slots x 16k fp8 (29 KiB/pos, 1.7 GiB) + headroom fits 3072.
|
||||
IMAGE="stilldeadcode/radiance@$PIN_1013"
|
||||
mkdir -p "$KLD_REF_DIR"
|
||||
ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
|
||||
--vram-kv-mib 3072
|
||||
--max-num-seqs 4 --max-num-batched-tokens 32768)
|
||||
;;
|
||||
calib)
|
||||
# 1.2.0: the calibration tap (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS)
|
||||
# lives in the 1.2.0 qwen4exp plugin; 1.0.13 has only the DIR half with an
|
||||
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
|
||||
# Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus
|
||||
# ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB
|
||||
# card — pool minus the 2.17 GiB fixed holds ~237k fp8 tokens a rank, so
|
||||
# the vendor's big-step calibration shape is affordable: MOE-W4.md:244
|
||||
# measures 640 tok/s at a 32k step against 930 at 55k, and the doc's own
|
||||
# run drives MNBT 65536. 8 slots x 16k = 131k resident tokens fits the
|
||||
# pool; the step batches 65k. KLD's observed rate confirmed step size,
|
||||
# not slot count, is the throughput lever.
|
||||
IMAGE="stilldeadcode/radiance@$PIN_120"
|
||||
mkdir -p "$CALIB_DIR"
|
||||
ARGS+=(--port 7770 --vram-kv-mib 3072
|
||||
--max-num-seqs 8 --max-num-batched-tokens 65536
|
||||
--prefix-cache-dir /data/calib-kvcache)
|
||||
;;
|
||||
probe)
|
||||
IMAGE="stilldeadcode/radiance@$PIN_1013"
|
||||
ARGS+=(--port 7770 --max-num-seqs 8 --max-num-batched-tokens 4096)
|
||||
;;
|
||||
*) echo "unknown mode: $MODE" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
# The prod template copy, readable from the container: mount the models tree
|
||||
# ro plus the state dir rw (KLD record + prefix tier land there).
|
||||
cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \
|
||||
/home/dank/models/tpl.jinja 2>/dev/null || true
|
||||
|
||||
KLD_MOUNT=()
|
||||
if [ "$MODE" = kld ]; then
|
||||
# Nested under the /data mount, so these must come after it (line order
|
||||
# in podman run below): the corpus read-only, the record dir writable.
|
||||
KLD_MOUNT=(-v "$KLD_CORPUS:/data/kld.jsonl:ro"
|
||||
-v "$KLD_REF_DIR:/data/kld-ref")
|
||||
fi
|
||||
EXTRA_ENV=()
|
||||
if [ "$MODE" = calib ]; then
|
||||
EXTRA_ENV=(-v "$CALIB_DIR:/data/calib"
|
||||
-e RADIANCE_CALIB_DIR=/data/calib
|
||||
-e RADIANCE_CALIB_TOKENS=10000000)
|
||||
fi
|
||||
|
||||
podman run --rm --name "$C" \
|
||||
--network host --init \
|
||||
--security-opt seccomp=unconfined \
|
||||
--ulimit memlock=-1:-1 \
|
||||
--device /dev/kfd --device /dev/dri \
|
||||
-v /home/dank/models:/models:ro \
|
||||
-v /home/dank/radiance-state:/data \
|
||||
"${KLD_MOUNT[@]}" \
|
||||
"${EXTRA_ENV[@]}" \
|
||||
"$IMAGE" "${ARGS[@]}" 2>&1 | tee "$LOG"
|
||||
rc=${PIPESTATUS[0]}
|
||||
echo "swift15-bf16 $MODE exited rc=$rc; log: $LOG"
|
||||
exit "$rc"
|
||||
@@ -51,7 +51,7 @@ set +a
|
||||
: "${FRANK_LARGE_VISION_TOWER:?FRANK_LARGE_VISION_TOWER must name the tower}"
|
||||
: "${FRANK_LARGE_VISION_BYTES:?FRANK_LARGE_VISION_BYTES must pin the tower size}"
|
||||
: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}"
|
||||
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-large)}"
|
||||
: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-dragoon)}"
|
||||
: "${HALOGEN_CTX:?HALOGEN_CTX must be set (per-slot window)}"
|
||||
: "${HALOGEN_KV_POOL_POSITIONS:?HALOGEN_KV_POOL_POSITIONS must be set}"
|
||||
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
|
||||
|
||||
@@ -50,7 +50,8 @@ set +a
|
||||
: "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}"
|
||||
: "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}"
|
||||
: "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}"
|
||||
: "${FRANK_PRIMARY_EXPERT_VS_CACHE:?FRANK_PRIMARY_EXPERT_VS_CACHE must be set}"
|
||||
: "${FRANK_PRIMARY_VRAM_WEIGHTS_MIB:?FRANK_PRIMARY_VRAM_WEIGHTS_MIB must be set}"
|
||||
: "${FRANK_PRIMARY_VRAM_KV_MIB:?FRANK_PRIMARY_VRAM_KV_MIB must be set}"
|
||||
: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}"
|
||||
: "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}"
|
||||
: "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}"
|
||||
@@ -186,7 +187,8 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
|
||||
--placement "$FRANK_PRIMARY_PLACEMENT" \
|
||||
--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \
|
||||
--gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \
|
||||
--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \
|
||||
--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \
|
||||
--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \
|
||||
--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
|
||||
--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \
|
||||
--prefix-cache-dir /data/kvcache/frank-primary \
|
||||
|
||||
@@ -1,28 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
||||
# (RCO-IQ3_XXS, **MTP head in the file**) as frank-secondary on Dankclaw's
|
||||
# RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
||||
# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
|
||||
# frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
|
||||
#
|
||||
# 2026-10-06 (Frank): TWO 131,072 conversations (262144 / 2) at the 128k
|
||||
# cap, ubatch 512, and MTP BACK ON (draft-mtp, depth 3 — the payload is
|
||||
# 1.56 GiB lighter than the IQ3_S-mtp it replaced, which is what funds
|
||||
# the draft ctx plus the second slot; ngram stays skipped — on a qwen35
|
||||
# hybrid it disables recurrent-state rollback). The q8_0 KV contract
|
||||
# (FP8 class) is unchanged; the per-tensor flags stay explicit either
|
||||
# way.
|
||||
# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
|
||||
# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
|
||||
# room the 26B MoE Gemma4 shape lacked, so quantized V retires. MTP stays off:
|
||||
# this GGUF carries no MTP tensors at all (verified from its tensor-name
|
||||
# list), so there is no in-file head to draft with. Verbosity 4 ships so the
|
||||
# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the journal.
|
||||
#
|
||||
# Architecture: qwen35 hybrid — 64 real layers, 16 full-attention + 48
|
||||
# Gated DeltaNet (full_attention_interval 4), native 262144 window, plus
|
||||
# the blk.64 MTP head (nextn_predict_layers=1). KV lives only in the 16
|
||||
# attention layers (34 KiB/token at q8_0); the DeltaNet state is per-slot
|
||||
# fixed size.
|
||||
# Vision comes from the projector file (the GGUF carries no vision
|
||||
# tensors; the publisher SHIPS mmproj-BF16.gguf in the same repo), so
|
||||
# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the
|
||||
# documented text-only opt-out.
|
||||
# Architecture: qwen35 — dense 9B, 32 layers with full_attention_interval 4
|
||||
# (8 full-attention layers at 3,7,...,31 + 24 Gated DeltaNet linear layers),
|
||||
# 16 heads / 4 KV heads × 256 dim, native 262,144, vocab 248,320. The KV pool
|
||||
# scales with the 8 full layers: 32 KiB/token at f16 K + f16 V (12.25 GiB at
|
||||
# the 401,408 pool); the DeltaNet layers cost a fixed recurrent state per
|
||||
# slot. Feasibility is priced from the MiMo-V2.6-Distill-Qwen-9B boots (same
|
||||
# geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279
|
||||
# at ubatch 2048 — the boot ledger, not arithmetic, is the referee.
|
||||
# Vision comes from the publisher's own BF16 projector (the model is natively
|
||||
# multimodal), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
|
||||
# the documented text-only opt-out.
|
||||
#
|
||||
# Sampling: llama.cpp takes it per request (the fleet harness sends its
|
||||
# recipe from models.yml); greedy is the harness default.
|
||||
# Sampling: llama.cpp takes it per request (the fleet harness sends the
|
||||
# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
|
||||
#
|
||||
# Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides.
|
||||
set -euo pipefail
|
||||
@@ -38,39 +38,46 @@ fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
|
||||
: "${MODEL_FILE:=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
|
||||
: "${ALIAS:=frank-secondary}"
|
||||
: "${MODEL_FILE:=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
|
||||
: "${ALIAS:=frank-zealot}"
|
||||
: "${BIND_ADDR:=0.0.0.0}"
|
||||
: "${LISTEN_PORT:=7771}"
|
||||
# ctx-size is the POOL llama.cpp divides across slots: 262144 / 2 gives
|
||||
# TWO 131,072 conversations — Frank's 128k cap each. 2026-10-06 (Frank):
|
||||
# drop MTP, drop ubatch to 512, take two slots; the fit is tight by his
|
||||
# explicit order (frank-secondary.env carries the arithmetic and the floor
|
||||
# violation).
|
||||
: "${CTX_SIZE:=262144}"
|
||||
# ctx-size is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
|
||||
# per conversation, inside the model's trained 262,144. The harness
|
||||
# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
|
||||
# with a 400, never truncated.
|
||||
: "${CTX_SIZE:=401408}"
|
||||
: "${PARALLEL:=2}"
|
||||
# Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
|
||||
# Frank's 2026-10-06 tight-fit order ships 4096/512 (claws back ~0.3 GiB
|
||||
# toward the second slot); ubatch 1024 is the first A/B back if a future
|
||||
# boot lands with headroom (a miss fails loud at graph capture). The
|
||||
# logical batch is VRAM- and throughput-neutral at a fixed ubatch.
|
||||
# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
|
||||
# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
|
||||
# throughput-neutral at a fixed ubatch.
|
||||
: "${BATCH_SIZE:=4096}"
|
||||
: "${UBATCH_SIZE:=512}"
|
||||
# KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
|
||||
# there is no true fp8 KV dtype). Explicit either way so the contract is
|
||||
# visible in the argv.
|
||||
: "${CACHE_TYPE_K:=q8_0}"
|
||||
: "${CACHE_TYPE_V:=q8_0}"
|
||||
# Speculative decoding is ON by default (Frank, 2026-10-06 evening):
|
||||
# draft-mtp against the head that is IN the file. SPEC_TYPE none strips
|
||||
# BOTH speculative flags (flags move as a set) — a no-spec baseline must
|
||||
# not carry half a spec pair.
|
||||
: "${SPEC_TYPE:=draft-mtp}"
|
||||
: "${UBATCH_SIZE:=2048}"
|
||||
# Exact (unquantized) KV: f16 K + f16 V, restoring the MiMo-era contract that
|
||||
# the 26B Gemma4 shape could not afford. 32 KiB/token at this geometry
|
||||
# (8 full-attention layers × 4 KV heads × 256 dim); 12.25 GiB at the 401,408
|
||||
# pool. Explicit either way so the contract is visible in the argv, and
|
||||
# --flash-attn stays required for any quantized-V fallback (q8_0 = llama.cpp's
|
||||
# FP8 class; no true fp8 KV dtype exists in this build).
|
||||
: "${CACHE_TYPE_K:=f16}"
|
||||
: "${CACHE_TYPE_V:=f16}"
|
||||
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
|
||||
# there is no in-file head (the law: MTP is a property of the file, not of the
|
||||
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
|
||||
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
||||
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
||||
: "${SPEC_TYPE:=none}"
|
||||
: "${SPEC_DRAFT_N_MAX:=3}"
|
||||
: "${SPEC_DRAFT_FILE:=}"
|
||||
: "${NGL:=99}"
|
||||
# Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
|
||||
# 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
|
||||
# the VRAM-fit forensics need. Levels above 4 flood the journal.
|
||||
: "${VERBOSITY:=3}"
|
||||
# Vision projector. No colon on purpose: an explicitly empty value means
|
||||
# "text-only, on purpose" and must NOT be replaced by the default path.
|
||||
: "${MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
|
||||
: "${MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
|
||||
|
||||
# Fail loudly rather than half-loading: a missing file is a download that
|
||||
# has not finished, and Restart=always would turn it into a crash loop that
|
||||
@@ -89,6 +96,10 @@ fi
|
||||
# baseline must not carry half a spec pair.
|
||||
if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
||||
EXTRA_ARGS+=(--spec-type "$SPEC_TYPE" --spec-draft-n-max "$SPEC_DRAFT_N_MAX")
|
||||
if [[ -n "$SPEC_DRAFT_FILE" ]]; then
|
||||
[[ -e "$SPEC_DRAFT_FILE" ]] || { echo "frank-secondary: missing $SPEC_DRAFT_FILE" >&2; exit 1; }
|
||||
EXTRA_ARGS+=(--spec-draft-model "$SPEC_DRAFT_FILE")
|
||||
fi
|
||||
fi
|
||||
|
||||
# The UVM device node is created lazily by nvidia-modprobe on the first CUDA
|
||||
@@ -113,6 +124,7 @@ ARGS=(
|
||||
--flash-attn on
|
||||
--cache-type-k "$CACHE_TYPE_K"
|
||||
--cache-type-v "$CACHE_TYPE_V"
|
||||
--log-verbosity "$VERBOSITY"
|
||||
--jinja
|
||||
--reasoning auto
|
||||
--reasoning-format deepseek
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Streaming absmax audit of the Swift1.5 PLE n-gram table (BF16).
|
||||
|
||||
The PLE fp8 encoding uses ONE fixed scale (1.9931793212890625e-04, the
|
||||
official checkpoint's plateau value). fp8_e4m3 saturates at 448, so the
|
||||
fixed scale is legal iff absmax <= 448 * SCALE. This walks every
|
||||
ngram_embedding.shard_*.weight tensor in the checkpoint (mmap, no copy),
|
||||
computes the absmax from the BF16 bit pattern (clear the sign bit, max
|
||||
the remaining uint16 — BF16 magnitudes are monotone in their bits), and
|
||||
reports the headroom. Exit 0 only when headroom >= 1.5x.
|
||||
|
||||
Run from any cwd EXCEPT /tmp on dankserver (/tmp/inspect.py shadows the
|
||||
stdlib). Usage: verify-swift15-ple.py <ckpt-dir>
|
||||
"""
|
||||
import json
|
||||
import mmap
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
|
||||
SCALE = 1.9931793212890625e-04 # bf16 0x3951, the official plateau scale
|
||||
FP8_E4M3_MAX = 448.0
|
||||
|
||||
import numpy as np
|
||||
gmax = 0.0
|
||||
gfile = ""
|
||||
over_cap = 0
|
||||
total = 0
|
||||
|
||||
ckpt = sys.argv[1]
|
||||
index = json.load(open(os.path.join(ckpt, "model.safetensors.index.json")))
|
||||
shards: dict[str, list[str]] = {}
|
||||
for k, f in index["weight_map"].items():
|
||||
if "ngram_embedding.shard_" in k and k.endswith(".weight"):
|
||||
shards.setdefault(f, []).append(k)
|
||||
|
||||
files = len(shards)
|
||||
tensors = sum(len(v) for v in shards.values())
|
||||
print(f"n-gram shards: {files} files, {tensors} tensors")
|
||||
|
||||
import math
|
||||
|
||||
cap = FP8_E4M3_MAX * SCALE
|
||||
# largest bf16 <= cap: truncate the float32 bits (cap > 0); bf16
|
||||
# magnitudes are monotone in their bits, so |w| > cap <=> bits > cap_bits
|
||||
cap_bits = int((np.float32(cap).view(np.uint32) >> 16) & 0xFFFF)
|
||||
for fname, keys in sorted(shards.items()):
|
||||
path = os.path.join(ckpt, fname)
|
||||
with open(path, "rb") as fh:
|
||||
hlen = struct.unpack("<Q", fh.read(8))[0]
|
||||
header = json.loads(fh.read(hlen))
|
||||
base = 8 + hlen
|
||||
mm = mmap.mmap(fh.fileno(), 0, prot=mmap.PROT_READ)
|
||||
for k in keys:
|
||||
meta = header[k]
|
||||
off, end = meta["data_offsets"]
|
||||
a = np.frombuffer(mm, dtype=np.uint16, count=(end - off) // 2,
|
||||
offset=base + off)
|
||||
v = a & 0x7FFF
|
||||
m = int(v.max())
|
||||
over_cap += int((v > np.uint16(cap_bits)).sum())
|
||||
total += v.size
|
||||
del a
|
||||
if m > gmax:
|
||||
gmax = m
|
||||
gfile = k
|
||||
mm.close()
|
||||
|
||||
# rebuild the BF16 value: the 16 bits are the HIGH half of the float32
|
||||
absmax = np.frombuffer(b"\x00\x00" + np.uint16(gmax).tobytes(),
|
||||
dtype=np.float32)[0]
|
||||
|
||||
print(f"absmax = {absmax!r} (tensor: {gfile})")
|
||||
print(f"2^log2 = 2^{math.log2(absmax):.3f}")
|
||||
print(f"fp8 cap = 448 * {SCALE:.8g} = {cap:.8g}")
|
||||
print(f"headroom = {cap / absmax:.3f}x")
|
||||
ok = cap / absmax >= 1.5
|
||||
print("VERDICT:", "PASS (>= 1.5x headroom over the fixed scale)" if ok
|
||||
else "FAIL — fixed scale does not cover the Swift PLE")
|
||||
print(f"values > cap = {over_cap} of {total} ({over_cap / total:.3e})")
|
||||
nxt_bits = int((np.float32(absmax / FP8_E4M3_MAX).view(np.uint32)
|
||||
>> 16) & 0xFFFF) + 1
|
||||
nxt_scale = np.frombuffer(b"\x00\x00" + np.uint16(nxt_bits).tobytes(),
|
||||
dtype=np.float32)[0]
|
||||
print(f"next-bf16 scale covering absmax: {nxt_scale:.8g} "
|
||||
f"(0x{nxt_bits:04x}, headroom {FP8_E4M3_MAX * nxt_scale / absmax:.4f}x)")
|
||||
sys.exit(0 if ok else 1)
|
||||
@@ -13,6 +13,18 @@ registry; single source of truth — there is no copy on the servers) and
|
||||
`~/.omp/agent/config.yml` (roles, compaction, global samplers). Nothing in
|
||||
this file is repo-managed; edits are harness-side only.
|
||||
|
||||
**Model ids are the StarCraft unit names (official 2026-10-08, on the wire
|
||||
since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
|
||||
`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
|
||||
:7771, the 3090 compaction hub) — formerly `frank-primary`, `frank-large`,
|
||||
`frank-secondary`. The SERVICES keep the old names (units, scripts, env
|
||||
files, runbook pages); pages written in the old era name the service,
|
||||
the harness names the model. The wire now matches: each endpoint's
|
||||
`/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
|
||||
picker/grants, the homepage docs and the raw API all use one id per model
|
||||
(see [[OpenWebUI]] for the grant re-key and the [[frank-primary-radiance-swift15-w4dn8]]
|
||||
note on radiance echoing the requested name).
|
||||
|
||||
## Compaction chain arithmetic (verify before ANY window change)
|
||||
|
||||
When a fleet endpoint's context grows past its harness-declared window, OMP
|
||||
|
||||
@@ -24,22 +24,41 @@ https://github.com/open-webui/open-webui
|
||||
| Upstream latest | 0.11.4 (2026-09-21) — security/access-control advisory; `:main` is the rolling image, not a stable pin. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
|
||||
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
|
||||
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
|
||||
| Backends | Three fleet endpoints: `frank-large` (host.containers.internal:7770), `frank-medium` (192.168.0.235:7770), `frank-secondary` (host.containers.internal:7771) — see [The LLM backends](#the-llm-backends) |
|
||||
| Backends | Picker exposes two models — `frank-zealot` (host.containers.internal:7771) and `frank-dragoon` (host.containers.internal:7770, added 2026-10-09). The `frank-archon` (Dankserver) connection remains configured in the DB, disabled — see [The LLM backends](#the-llm-backends) |
|
||||
| Network | Pinned bridge `systemd-open-webui` (`10.89.3.0/24`, gateway `10.89.3.1`) — see [History](#history) |
|
||||
| Persistence | Quadlet unit `open-webui.service`; `[Install] WantedBy=default.target` |
|
||||
| Deploy | `scripts/deploy-open-webui-quadlet.sh dankclaw` |
|
||||
|
||||
## The LLM backends
|
||||
|
||||
OpenWebUI serves the three fleet LLM endpoints. The model picker is built from
|
||||
the `model` table in `webui.db` (one row per model), NOT from live connection
|
||||
discovery — so the picker shows exactly the models seeded there.
|
||||
The picker is built from the `model` table in `webui.db` (one row per model)
|
||||
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
|
||||
connection discovery — so the picker shows exactly what is seeded there.
|
||||
As of 2026-10-09 the picker exposes **two models: `frank-zealot` and
|
||||
`frank-dragoon`**.
|
||||
|
||||
| Picker model | Connection URL | Runtime | Host |
|
||||
|--------------|----------------|---------|------|
|
||||
| `frank-large` | `http://host.containers.internal:7770/v1` | unsloth llama.cpp (Strix Halo iGPU, 200k ctx) | Dankclaw |
|
||||
| `frank-medium` | `http://192.168.0.235:7770/v1` | vllm-radiance TP=2 (FP8, DFlash2, 262k ctx) | Dankserver |
|
||||
| `frank-secondary` | `http://host.containers.internal:7771/v1` | llama.cpp (RTX 3090, Gemma4-12B, 262k ctx) | Dankclaw |
|
||||
| Connection (`openai.api_configs` idx) | URL | Enabled | `model_ids` | Runtime | Host |
|
||||
|---|---|---|---|---|---|
|
||||
| 0 | `http://host.containers.internal:7770/v1` | yes (2026-10-09) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
|
||||
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
|
||||
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090, 2×200,704) | Dankclaw |
|
||||
|
||||
The `model` table carries one decoration row per picker model:
|
||||
`frank-zealot` (context length 200,704 = the per-slot window; description:
|
||||
Qwen3.5-9B Q6_K + BF16 vision mmproj) and `frank-dragoon` (context length
|
||||
262,144 = the per-slot `HALOGEN_CTX`; description: Swift1.5 v2 abliterated,
|
||||
MTP, vision ON).
|
||||
|
||||
A third gate sits *after* the table, and it is the one that bites family
|
||||
accounts (2026-10-07): a model row is only visible to a non-admin user when
|
||||
its **access permission** grants them. A seeded row defaults to admin-only,
|
||||
so the picker renders "No results found" for every regular user even though
|
||||
the connection is enabled, `model_ids` is correct, and the endpoint answers
|
||||
`/v1/models` — and `GET /api/v1/models` returns `[]` for that user while an
|
||||
admin token returns the model. Fix: set the row's `access_control` to public
|
||||
(the `access` column on the `model` row) via the admin UI (Admin Panel →
|
||||
Models → visibility) or the DB. The verification battery below must therefore
|
||||
run **as a non-admin user**, not only as an admin.
|
||||
|
||||
Two gotchas make the wiring non-obvious:
|
||||
|
||||
@@ -62,7 +81,9 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
||||
|
||||
> All three endpoints are bare-metal systemd services managed by this
|
||||
> repository. Model swaps, context sizing, and spec tuning happen there. See
|
||||
> the [LLM knowledge base](../LLM/index.md).
|
||||
> the [LLM knowledge base](../LLM/index.md). Disabling a connection here only
|
||||
> hides it from this picker — the endpoint itself keeps serving every other
|
||||
> client (the harness, Hermes, API consumers).
|
||||
|
||||
## Container introspection (DB and backend python)
|
||||
|
||||
@@ -94,18 +115,38 @@ The Ollama connection is disabled (`ollama.enable=false`).
|
||||
|
||||
## Verification battery
|
||||
|
||||
1. `GET /api/v1/models/list?limit=100` → returns EXACTLY the intended
|
||||
models. Response key is `items`, not `data`.
|
||||
2. `POST /openai/chat/completions` per model with a trivial prompt ("Reply
|
||||
1. `GET /api/models` (the picker endpoint; response key `data`) → lists
|
||||
EXACTLY the intended models, **as an admin AND as a `role='user'` token**.
|
||||
Do NOT gate on `GET /api/v1/models/list` (the Admin Panel → Models page):
|
||||
on the 0.6.x image it returns `items: []` for connection-backed models
|
||||
(workspace-list only; measured 2026-10-09 on both picker ids).
|
||||
2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
|
||||
with exactly one word: pong") → each returns content `pong`. Reasoning
|
||||
models need `max_tokens>=100` — the thinking block eats the budget.
|
||||
models need a generous budget: frank-secondary (Qwen3.5 template) returns
|
||||
empty content + `finish_reason: "length"` at `max_tokens 200` (the
|
||||
thinking block eats it) and answers `pong` at 1500.
|
||||
3. A 500 "Server Connection Error" → `podman logs open-webui` for
|
||||
`ClientConnectorError`: a network-reachability problem (host LAN IP vs
|
||||
`host.containers.internal`), not a model problem.
|
||||
|
||||
**Durability:** the model table + connections are DB-only (not git-managed);
|
||||
a fresh data volume loses them. For reproducibility, add an idempotent
|
||||
first-start seed script or document the setup in the service doc.
|
||||
a fresh data volume loses them — including the public access grant above.
|
||||
For reproducibility, add an idempotent first-start seed script or document
|
||||
the setup in the service doc.
|
||||
|
||||
## API access (auth-gated)
|
||||
|
||||
Every `/api/*` endpoint requires an authenticated session (SSO login JWT)
|
||||
or a per-user API key (`auth.enable_api_keys=true`; keys are minted per
|
||||
account under Profile → API Keys). Anonymous web visitors get the `pending`
|
||||
role (`ui.default_user_role=pending`), which cannot chat. Verified
|
||||
2026-10-07 from loopback `:3005` and over the public vhost:
|
||||
`GET /api/v1/models/list`, `GET /api/v1/models` and
|
||||
`POST /api/chat/completions` all return **401** with no key (also with a
|
||||
bogus bearer token); only `/api/config`, `/api/version` and `/health`
|
||||
answer publicly, and none of them expose the model list. An external
|
||||
program needs an account + API key — there is no keyless access.
|
||||
|
||||
|
||||
## Web search
|
||||
|
||||
@@ -149,6 +190,18 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
|
||||
into the existing local account with the same email — the pre-SSO admin
|
||||
(frank8465@protonmail.com) keeps its data. `ENABLE_OAUTH_SIGNUP=true` lets
|
||||
other `openwebui_users` create accounts on first login.
|
||||
- **First login mints role `pending`.** `ui.default_user_role=pending` (config
|
||||
DB, verified 2026-10-09), so a freshly-provisioned `openwebui_users` account
|
||||
completes SSO and then cannot chat (picker empty) until its role is flipped:
|
||||
`UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` —
|
||||
run with host `python3` over ssh against
|
||||
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
|
||||
Dankclaw; table is `user`, singular; no `active` column). No per-user model
|
||||
grant is needed: both picker models (`frank-zealot`, `frank-dragoon`) carry
|
||||
the global `access_grant` row `(model, <id>, user, *, read)`, so any
|
||||
`role='user'` account sees them. Flip on FIRST-LOGIN SIGHTING — the row does
|
||||
not exist until the account signs in (2026-10-09: five provisioned accounts
|
||||
had no row).
|
||||
- **IdP reachability:** the container is on its own bridge (10.89.3.0/24) and
|
||||
cannot reach `authentik-server:9000` by name. It reaches the IdP over the
|
||||
public URL `https://auth.franksdank.cloud` — Caddy publishes `0.0.0.0:443`
|
||||
@@ -218,7 +271,7 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
|
||||
- The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
|
||||
- The default admin account is created on first login; secure it immediately.
|
||||
- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
|
||||
- The on-host LLM services (frank-large `:7770`, frank-secondary `:7771`) are reached via `host.containers.internal`, NOT the host LAN IP: the container's netavark bridge cannot hairpin to its own host's non-published ports via `192.168.0.143` (connection refused), and the bridge gateway (`10.89.3.1`) lives in the rootless netns while llama-server runs in the host netns, so it refuses too. `host.containers.internal` (169.254.1.2) resolves to the host on the bridge and works — verified end-to-end for both endpoints. The Dankserver backend (frank-medium) is a different host, reached via its LAN IP `192.168.0.235:7770`.
|
||||
- The on-host LLM services (Dankclaw `:7770`, `:7771`) are reached via `host.containers.internal`, NOT the host LAN IP: the container's netavark bridge cannot hairpin to its own host's non-published ports via `192.168.0.143` (connection refused), and the bridge gateway (`10.89.3.1`) lives in the rootless netns while llama-server runs in the host netns, so it refuses too. `host.containers.internal` (169.254.1.2) resolves to the host on the bridge and works — verified end-to-end for both endpoints. The Dankserver backend is a different host, reached via its LAN IP `192.168.0.235:7770`.
|
||||
|
||||
## History
|
||||
|
||||
@@ -230,6 +283,52 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
|
||||
|
||||
- **2026-09-24 — flipped to SSO-only.** The OIDC round-trip verified in a real browser (login view renders no password form, only "Continue with authentik", and the flow lands on a signed-in session), so the local username/password path was retired: `ENABLE_LOGIN_FORM=false` + `ENABLE_PASSWORD_AUTH=false` in the quadlet and `ui.enable_login_form` set to `false` in the config DB (the DB wins over env, so both were changed together). `qa-e2e-admin` was demoted out of the `authentik Admins` group (it keeps `openwebui_users` / `homepage-admins` for app access).
|
||||
|
||||
- **2026-10-07 — picker narrowed to `frank-secondary` only.** Per Frank: the only model accessible in OpenWebUI is `frank-secondary`. Rebuilt the `model` table to a single `frank-secondary` row (the old rows `frank-large`/`frank-primary`/`frank-small` are deleted), set `openai.api_configs` indices 0 (frank-large) and 1 (frank-primary) to `enable: false`, and retargeted index 2's `model_ids` from the stale `frank-small` label to `frank-secondary` — llama.cpp serves by alias, so the old label addressed a model name the server does not know. Pre-change snapshot: `/home/dank/Containers/open-webui/data/webui.db.bak-20261007` (+ `-wal`/`-shm`). Verified: `/api/v1/models/list` returns exactly `["frank-secondary"]`, chat completion answers `pong` (needs `max_tokens≥1500` — the Qwen3.5 thinking block consumes a 200-token budget), and anonymous API probes are all 401 (see [API access](#api-access-auth-gated)).
|
||||
|
||||
- **2026-10-09 — re-keyed to the Protoss ids (wire rename).** The endpoints
|
||||
began serving `frank-zealot` / `frank-dragoon` / `frank-archon`, so three
|
||||
id-keyed objects had to move in the same window as the alias flip:
|
||||
`model.id`+`name` (`frank-secondary` → `frank-zealot`, `params`
|
||||
`{"context":{"length":200704}}` and the `meta.description` kept),
|
||||
`access_grant` (`resource_type='model'`, `resource_id`, principal
|
||||
`user`/`*`, permission `read`), and `openai.api_configs` per-backend
|
||||
`model_ids` (index 2 = `:7771` is the enabled family backend; indices 0/1
|
||||
= `:7770` and Dankserver `:7770` stay `enable: false`). Snapshot first:
|
||||
`/home/dank/Containers/open-webui/data/webui.db.bak-20261009`. Apply from
|
||||
inside the container (`podman exec -i open-webui python3 -`, sqlite3; the
|
||||
image has no sqlite3 binary) then `systemctl --user restart open-webui.service`
|
||||
(it is a rootless quadlet) to drop the cached model list. Verify in a real
|
||||
browser as the family QA account: the welcome screen selects
|
||||
`frank-zealot` and answers. **A stale id fails silently** — the picker
|
||||
simply lists nothing, because OpenWebUI intersects the backend's
|
||||
`/v1/models` with `model_ids`.
|
||||
|
||||
Law: **the model id is a foreign key in three tables.** Any future alias
|
||||
change on an endpoint OpenWebUI fronts must ship with this re-key, in the
|
||||
same deploy.
|
||||
|
||||
- **2026-10-09 — composer is `contenteditable`, not `<textarea>`.** A
|
||||
Playwright probe that waits for `textarea` after SSO times out on the
|
||||
welcome screen: OpenWebUI 0.6 renders the composer as a contenteditable
|
||||
div. The stable anchor is the model button (`getByRole("button", {
|
||||
name: /selected model/i })`) and the welcome text "How can I help you
|
||||
today?"; that snapshot is also the cheapest proof the picker is populated.
|
||||
|
||||
- **2026-10-09 — `frank-dragoon` added to the picker.** Per Frank: expose the
|
||||
halogen endpoint (service `frank-large`, `host.containers.internal:7770`)
|
||||
beside `frank-zealot`. Three id-keyed writes (the trio the 10-07 rebuild
|
||||
proved): `openai.api_configs` idx 0 `enable: true` (its `model_ids` already
|
||||
carried `frank-dragoon` from the wire rename), a `model` row (`id`/`name`
|
||||
`frank-dragoon`, `params.context.length` 262144 = the per-slot
|
||||
`HALOGEN_CTX`, description of the v2 abliterated payload), and the global
|
||||
`access_grant` `(model, frank-dragoon, user, *, read)`. Snapshot
|
||||
`webui.db.bak-20261009b`, apply with host `python3` over ssh, then
|
||||
`systemctl --user restart open-webui.service`. Verified: `/api/models`
|
||||
lists `frank-dragoon` as admin AND as the QA `role='user'` account, and
|
||||
`POST /api/chat/completions` through the app answers `pong` at
|
||||
`max_tokens 1500`. Halogen's `/v1/models` is OpenAI-shaped, so discovery
|
||||
would have worked too — `model_ids` stays authoritative.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Dankclaw Stack]]
|
||||
|
||||
@@ -449,8 +449,18 @@ access is decided entirely by GROUP MEMBERSHIP consumed downstream —
|
||||
`u.app_entitlements(app)`.
|
||||
- `jellyfin_users` / `jellyfin_admins` → Jellyfin via the LDAP outpost
|
||||
(filter `(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`).
|
||||
- `Non-Admin Users` = the plain-user marker group (precedents: alex,
|
||||
qa-e2e).
|
||||
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
|
||||
every human account).
|
||||
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
|
||||
debo, freya — kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
|
||||
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
|
||||
`<user>@franksdank.cloud` + display name +
|
||||
`is_active=True` + the password set via `set_password`, membership in
|
||||
`Non-Admin Users` + `jellyfin_users` + `openwebui_users`. `alex` was
|
||||
recreated from scratch on 2026-10-08 into this shape (the old row had an
|
||||
empty email and no `openwebui_users`; audit first — the row carried no MFA
|
||||
device, token, entitlement, or proxy session, and events survive a user
|
||||
delete because `Event.user` is a denormalized JSON field, not a FK).
|
||||
- EXCLUDE from a user-level account: `homepage-admins`, `jellyfin_admins`,
|
||||
`openwebui_admins`, `authentik Admins`, `authentik Read-only`.
|
||||
- Other user-category vhosts (SearXNG, Gitea, Calibre-Web, RomM, Copyparty,
|
||||
|
||||
@@ -36,6 +36,211 @@ deliberately unprobeable) via the `wip` value (`"open"` / `"closed"`).
|
||||
Tautulli (Plex statistics, `192.168.0.235:8181`) joined the admin page's
|
||||
Media Library Apps section.
|
||||
|
||||
**Amber CRT theme + API docs (2026-10-07):** the palette moved from green
|
||||
phosphor to amber CRT-AMBIOS (`--fg #ffb000` on `--bg #0a0600`), and the font
|
||||
stack now leads with **VT323** loaded from the Google Fonts CDN (the deploy
|
||||
also ships `shared/homepage/img/` beside the page, so `/img/*` serves the docs
|
||||
screenshots — the post-auth listener serves files only through the `@data`
|
||||
path allowlist, so new top-level assets must be added there (the `@home`
|
||||
catch-all answers anything else with `index.html`, i.e. a PNG arrives as
|
||||
`200 text/html` and renders broken; pinned in `homepage_caddy.bats` — this
|
||||
exact bug bit the screenshots on 2026-10-08). The docs page gained a full
|
||||
(`/docs#api-access`): per-user OpenWebUI API keys (User menu → Settings →
|
||||
Account → API keys), the base URL `https://ai.franksdank.cloud/api/v1`, and
|
||||
copy-paste configs for **Opencode** (`~/.config/opencode/opencode.json`) and
|
||||
**Pi** (`~/.pi/agent/models.json`) pointing at `frank-secondary`. Shipped
|
||||
together with the OpenWebUI side: `features.api_keys=true` for regular users
|
||||
and the restored `frank-secondary` model row + public `read` access grant
|
||||
without which the picker showed "No results found" for every family account
|
||||
(see [[OpenWebUI]]).
|
||||
|
||||
**Capability equivalence notes (2026-10-08):** each fleet card (and the
|
||||
`/docs#models` section) carries a `details.equiv` line — frank-large
|
||||
"equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier
|
||||
intelligence)", frank-primary "equivalent to Claude Opus 4.8 / OpenAI-Terra
|
||||
(same era)", frank-secondary "equivalent to last year's frontier level
|
||||
(2025)". Rendered by `descHTML()` as `.desc-equiv` (head color, italic);
|
||||
pinned in `homepage_config.bats`.
|
||||
|
||||
**Protoss rename + Model Information component (2026-10-08):** the model
|
||||
cards became the StarCraft trio — `frank-large` → **frank-dragoon**
|
||||
(iGPU), `frank-primary` → **frank-archon** (Dankserver), `frank-secondary`
|
||||
→ **frank-zealot** (3090). The endpoint/API ids did NOT change in this wave; only the
|
||||
family-facing names. The three cards fused into one **Model Information**
|
||||
component (`app.js` `renderModelInfo()`, `.model-info`/`.mi-*` styles):
|
||||
Protoss unit art (`/img/unit-<unit>.jpg`, hex-clipped), per-unit accent
|
||||
colours (Dragoon gold, Archon cyan, Zealot blue), smaller type (15px vs
|
||||
19px body), and clickable unit-voice lines that play the original
|
||||
StarCraft `.ogg` quotations from `/sfx/` (fetched from
|
||||
starcraft.fandom.com via its MediaWiki API; the quote→file mapping lives
|
||||
in the `model.sfx`/`model.voice.sfx` fields). New `services.json`
|
||||
per-model fields: `unit`, `tagline`, `sfx[]`, `voice`, `supervision`
|
||||
(dragoon/archon autonomous-safe, zealot tight-supervision), and `warn`
|
||||
(uncensored legal note, rendered as a hover tooltip, not wall text).
|
||||
A compaction-model recommendation for frank-zealot rides in its `use`
|
||||
text. The Caddy `@data` allowlist grew `/sfx/*`; `deploy_homepage` copies
|
||||
`sfx/`. A `wip: "down"` badge state ("WIP: DOWN", red) was added; Seerr,
|
||||
Calibre-Web, RomM, and Jellyfin are WIP: DOWN with their Open links
|
||||
removed. Card notes added: Copyparty holds every other media app's
|
||||
content (including the WIP ones); Gitea and Plex need logins separate
|
||||
from the dankcloud SSO; SearXNG is obfuscated, not anonymous (upstream
|
||||
API keys tie to Frank). The user docs `/docs#models` section carries the
|
||||
same renames, api-id mapping, supervision and compaction notes.
|
||||
|
||||
**Icons, card polish, admin identity gate (2026-10-08, later wave):** every
|
||||
app card now carries hex-clipped art beside its name (`services.json` `art`
|
||||
field, files under `img/apps/`): 19 brand marks tinted amber from the
|
||||
Simple Icons CDN plus 19 hand-improvised monogram SVGs (amber hex + VT323
|
||||
initials) for apps with no mark — Copyparty, Marreta, Hermes (+dashboard),
|
||||
node-exporter, Gluetun, OpenWebUI, Calibre-Web, RomM, Seerr, Loki, Alloy,
|
||||
Alertmanager, Valkey, Prowlarr, Readarr, Lidarr, Tautulli, ComfyUI. Model
|
||||
cards render in the order zealot → dragoon → archon (registry order is the
|
||||
contract; `homepage_config.bats` pins it). Dragoon and zealot share one
|
||||
common uncensored legal warning (adult content, harmful/hateful/illegal
|
||||
descriptions, "sensitive offensive cybersecurity topics", severe legal
|
||||
risk, "don't ruin this server for everybody"), rendered as the hover
|
||||
label; archon's ask-Frank rule is a bordered red `.mi-perm` callout.
|
||||
SearXNG's obfuscation-not-anonymity note moved out of its description into
|
||||
a card-level `warn` (`.svc-warn` hover label) and the same hover treatment
|
||||
in the docs. Gitea's note now documents self-service signup — no need to
|
||||
ask Frank. Footer: docs / admin / account settings are bordered glowing
|
||||
`.foot-btn` buttons; **the admin button is identity-gated** — Caddy's
|
||||
`/whoami` echoes the outpost-injected `X-Authentik-Username`/`X-Authentik-Groups`
|
||||
headers (the outpost strips client-supplied copies, so the echo is
|
||||
spoof-proof), and `app.js` shows the button only to `homepage-admins`
|
||||
members. The `/admin*` 403 stays the real gate; the e2e specs pin both
|
||||
sides (standard user: hidden + 403; admin: visible). Docs were swept for
|
||||
naming consistency: Protoss names primary everywhere, bracketed api ids
|
||||
at the model headings, and a note that a future switchover renames the
|
||||
ids themselves (OpenWebUI picker + raw API + models.yml all move together
|
||||
— not done yet).
|
||||
|
||||
**Icons v2, clickable tiles, header nav (2026-10-08, third wave):** five
|
||||
icons replaced on request — Copyparty gets an improvised amber folder,
|
||||
Marreta a crumbling wall (bricks + red crack), Calibre-Web a book,
|
||||
OpenWebUI its official logo (`openwebui.com/logo.png`), Seerr the official
|
||||
`seerr.dev/os_logo_filled.svg`. Tiles with an open link are now clickable
|
||||
whole (`.panel.clickable`, hover lift + glow; clicks on inner links pass
|
||||
through). Any panel whose live state is DOWN gets a red gradient wash +
|
||||
slow pulsing glow (`.panel.down`, and `.mi-card:has(.badge.down)` for
|
||||
model cards). docs/admin/account-settings moved from the footer to the
|
||||
header (`nav.top-links`). The zealot compaction advice moved out of the
|
||||
Use paragraph into a bordered glowing `model.compaction` callout
|
||||
(`.mi-comp`), pinned by the component seam test.
|
||||
|
||||
**Protoss theme, brand-color icons, portraits (2026-10-08, fourth wave):**
|
||||
the full-page angular treatment: chamfered 14 px-cut panels and model
|
||||
cards, hex-lattice + amber-glow backdrop (`body::after`), scanline
|
||||
overlay (`body::before`), diamond-glyph section titles with gradient
|
||||
underlines, the gradient MODEL INFORMATION gateway, a blinking cursor
|
||||
beside the title, amber scrollbars, cyan text selection, cyan hover glow
|
||||
on clickable tiles, and a red pulsing glow on down tiles/cards (the
|
||||
`data-state="down"` attribute `applyStatus` now sets). Icons recut to
|
||||
official brand colors (Simple Icons CDN): SearXNG, Gitea, Plex, Jellyfin,
|
||||
Sonarr, Radarr, Lidarr, Readarr, qBittorrent, Prowlarr, Seerr, OpenWebUI,
|
||||
Calibre-Web (amber glyph on amber tile). Handcrafted monograms carry
|
||||
per-app colors: Copyparty gold, Marreta brick, Grafana orange, Loki
|
||||
indigo, Neo4j network, Alloy violet, Promtail pink, Alertmanager red,
|
||||
Prometheus orange, Node Exporter teal, cAdvisor sky, Guake yellow,
|
||||
Authentik blue, Seer green radar, RomM a purple GameBoy. New per-service
|
||||
fields: `art_bg` (art backdrop — OpenWebUI's dark logo needs `#f5f5f5`),
|
||||
`warn` on OpenWebUI (searches ride the Seerr→SearXNG path),
|
||||
`model.compaction` (the amber "set this as compaction model" callout
|
||||
frank-zealot carries). frank-dragoon's card portrait is the SC2 unit
|
||||
portrait (fandom wiki `Dragoon_portrait.jpg`, webp→jpeg). The whole
|
||||
effect layer is stripped under `data-bare="1"` (the e2e bare page).
|
||||
Unit: `tests/homepage_config.bats` (referenced art files exist, model
|
||||
blocks well-formed, deployer ships the `img/` tree).
|
||||
|
||||
**Docs reorder + model source links (2026-10-08, fifth wave):** the docs
|
||||
page section order is now getting-in → troubleshooting → media → chat →
|
||||
models → api-access (non-AI first, AI last). The *arr stack (Prowlarr,
|
||||
Sonarr, Radarr, Lidarr, Readarr, qBittorrent) is out of the family docs
|
||||
entirely — admin tooling; Seerr is presented as the front door for
|
||||
requests. Each fleet model gained a Hugging Face source link: card field
|
||||
`model.hf` rendered as the cyan `model on huggingface ↗` line under the
|
||||
role, and the docs `Model:` bullets link the checkpoint names (zealot →
|
||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive, dragoon →
|
||||
Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, archon →
|
||||
Qwen/Qwen3.8-Flash-Next-FP8).
|
||||
|
||||
**Navy Protoss console, maximal cornering, font swap (2026-10-08, sixth
|
||||
wave):** the amber CRT palette is retired. The page is now deep space
|
||||
navy (`#050d16`) with warp-blue/psychic-cyan accents (`--proto-blue
|
||||
#3fa9ff`, `--proto-cyan #7fe7ff`) — StarCraft Protoss console colours.
|
||||
Cornering went maximal: every panel, model card, badge, button, callout
|
||||
and the MODEL INFORMATION gateway are octagon-cut (all four corners
|
||||
chamfered, 18–24 px), and panels/model cards gained a second inset
|
||||
octagonal frame line (`::before`) — a double-bevel gateway look. Section
|
||||
titles are chamfered plates with the diamond glyph inside the plate.
|
||||
Fonts: body is **Share Tech Mono** (legible retro terminal), headings
|
||||
are **Rajdhani 600** (the StarCraft UI face) — VT323 is retired; all
|
||||
three pages load the pair from Google Fonts. Four tiles that had
|
||||
reverted to initials got pictograms: Calibre-Web an open book (orange),
|
||||
RomM a purple GameBoy, Copyparty a blue folder with upload arrow,
|
||||
Marreta a crumbling brick wall with a red crack (`img/apps/app-*.svg`).
|
||||
frank-dragoon's card accent is phase-crystal teal (`#5fe0d0`), matching
|
||||
the new palette. Bare mode strips the frames, octagon cuts and fonts.
|
||||
|
||||
**Simplified console (2026-10-08, seventh wave):** "too busy — clean up
|
||||
and simplify/solidify." The sixth-wave effect stack was stripped to
|
||||
essentials. Removed: the scanline overlay (`body::before`), the hex
|
||||
lattice + twin radial backdrop (one quiet pylon glow above the horizon
|
||||
remains), the panel/model-card inner frame lines, all ambient text
|
||||
glows (body, title, model names, callouts), the blinking title cursor,
|
||||
the gradient section-title plates (headings are plain Rajdhani with a
|
||||
hollow diamond glyph + steel underline), the gradient `border-image`
|
||||
header rule (solid 2px warp-blue), the pulsing down-state animation
|
||||
(down tiles and cards now show a STATIC red border + tint), and the
|
||||
button/hover glows. Kept: octagon-cut panels (softer 14 px), unit-color
|
||||
model cards, the pictograms, and the fonts. Motion is reserved for
|
||||
state — the only animation left is the speaking-voice pulse. The result
|
||||
is flat navy panels on crisp 1px borders: same Protoss direction, half
|
||||
the pixels.
|
||||
|
||||
**Model ids official (2026-10-08):** the card ids `frank-zealot`,
|
||||
`frank-dragoon`, `frank-archon` are now the harness's official model ids
|
||||
in `models.yml`/`config.yml` (see [[OMP Harness]]); the
|
||||
systemd services keep the `frank-primary`/`frank-large`/
|
||||
`frank-secondary` names. Homepage, docs page and harness now agree on
|
||||
one name per model.
|
||||
|
||||
**Api ids renamed on the wire (2026-10-09):** the promise in
|
||||
`docs.html` is kept — the served model ids are now `frank-zealot`,
|
||||
`frank-dragoon`, `frank-archon`, so the docs page no longer carries
|
||||
`api id:` annotations, and the curl/opencode/Pi examples send
|
||||
`frank-zealot`. The systemd services keep the old names. `docs.html`
|
||||
names the fleet trio once each; `homepage_config.bats` pins the card
|
||||
ids, which did not change.
|
||||
|
||||
**Console polish (2026-10-08, eighth wave):** five fixes — octagon
|
||||
clip-paths replaced with `border-radius` (the clipped corners read as
|
||||
gaps in the 1px border); section-title diamonds moved from absolute
|
||||
positioning to a fixed-width inline glyph (aligned, wrap-safe); closed
|
||||
WIP items grey out (dim type, desaturated art, muted border via
|
||||
`:has(.badge.wip-closed)`); the Dragoon tagline is one spliced track
|
||||
(`sfx/dragoon-tagline.ogg` — loudnorm-matched, silence-trimmed, 350 ms
|
||||
beat; the two SC2 source oggs are trashed); privacy notes became
|
||||
**`<details>` disclosure chips** ("⚠ privacy note ▸") pinned to the
|
||||
bottom of each box (`margin-top:auto` in a flex-column panel/card) —
|
||||
collapsed by default, tap-to-expand, the touch-safe replacement for the
|
||||
unreachable hover tooltips. The panel click-to-open handler ignores
|
||||
`.svc-warn` taps so the chip never navigates. Same chips on the docs
|
||||
page (privacy + uncensored-model warnings). The API-access section was
|
||||
reblessed against the live Open WebUI 0.6 UI: steps now match the real
|
||||
Show/copy/rotate flow and `img/openwebui-api-keys.png` is a fresh
|
||||
screenshot (old shot trashed). The troubleshooting DNS bullet dropped
|
||||
its Pi-hole specifics — family members off the LAN cannot use
|
||||
house-DNS advice; the bullet now just says the names resolve on the
|
||||
home network.
|
||||
|
||||
**Archon card runs the fleet's own model (2026-10-08):** the
|
||||
`frank-archon` homepage card and docs Model bullet now say the payload
|
||||
is our in-house GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next and link
|
||||
our HF publication (`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`)
|
||||
in place of the FP8-era base-model link; capacity line says three
|
||||
submissions (`--max-num-seqs 3`).
|
||||
|
||||
## Design decision — 2026-08-22
|
||||
|
||||
The live up/down check runs server-side. A checker on Dankclaw probes each backend and writes a same-origin `status.json`. The page fetches only that file and renders the badges. The browser never calls a backend directly. The checker runs as a systemd timer every 3 minutes.
|
||||
@@ -111,7 +316,7 @@ flowchart TD
|
||||
## Service classification
|
||||
|
||||
**User services** (root page), in section order:
|
||||
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-large, frank-medium, frank-secondary — status + docs link, no Open link).
|
||||
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-large, frank-primary, frank-secondary — status + docs link, no Open link).
|
||||
- **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
|
||||
- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
|
||||
|
||||
@@ -131,7 +336,10 @@ shared/homepage/
|
||||
├── style.css # full theme; a toggle strips to bare HTML
|
||||
├── app.js # loads status.json once, renders the panels
|
||||
├── services.json # service config (user + admin)
|
||||
└── fonts/ # Terminus (woff2) for @font-face
|
||||
├── img/ # served at /img: docs screenshots, unit portraits
|
||||
│ ├── apps/ # (unit-*.jpg) and app card icons (apps/*.svg)
|
||||
├── sfx/ # StarCraft unit voice .ogg files served at /sfx
|
||||
└── fonts/ # optional self-hosted fonts (deployed with the page; VT323 comes from the Google Fonts CDN, this dir is absent)
|
||||
```
|
||||
|
||||
`status.json` is generated at runtime in `/srv/homepage/`. It is not committed.
|
||||
|
||||
@@ -0,0 +1,260 @@
|
||||
# Quantization Production — making our own quants, per card and runtime
|
||||
|
||||
**Research page (2026-10-08).** Question: how do WE produce quantizations
|
||||
for the RTX 3090, the R9700 (gfx1201), and Strix Halo (gfx1151); what is
|
||||
native to each runtime; and the state of the named prior-art families
|
||||
(REAP, Heretic, abliteration, HauhauCS K_P "Perfect"). Marked facts:
|
||||
**[repo-verified]** = evidenced by a file in this repo; **[verified]** =
|
||||
checked first-hand against the cited primary source today; the rest are
|
||||
survey claims to re-verify before acting.
|
||||
|
||||
## Runtime-native quant matrix
|
||||
|
||||
| Card | Runtime | Container | Native weight encodings | Production tooling we own |
|
||||
|---|---|---|---|---|
|
||||
| RTX 3090 (sm_86) | llama.cpp CUDA | GGUF | K-quants (Q4_K_M…Q8_0), I-quants (IQ1_S…IQ4_XS), ML, f16/bf16; any tensor type per-tensor | `llama-quantize` + `ggml-imatrix` (built by `scripts/build-llama-rdna4.sh` targets) **[repo-verified]** |
|
||||
| R9700 (gfx1201) | radiance (StillDeadcode) | `.rad` | per-tensor encodings from a recipe: GPTQ w4 (`w4nl64a8h`), bf16, int8, fp8_e4m3, u2, MXFP4/MXFP6 (vLLM-radiance era) | full pipeline: `rad-convert` + calibration tap + `derive-protected-experts.py` + `kld-gate-swift15.sh` **[repo-verified]** — CPU host-backend, the card is irrelevant to production |
|
||||
| R9700 (gfx1201) | vLLM (tcclaviger/R4D) | safetensors | native block-FP8, MXFP4/DSFP4, GPTQ/AWQ, compressed-tensors | GPTQModel/AutoRound are hardware-agnostic torch (CPU-scale); AMD decode kernels are the gate **[survey]** |
|
||||
| Strix Halo (gfx1151) | halogen-flash-server (peonist, **closed**) | `.hgn` | vendor-defined (w4b era, v2 checkpoint+ngram+vision files) | **NONE public** — no `.hgn` quantizer exists; every `.hgn` on the fleet was made by peonist/Quat3rnion **[repo-verified gap]** |
|
||||
| Strix Halo (gfx1151) | llama.cpp / gufo / Kairic / ROCmFPX forks | GGUF | K/I quants (gufo UD-* = imatrix mixed profiles), custom types: `Q4_0_ROCMFP4_STRIX`, `Q8_0_ROCMFPX` (type 103), Kairic IU4 | we built the RUNTIMES (`scripts/build-kairic-edge-gfx1151.sh`, ROCmFPX pin `36e9acd`), never the custom-type quantizers — those files came from agentionai/kingjones **[repo-verified]** |
|
||||
|
||||
- **The lock is asymmetric: llama.cpp on the 3090 is a CHOICE.** radiance
|
||||
demands `.rad`, halogen demands `.hgn` (the v2 checkpoint loads in no
|
||||
other engine); sm_86 decodes GGUF, GPTQ/AWQ-Marlin, INT8, weight-only
|
||||
FP8, and EXL3 natively — see the 3090 menu under Per-card playbooks.
|
||||
|
||||
Laws that steer everything here:
|
||||
- **Production is hardware-agnostic; decode is not.** GPTQ/imatrix/ablation
|
||||
surgery runs in torch/CPU anywhere. The card only picks which ENCODED
|
||||
FORM it can DECODE — hence the one-table-one-encoding law, the GGUF
|
||||
custom-type histogram law ([[GGUF Compatibility]]), and the fact that
|
||||
our w4 convert ran 5 h 50 m on the host backend while the GPUs served
|
||||
nobody (`scripts/convert-swift15-w4dn8.sh`) **[repo-verified]**.
|
||||
- **A custom tensor type locks the file to its fork** (ROCmFPX type 103
|
||||
precedent). K_P quants deliberately avoid that trap (below).
|
||||
|
||||
## The named families
|
||||
|
||||
### REAP — Router-weighted Expert Activation Pruning (Cerebras, 2025)
|
||||
- arXiv 2510.13999; `github.com/CerebrasResearch/reap` **[verified]**.
|
||||
- One-shot MoE surgery: score every expert by router gate-values ×
|
||||
activation energy, prune the low-utility tail, renumber+export a
|
||||
SMALLER MoE checkpoint (safetensors; GGUF by conversion). Published:
|
||||
GLM-4.5-Air −25 % (82B→~A12B class), GLM-4.7-REAP-268B-A32B-FP8;
|
||||
near-lossless at 25–50 % compression on generative benchmarks, and
|
||||
pruning BEATS expert-merging there **[verified from paper/README]**.
|
||||
- Fleet relevance: it is the **inverse of our protected-experts law**.
|
||||
`derive-protected-experts.py` ranks experts by calibration-gram energy
|
||||
share and keeps the hottest 24/layer at bf16; REAP's saliency is the
|
||||
same statistic and deletes the coldest. REAP the Flash-Next MoE (all
|
||||
49,152 experts w4, 512/layer) by 25–50 % → resident set ~58–87 GiB →
|
||||
the expert pool fits with room for KV/cache, killing the disk tier.
|
||||
A REAPed checkpoint re-enters our normal pipeline: stage → calibrate →
|
||||
rad-convert → KLD gate. Hardware-agnostic; runs on the host.
|
||||
|
||||
### Heretic (`p-e-w/heretic`) — automated abliteration + layer pruning
|
||||
- `github.com/p-e-w/heretic`, AGPLv3 **[verified]**. NOT a quantizer.
|
||||
Fully-automatic censorship removal: directional ablation (refusal
|
||||
direction from PCA of harmful/harmless residuals), per-layer ablation
|
||||
weights, optional LAYER DROPPING (model shrinks), auto-tuned by
|
||||
co-minimizing refusals and KL divergence from the original. Runs on
|
||||
our 3090 (torch, ~24–48 GB working set for 9–27B classes; long batch
|
||||
probes) **[verified README; runtime cost survey]**.
|
||||
- Output is a normal HF checkpoint (fp16/bf16) → feeds GGUF, safetensors
|
||||
(vLLM), and `.rad` conversion. Order law: **surgery → calibrate →
|
||||
quantize**; calibrate GPTQ/imatrix on the SURGICAL output, and keep
|
||||
refusal-shaped tokens out of the calibration corpus.
|
||||
- Fleet note: our abliterated bases (Quat3rnion v2, HauhauCS
|
||||
Aggressive/Balanced, ukisai Swift1.5) are this family's output; the
|
||||
Quat3rnion repo lore "the abliteration workflow appends, compaction
|
||||
drops dead padding" is exactly Heretic-style layer-editing leaving
|
||||
dead blocks (89.8 → 66.7 GiB r2) **[repo-verified lore]**.
|
||||
|
||||
### Ablation / abliteration (directional ablation)
|
||||
- Arditi et al. 2024, arXiv 2406.11717: refusal is mediated by a single
|
||||
direction; orthogonalizing weights against it removes refusals with a
|
||||
rank-1 edit. Open implementations: `p-e-w/heretic` (automated, above),
|
||||
Artio/abliteration-toolkits, `EleutherAI/repeng`-style rep-overlap
|
||||
variants **[verified (tool existence: heretic; rest survey)]**.
|
||||
- Hardware-agnostic fp-editing. On a MoE the edit touches attention/norm
|
||||
weights, so it composes with any expert quantization.
|
||||
- Fleet status: frank-large runs an abliterated base **by Frank's choice**
|
||||
(AGENTS: censored-base refusals were the accepted trade of the 27B era);
|
||||
frank-archon's ukisai Swift1.5 is an abliterated finetune of Qwen3.8.
|
||||
|
||||
### HauhauCS QX_K_P "Perfect" quants
|
||||
- Our frank-secondary publisher. Method, from the publisher's own cards
|
||||
**[verified]**: (1) abliterate + finetune ("Aggressive"/"Balanced"),
|
||||
(2) quantize with **imatrix calibration on the ABLITERATED weights**,
|
||||
(3) K_P = a **per-model, per-tensor mixed-precision profile** —
|
||||
"model-specific analysis to selectively preserve quality where it
|
||||
matters most", e.g. Q8_K_P = a census of Q8_K + F32 + F16 tensors
|
||||
(FriendliAI's safetensor census of a Q8_K_P shows exactly that
|
||||
399/353/99 tensor split). Result: +1–2 quality rungs for +5–15 %
|
||||
size. **It is a standard GGUF** — llama-quantize with per-tensor type
|
||||
overrides (`--tensor-type`), imatrix on; no fork, no custom types
|
||||
(LM Studio just fails to NAME the mixed file, shows "?").
|
||||
- Reproducible with stock tooling: `ggml-imatrix` on the target model +
|
||||
`llama-quantize --tensor-type <pattern>=<type>` per the model's
|
||||
sensitivity census. Both binaries already build from
|
||||
`scripts/build-llama-rdna4.sh` **[repo-verified]**.
|
||||
- Our own ladder precedent: the fleet's UD-Q8_K_XL (unsloth "Dynamic" =
|
||||
same family: imatrix + importance-tiered per-tensor types) and
|
||||
agentionai FAST imatrix ROCmFP4-FAST (we ran the imatrix step
|
||||
ourselves on the v2 joined-PLE file — the one self-made quant in repo
|
||||
history) **[repo-verified]**.
|
||||
|
||||
## Per-card playbooks
|
||||
|
||||
### RTX 3090 — the unlocked card (frank-secondary runs llama.cpp BY CHOICE)
|
||||
The 3090 is the only fleet device with no format lock: radiance needs
|
||||
`.rad`, halogen needs `.hgn`, and the v2 checkpoint loads in NO other
|
||||
engine. sm_86 + CUDA is the industry's default target, so the runtime
|
||||
menu is open and every self-produced artifact format has a native
|
||||
decoder here. This makes the 3090 the natural **quant R&D bench**:
|
||||
produce on the host, DECODE AND SCORE on the 3090 against the fleet's
|
||||
own KLD/perplexity gates.
|
||||
|
||||
Runtime menu on sm_86 (verified against each project's support matrix):
|
||||
|
||||
| Runtime | Native quant on Ampere | Notes |
|
||||
|---|---|---|
|
||||
| llama.cpp CUDA (deployed) | K/I/ML quants, f16 | imatrix + `--tensor-type` K_P ladder |
|
||||
| vLLM CUDA | GPTQ INT4 + AWQ via **Marlin** (sm_80/86 native), INT8 W8A8 SmoothQuant, **FP8 as W8A16 weight-only via Marlin** (FP8 W8A8 compute is Ada/Hopper-only) | full OpenAI server, batching |
|
||||
| SGLang | GPTQ/AWQ (vLLM kernel lineage); Ampere FP8-via-Marlin tracked upstream | CUDA-first engine |
|
||||
| ExLlamaV3 | **EXL3** trellis (QTIP) 2–8 bpw; runs on Ampere incl. the 3090 | the quantizer (`convert.py`) runs on this same card |
|
||||
| TensorRT-LLM | INT4 AWQ/GPTQ (FP16 acts), INT8 SmoothQuant; FP8 and INT4-AWQ-with-FP8-acts need SM≥89, NVFP4 is Blackwell | engine-build workflow |
|
||||
|
||||
Hardware ceilings that no software unlocks on this card: no FP8 matmul
|
||||
hardware (sm_86 < 89), no FP4 (Blackwell). Ampere's fast lanes are
|
||||
**INT4 (Marlin/AWQ/GPTQ), INT8 (SmoothQuant), weight-only FP8, f16**.
|
||||
|
||||
Playbook A — GGUF ladder (status quo, llama.cpp kept):
|
||||
1. Surgery on host/3090: abliterate/prune (Heretic) — torch.
|
||||
2. `ggml-imatrix` (CUDA build) over a calibration corpus (our
|
||||
`/home/dank/corpora` style set; ≥1M tokens; the fleet's KLD corpus
|
||||
is a reasonable seed).
|
||||
3. `llama-quantize` F16 → target rung; K_P-style `--tensor-type`
|
||||
overrides for attention/KV/first-layer upgrades; `--use-repack` for
|
||||
IQ rungs.
|
||||
4. Validate: perplexity + KLD vs BF16 + the K-quant rank checks.
|
||||
|
||||
Playbook B — safetensors ladder (the unlocked option): GPTQModel /
|
||||
AutoRound / AWQ run CPU-side, emit GPTQ/AWQ/FP8 checkpoints that vLLM
|
||||
or SGLang serves on the 3090 with Marlin kernels — same KLD-gate
|
||||
discipline as the radiance pipeline. A REAPed + abliterated Flash-Next
|
||||
cut to GPTQ-w4 for the 3090 is a bench experiment that needs no new
|
||||
hardware; EXL3 (`convert.py` on the 3090, exllama serve) is the
|
||||
quality-per-bit extreme of the same menu.
|
||||
Caveats before any swap: the serving contract (OpenAI server,
|
||||
`reasoning_content`, vision mmproj, compaction-hub window
|
||||
200,704 ×2 slots) must be met by the challenger runtime, and the
|
||||
24 GB card runs ONE server — benching follows the stop-and-mask law.
|
||||
|
||||
#### Worked example — the frank-archon w4dn8 recipe ported to the 3090
|
||||
(audit 2026-10-08, target `Qwen3.6-35B-A3B`)
|
||||
**The methodology ports; the encoding must be translated.** llama.cpp
|
||||
decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
|
||||
imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
|
||||
BF16 on host CPU), IQ4-class experts, hot-expert protection as
|
||||
`--tensor-type "=f16"` patterns over the derive-the-hot-experts law
|
||||
(statistic: imatrix importances), Q6_K trunk, KLD/perplexity gate at
|
||||
the deployed slot shape. Production stays host-side; the 3090 is only
|
||||
the decode target.
|
||||
KV is cheap on this arch: 10 full layers × 2 KV heads × 256 =
|
||||
**20 KiB/token f16** (3.8 GiB per 200k slot); DeltaNet state is
|
||||
per-slot, tens of MiB. Size verdict (weights ≈ 32.2 B experts + 3 B
|
||||
trunk):
|
||||
- Archon-faithful (w4 experts + bf16 protected + i8 trunk) ≈ 22 GiB →
|
||||
27 GiB with KV+buffers → **over 24; that shape is 2×3090-class.**
|
||||
- Fits 1×200,704 text-only: IQ4_XS experts (17.4) + Q6_K trunk + q8_0
|
||||
KV (1.9; `GGML_CUDA_FA_ALL_QUANTS=ON` build) ≈ 21.5 ✓
|
||||
- Compaction-hub shape (2×200,704 + mmproj): experts at IQ3_M (14.4) ≈
|
||||
24.0 — the exact Swift IQ3_XXS 2-slot edge, MTP precedent exists.
|
||||
- Literal GPTQ-w4 encoding decodes only via vLLM/SGLang Marlin (Playbook
|
||||
B); compressed-tensors expresses the protected set per-module.
|
||||
Shelf check: HauhauCS publishes this model's K_P ladder (IQ2_M 10.86,
|
||||
IQ4_XS 17.44, Q4_K_M 19.71, Q4_K_P 21.82, Q5_K_P 26.1, Q6_K_P 28.5,
|
||||
Q8_K_P 40.6 GiB + mmproj 0.84) — Q4_K_P IS the protected-tensor blend.
|
||||
The self-made delta: calibration on OUR corpora, our hot-expert law,
|
||||
and a KLD gate scored at the serving shape.
|
||||
|
||||
### R9700 — frank-primary (radiance)
|
||||
The full self-quant pipeline already exists end to end: fetch BF16 →
|
||||
record KLD reference (1.0.13) → capture calibration grams (1.2.0 tap,
|
||||
`RADIANCE_CALIB_DIR/TOKENS`) → derive protected experts (energy-share) →
|
||||
`convert-swift15-w4dn8.sh` (1.0.13, host backend) → KLD gate (MNBT 2048
|
||||
law, pool 40960, mean KL ≤ 0.03, top-1 ≥ 95 %). New model = new recipe
|
||||
+ new `.so` arch plugin; every step is scripted. Prior art to fold in:
|
||||
REAP the expert set BEFORE step 3; ablate BEFORE calibration.
|
||||
Native encodings beyond w4: i8 trunk, fp8 HC/PLE/KV, bf16 protected,
|
||||
u2 draft — all recipe-expressible (see the w4dn8 runbook).
|
||||
|
||||
#### Recipe portability: the w4dn8 shape onto other MoEs (2026-10-08 audit)
|
||||
The pin 1.0.13 container ships SIX arch plugins
|
||||
(`/opt/radiance/share/radiance/architectures/`): `qwen4exp_fp8`,
|
||||
**`qwen35moe_fp8`**, `qwen35_fp8`, `qwen35_bf16`, `llama_fp8`,
|
||||
`llama_dense_fp8` (census via `podman run --rm --entrypoint sh <image> -c`
|
||||
— the image has no `ls`/`strings`; grep the `.so` files directly, and
|
||||
the image lives in ROOTLESS storage as `docker.io/stilldeadcode/radiance`).
|
||||
`grep -c` receipts: `qwen35moe_fp8.so` carries the `RADIANCE_CALIB` tap
|
||||
(2) and gram export (2) plus MTP (4) and NO ngram (0) — i.e. the full
|
||||
calibration→GPTQ path exists for the qwen3.5-MoE family; GPTQ itself
|
||||
lives in `rad-convert`, not the plugin (0 hits in both `.so`).
|
||||
|
||||
Worked example — `Qwen/Qwen3.6-35B-A3B` (config: `qwen3_5_moe_text`,
|
||||
40 layers, `full_attention_interval=4` DeltaNet hybrid, **256 experts
|
||||
top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
|
||||
248320, multimodal wrapper with vision tower):
|
||||
- Expert set ~32 B params → w4 ≈ 16 GiB; trunk (attn/DeltaNet/embed/
|
||||
lm_head ~3 B) at i8 ≈ 3 GiB ⇒ **~20 GiB total — fully VRAM-resident
|
||||
on ONE R9700** (32 GiB card): no TP=2, no host pool, no disk tier,
|
||||
no `--ngram-placement`, no 47.7 GiB PLE copy, no HC term. The recipe
|
||||
loses three lines (PLE/HC/protected-count scales with 256 experts).
|
||||
- Convert cost scales with the file: Flash-Next's 335 GiB took 5 h 50 m
|
||||
host-backend ⇒ ~70 GiB BF16 ≈ 1–1.5 h. KLD reference boot is trivial
|
||||
(BF16 ~65 GiB in host pool beside the pool KV needs).
|
||||
- Unresolved before any real campaign: does `qwen35moe` decode the
|
||||
`Qwen3_5MoeForConditionalGeneration` vision wrapper (assume text-only
|
||||
until proven), and does its MTP cast to the fp8/u2 encodings like
|
||||
qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.
|
||||
|
||||
### Strix Halo — frank-large (halogen) and friends
|
||||
- `.hgn`: **closed production door.** No public quantizer exists
|
||||
(verified absence across peonist-ai repos); the format's lock is the
|
||||
engine's lock. We consume vendor artifacts; the repo-side tooling is
|
||||
compaction/verification only.
|
||||
- Self-quant work on this card therefore runs through **GGUF runtimes**:
|
||||
the same imatrix/llama-quantize flow as the 3090 (HIP/Vulkan decode of
|
||||
K/I quants, gufo UD-* precedent, `--use-repack` off on ROCm), or the
|
||||
custom-type forks (ROCmFP4 STRIX, Kairic IU4) whose PRODUCERS we have
|
||||
never been — the quantizers for types 101/103 ship in those forks'
|
||||
trees; building one is a fork build, not a flag.
|
||||
- Native-FP4 note: RDNA4/Strix get hardware FP4 matmuls (hipBLASLt
|
||||
MXFP4/DSFP4), which is the engine-level path (radiance/vLLM), not a
|
||||
GGUF decode.
|
||||
|
||||
## Sources
|
||||
- REAP: arXiv 2510.13999, github.com/CerebrasResearch/reap,
|
||||
cerebras/GLM-4.5-Air-REAP-82B-A12B-FP8, cerebras/GLM-4.7-REAP-268B-A32B-FP8.
|
||||
- Heretic: github.com/p-e-w/heretic (AGPLv3).
|
||||
- Abliteration: arXiv 2406.11717 (Arditi et al.).
|
||||
- K_P: HauhauCS model cards (e.g. `HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive`
|
||||
README "What are K_P quants?"), FriendliAI Q8_K_P tensor census,
|
||||
fleet payload tree (BF16/Q4_K_M/Q6_K/Q8_0 + mmproj ladder in
|
||||
`HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive`).
|
||||
- EXL3: github.com/turboderp-org/exllamav3 (QTIP lineage; CUDA; Ampere
|
||||
decode incl. 3090 documented in v1.0 performance write-ups).
|
||||
- Ampere capability edges: vLLM FP8 docs ("only Hopper and Ada …
|
||||
W8A8; Ampere supported for W8A16 weight-only utilizing Marlin"),
|
||||
vLLM Supported-Hardware matrix (Marlin GPTQ/AWQ on SM 8.0/8.6),
|
||||
TensorRT-LLM Support Matrix (FP8 and INT4-AWQ-with-FP8-acts need
|
||||
SM≥89; NVFP4 Blackwell), SGLang issue #12887 (Ampere FP8-via-Marlin).
|
||||
- Repo: `scripts/convert-swift15-w4dn8.sh`,
|
||||
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
|
||||
`scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
|
||||
`scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
|
||||
[[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
|
||||
[[frank-primary-radiance-swift15-w4dn8]],
|
||||
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]],
|
||||
[[frank-large-flash-server-v2-swift15]], [[GGUF Compatibility]].
|
||||
@@ -80,21 +80,21 @@ resume law throughout; no token — the repos are public).
|
||||
|
||||
| Env | Value | Why |
|
||||
|---|---|---|
|
||||
| `HALOGEN_CTX` | 200192 | per-slot HARD ceiling (a longer prompt is a 400 naming the limit, never truncation). Harness compaction: 70 % = 140,134; 140,134 + 57,344 = 197,478 ≤ 200,192 |
|
||||
| `HALOGEN_CTX` | 262144 | per-slot HARD ceiling (the checkpoint's native window; a longer prompt is a 400 naming the limit, never truncation). **RAISED from 200192 on 2026-10-07 — the 400-zone law:** the harness clamps `max_tokens = min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts the rendered prompt by ~3 % (measured twice: est 152,938 → server 157,388, request over room by 354; est 158,906 → server 165,301, over by 2,300), and compaction only fires at 85 % of the harness window — so with window == engine ceiling, every request in the est ~146k–184k zone was a guaranteed 400. Harness window 217,088 = ceiling − 45k absorbs the error: worst-case request ≈ 219k ≤ 262,144 (≈10 % under-count headroom) |
|
||||
| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
|
||||
| `HALOGEN_KV_POOL_POSITIONS` | 400384 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ ~10.7 GiB KV |
|
||||
| `HALOGEN_KV_POOL_POSITIONS` | 524288 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ 14.0 GiB KV |
|
||||
| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
|
||||
| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
|
||||
| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness always sends 57,344); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
|
||||
| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness sends at most its `maxTokens` 45,056); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
|
||||
| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
|
||||
| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
|
||||
| `HALOGEN_MODEL_ID` / publish | frank-large / 7770:8731 | unchanged aliases; engine port 8730 never published |
|
||||
| `HALOGEN_MODEL_ID` / publish | **frank-dragoon** / 7770:8731 | served id renamed 2026-10-09 (service/unit/env keep `frank-large`); engine port 8730 never published |
|
||||
| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |
|
||||
|
||||
**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
|
||||
weights ~62 GiB resident + KV ~10.7 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
|
||||
+ vision ~2 ≈ **82.7 GiB engine-side**, + 20 host reserve = ~103 with
|
||||
~18 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
|
||||
weights ~62 GiB resident + KV 14.0 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
|
||||
+ vision ~2 ≈ **86 GiB engine-side**, + 20 host reserve = ~106 with
|
||||
~15 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
|
||||
table rides as mmap'd page cache beside the checkpoint — the card's own
|
||||
quickstart arithmetic does not bill it as resident, but WATCH the ledger
|
||||
at boot and re-size if the engine prints a clamp. Prompt cache is ON by
|
||||
@@ -177,11 +177,14 @@ the 27B engine ran on. Only revisit if the ledger shows SVM complaints.
|
||||
## Client contract (harness)
|
||||
|
||||
`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
|
||||
`http://192.168.0.143:7770/v1`: `contextWindow: 200192`,
|
||||
`maxTokens: 57344`, `input: [text, image]`,
|
||||
`http://192.168.0.143:7770/v1`: `contextWindow: 217088` — deliberately
|
||||
BELOW the 262,144 engine ceiling: the 45k gap absorbs the harness's
|
||||
~3 % chars/4 est under-count (see the `HALOGEN_CTX` row above) —
|
||||
`maxTokens: 45056`, `input: [text, image]`,
|
||||
`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
|
||||
fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
|
||||
frank-secondary (the `compactionModel` steering is unchanged).
|
||||
frank-secondary (the `compactionModel` steering is unchanged; the input
|
||||
≈ 170k fits its 200,704 slot).
|
||||
|
||||
## Rollback
|
||||
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
# frank-primary — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2
|
||||
|
||||
**Status: LIVE — deployed 2026-10-08 (main `3ca64e1`, e2e 33/33).** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||||
|
||||
The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert)
|
||||
is replaced by a **GPTQ-calibrated 4-bit container** of
|
||||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB, abliterated
|
||||
finetune) converted with the recipe
|
||||
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine
|
||||
**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as
|
||||
the FP8-native era; 1.2.0 was used ONLY for the calibration serve).
|
||||
Same port, alias, container name, systemd unit. Rollback: revert the
|
||||
`MODEL_FILE` line in `scripts/frank-primary.env` + redeploy —
|
||||
`/home/dank/models/rad/qwen4exp-fp8-native.rad` stays on disk (trash
|
||||
policy; 173 GiB, purge when the window closes).
|
||||
|
||||
## Why this swap
|
||||
|
||||
- **Context economics at production memory budgets**: the FP8-native
|
||||
experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and
|
||||
left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing
|
||||
host and VRAM budget for KV and slots at equal-or-better measured
|
||||
decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
|
||||
- **Swift 1.5 base**: abliterated finetune of Flash-Next (Frank's
|
||||
fleet-personality call; frank-large serves the same family).
|
||||
- **GPTQ closes the precision gap**: RTN alone on BF16 weights lands
|
||||
measurably worse; GPTQ with Hessians from a 10M-token corpus of this
|
||||
model's own distribution recovers most of it, verified end-to-end by a
|
||||
KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).
|
||||
|
||||
## The payload chain
|
||||
|
||||
1. **Fetch**: `scripts/download-frank-primary-swift15.sh` →
|
||||
`/home/dank/models/Swift1.5-Qwen3.8-Flash-Next` (131 shards, 335.3
|
||||
GiB, per-shard LFS sha256 verified, no token on the command line).
|
||||
48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560,
|
||||
PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
|
||||
2. **BF16 container**: `run-swift15-bf16.sh convert` →
|
||||
`rad-convert` no-recipe (pure copy) at `--max-ctx 4096` →
|
||||
`qwen4exp-swift15-bf16.rad` (335.28 GiB). Serves on 1.0.13 as the
|
||||
KLD reference engine (disk-tier placement; boot ~35 min).
|
||||
3. **KLD reference**: `run-swift15-bf16.sh kld` (1.0.13) →
|
||||
`/home/dank/corpora/kld-ref` — `radiance-kld-1` format, `rows:
|
||||
50235`, 197 source-labelled docs, top-16 log-probs (f16), corpus
|
||||
fnv1a64 `a9d62b33c18b3d9c`. ~4 h at 32k prefill steps
|
||||
(`--max-num-batched-tokens 32768` — a 65536 step OOMs an arena
|
||||
tensor at `--max-model-len 16384`). **The wrapper's exit line is the
|
||||
receipt: never scp this script while it runs** (bash lazy-read;
|
||||
a mid-run swap killed a clean engine exit with rc=127, third fleet
|
||||
occurrence of the script-swap law). `kld-then-calib.sh SKIP_KLD=1`
|
||||
resumes from the `rows: 50235` receipt.
|
||||
4. **Calibration**: `run-swift15-bf16.sh calib` on **1.2.0** (the only
|
||||
engine with `RADIANCE_CALIB_TOKENS`; grams are engine-invariant
|
||||
bf16 activations — RADGRAM2 format byte-identical across versions)
|
||||
+ `calib-push.py` pushing `calib-corpus.jsonl` (10,014,932 tokens,
|
||||
1,294 prompts, assembled from real HF sources, chat-templated,
|
||||
source-labelled). The tap (in the `qwen4exp_fp8.so` architecture
|
||||
plugin) writes per-expert down-proj Gram matrices
|
||||
`gram.r<rank>.blk.L.ffn_down_exps.<e>.bin` (32-byte RADGRAM2 header +
|
||||
640×640 f32; `rows` = routing count, `trace` = activation energy)
|
||||
and stops the serve at the token threshold. TP=2: whole experts per
|
||||
rank (256/rank); gate_up input is rank-sliced (plugin warning — the
|
||||
recipe only casts the down projection).
|
||||
5. **Protected-expert derivation**: `derive-protected-experts.py
|
||||
/home/dank/corpora/calib` → the 24 highest-energy experts (the law
|
||||
is data-driven; Swift drew 24, not the fp8-era 10 — its routing is
|
||||
spikier: `blk.47 e445` alone carries 32 % of its layer's down energy
|
||||
at 70× routing share) promoted to whole-expert bf16.
|
||||
Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count
|
||||
per rank half; odd half gains the next most energetic.
|
||||
`apply-protected-cast.sh` splices the derived names between the
|
||||
`# PROTECTED-BEGIN/END` markers of the recipe (refuses double-derive).
|
||||
6. **Final convert**: `convert-swift15-w4dn8.sh` (1.0.13, host-backend
|
||||
CPU-only — no GPU visible to `rad-convert`; GPTQ solve is CPU BLAS) —
|
||||
gates: ≥48 gram files present, bf16 container already deleted
|
||||
(disk), ≥146 GiB free, output ≥108,000 MiB, `rad-info` shows the
|
||||
gptq receipt. No `--reuse`. Measured: 5 h 50 m (6.0–6.5 min/layer on
|
||||
49 GPTQ layers, load-average 8), output **116,416 MiB**.
|
||||
The plugin's `rad_arch_declare` enforces ONE ENCODING PER EXPERT
|
||||
TABLE (`one table of experts takes one encoding, and only a whole
|
||||
expert kept plain bf16 is served beside it`) — a split tier
|
||||
(down=fp8, gate_up=w4) is unservable: the kernel serves a single
|
||||
w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated,
|
||||
gate_up uncalibrated), protected experts bf16 beside them.
|
||||
|
||||
## The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)
|
||||
|
||||
| Tier | Encoding | Why |
|
||||
|---|---|---|
|
||||
| MoE experts, both `gate_up` and `down` (one w4 table/layer) | **GPTQ w4nl64a8h** — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane | the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law) |
|
||||
| 24 derived protected experts (whole) | **bf16** | energy-share law (step 5) |
|
||||
| shared expert + trunk attention/SSM linears | **int8**, scale per 128 cols, searched | trunk precision law |
|
||||
| hyper-cond + MTP head | **fp8_e4m3 row-groups** | hc+mtp fp8 law |
|
||||
| lm_head | **int8** | i8 lm_head law (E4M3 costs 0.6 top-1 alone) |
|
||||
| PLE n-gram table | fp8 at the checkpoint's fixed scale 1.9931793212890625e-04 | the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau |
|
||||
| embeddings, vision tower, QSA indexer | **bf16 passthrough** | untouched |
|
||||
|
||||
## The shape (`scripts/frank-primary.env`)
|
||||
|
||||
Unchanged from the FP8-native era: TP=2 + `wht6`, `--max-num-seqs 3`,
|
||||
`--max-model-len 200000`, MNBT 2048 (the MTP resonance law — 2560
|
||||
collapses decode to 2.6–7.7 t/s), fp8 KV, `expert_tiered` host pool
|
||||
40960 (the load-bearing floor), `--ngram-placement ram`, froggeric
|
||||
v22.5 template override. `MODEL_FILE` points at
|
||||
`/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416
|
||||
MiB measured**); the serve size floor `FRANK_PRIMARY_RAD_MIN_MIB` moves
|
||||
177000 → **110000**.
|
||||
|
||||
## Gates (measured at cutover)
|
||||
|
||||
- **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`,
|
||||
candidate on 1.0.13 vs `kld-ref`: **mean KL 0.0146** (median 0.00126,
|
||||
p90 0.0315, p99 0.194, max 4.12), **top-1 97.84 %**,
|
||||
**ppl ratio 1.0126** (candidate 1.6913 vs reference 1.6702), 50,235
|
||||
positions / 197 docs. Report: `/home/dank/radiance-state/kld-gate.json`
|
||||
(+ `.rows` per position). Gate shape: disk-tier, pool 40960, MNBT 3072
|
||||
(the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena,
|
||||
refused), KV 2048, headroom 512, no speculation. The engine ran rank 0
|
||||
fully on card 0 (25.88 GiB) with rank 1 host-side in this shape —
|
||||
scoring is placement-invariant; scored at 28 positions/s.
|
||||
- **KLD gate re-PASS (2026-10-08 04:49) at the production-legal shape**:
|
||||
MNBT 2048 (the w4 file-tier law: experts stream in place only at
|
||||
MNBT ≤ 2048 — the recording's 32768 demands a widened form the disk
|
||||
tier refuses), pool 40960, vram-kv 3072 fp8, tp 2, headroom 768.
|
||||
**mean KL 0.0131, top-1 97.92 %**. The Δ vs the first PASS (0.0015) is
|
||||
the float-order noise band of the chunk-size change — the verdict is
|
||||
chunk-size invariant, ~2.3× inside the 0.03 bar on both shapes.
|
||||
- **Throughput gate: PASS (2026-10-08 05:40)** — decode at 44.5k depth
|
||||
(streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve
|
||||
shape, EVC 0.82) **146–173 t/s**, engine timings **121.9 t/s**,
|
||||
draft accept **0.7246** (350/483); arm C (deployed vram-pair shape
|
||||
26071/3543) median **136.0 t/s**, warm TTFT 9.45 s. Control
|
||||
(fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm.
|
||||
The w4 expert table is resident — the fp8-native era's NVMe expert
|
||||
streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60
|
||||
t/s base decode. Conversation battery (short context): geomean
|
||||
172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot:
|
||||
the w4 resident set fits fully inside the 40 GiB pool at 0.82
|
||||
(ledger: mover full fill). Depth-output audit: the 44.5k-depth
|
||||
summary is a coherent ordered index of the concatenated corpus
|
||||
(no repetition, no NaN-token garbage) — the depth-corruption class
|
||||
of 2026-10-04 is clean.
|
||||
- Fleet probes: PASS, re-run on the deployed prod — vision (solid-red
|
||||
512 px → "Red"), tool_calls (`{"expr":"23 * 47"}`), reasoning_effort
|
||||
xhigh/medium/low/high all 200 (the Swift1.5 froggeric template accepts
|
||||
the full set), reasoning_content present, response_format json_object
|
||||
honoured. **Vision probe law:** budget ≥ 256 tokens and read
|
||||
`reasoning_content` as fallback — at 64 tokens thinking exhausts the
|
||||
budget and `content` comes back empty (the answer lands in the
|
||||
reasoning channel).
|
||||
- **Deployed verification (2026-10-08)**: `deploy.sh dankserver
|
||||
--service frank-primary` green incl. Playwright e2e 33/33; unit
|
||||
`active`, `/health` 200, `/v1/models` id `frank-primary` (the harness
|
||||
alias is the served-model-name, independent of the file — models.yml
|
||||
untouched). Deployed shape: fp8 KV (KLD-gate shape), spec depth 2,
|
||||
seqs 3, vram pair 26071/3543. Boot ledger: KV pool 3.46 GiB, expert
|
||||
pools 57.84 GiB, cards settled 1.41 GiB free, `serving on :7770`.
|
||||
Live decode probe (512 tokens, streamed, temp 0): ~190 t/s.
|
||||
|
||||
### Wire rename + the bench drop-in trap (2026-10-09)
|
||||
|
||||
`FRANK_PRIMARY_SERVED_NAME` moved `frank-primary` → **frank-archon**; the
|
||||
service, unit, scripts and env file keep the old name. Radiance is the only
|
||||
fleet engine whose `/v1/models` listing reflects the flag, and it echoes the
|
||||
**requested** model name in chat responses — so a chat probe that returns 200
|
||||
proves nothing about the served id. Always read `/v1/models`.
|
||||
|
||||
The first deploy after the edit came up green and `/v1/models` still said
|
||||
`frank-primary`. Cause: a campaign leftover drop-in
|
||||
`/etc/systemd/system/frank-primary.service.d/bench.conf` (2026-10-08 04:09)
|
||||
overrode BOTH `Environment=FRANK_PRIMARY_ENV_FILE` and `ExecStart` to the
|
||||
staging worktree `/home/dank/serverAppConfigs-swift15`. Every
|
||||
`deploy.sh dankserver --service frank-primary` since then restarted the
|
||||
staging tree: green health gate, old configuration, silently. Verified before
|
||||
removing: the staging `serve-frank-primary.sh` and `frank-primary.service`
|
||||
are byte-identical to the checkout, its `frank-primary.env` differed only by
|
||||
the alias line, and its HEAD (`8375567`) is an ancestor of `main`. The
|
||||
drop-in is staged in `/home/dank/.AGENTTRASH/20261009-frank-primary-bench-dropin/`.
|
||||
After `daemon-reload` + restart, `/v1/models` reports `frank-archon` (boot
|
||||
healthy in 102 s, shape unchanged: fp8 KV, spec depth 2, seqs 3, vram pair
|
||||
26071/3543).
|
||||
|
||||
**Deployer blind spot:** `deploy.sh` plans from repo paths and gates on the
|
||||
port. It cannot see host-side drop-ins, so unit drift is invisible to a green
|
||||
run. After an LLM deploy, verify the served configuration behaviourally
|
||||
(here: `podman inspect <name> --format {{json .Config.Cmd}}` and `/v1/models`).
|
||||
|
||||
|
||||
|
||||
## Deploy + verify
|
||||
|
||||
```bash
|
||||
sudo systemctl stop frank-primary # drained window
|
||||
bash convert-swift15-w4dn8.sh # if not already run
|
||||
./scripts/deploy.sh dankserver --service frank-primary
|
||||
```
|
||||
|
||||
Post-cutover checklist: unit active/enabled; `/health` 200; boot ledger
|
||||
(expert pool coverage now w4-sized, KV pool should be larger than the
|
||||
FP8-native era's 399,824 tokens); MTP draft counts in `timings`; froggeric
|
||||
probe; ≥256 px `image_url` vision probe answers the color; tool-call
|
||||
probe; `reasoning_effort` fold; Prometheus target green; `tests/run.sh`
|
||||
+ Playwright e2e green.
|
||||
|
||||
## Rollback
|
||||
|
||||
| Layer | Path |
|
||||
|---|---|
|
||||
| FP8-native radiance (immediate) | revert `MODEL_FILE` in `scripts/frank-primary.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk |
|
||||
| FreeToken PR #587 | git tag `frank-primary-freetoken-pr587-20261005` |
|
||||
| tcclaviger vLLM shape G | git tag `frank-primary-vllm-shapeG-20261003` |
|
||||
|
||||
Campaign artifacts on disk: BF16 checkpoint (335 GiB), `kld-ref` (24
|
||||
GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, logs
|
||||
`/home/dank/logs/swift15_*.log`. The BF16 `.rad` container is deleted
|
||||
before the final convert (disk gate) — the raw checkpoint is the
|
||||
re-convert source if the window ever reopens.
|
||||
|
||||
## Links
|
||||
|
||||
- [[frank-primary-radiance-qwen38-native-fp8]] — the replaced FP8-native era
|
||||
- [[LLM/index]] — fleet index
|
||||
- [[swift15-campaign-worklog]] — campaign record
|
||||
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
|
||||
@@ -0,0 +1,111 @@
|
||||
# frank-secondary — llama.cpp CUDA, Gemma4-26B-A4B Q4_K_M, 1×200,704 (Dankclaw)
|
||||
|
||||
**RETIRED 2026-10-07** — same day it went live. Superseded by
|
||||
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]] (Qwen3.5-9B Q6_K at
|
||||
2 × 200,704 with exact KV); the payload stays at
|
||||
`/home/dank/models/Gemma4-26B-A4B/` as the rollback. This page keeps the
|
||||
Gemma4 KV-billing law (the `attention_k_eq_v` pair-billing measurement) and
|
||||
the drafter-cost boots, both of which outlive the model.
|
||||
|
||||
## Payload
|
||||
|
||||
|File|Path|Bytes|
|
||||
|---|---|---|
|
||||
|Main (Q4_K_M)|`/home/dank/models/Gemma4-26B-A4B/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf`|16,796,015,520 on disk; the engine reports 16,780,192,888|
|
||||
|Vision (mmproj BF16)|`/home/dank/models/Gemma4-26B-A4B/mmproj-Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf`|~1.195 GiB|
|
||||
|MTP assistant (on disk, NOT loaded)|`/home/dank/models/Gemma4-26B-A4B/mtp-gemma-4-26B-A4B-it.gguf`|~0.252 GiB|
|
||||
|
||||
Repo: `HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-MTP`
|
||||
(base `google/gemma-4-26B-A4B-it`). Downloaded 2026-10-07 by plain
|
||||
`curl -fL` (no token needed, ungated).
|
||||
|
||||
## Shape
|
||||
|
||||
```
|
||||
--ctx-size 200704 --parallel 1 # ONE conversation, Frank's 200k cap
|
||||
--flash-attn on
|
||||
--cache-type-k f16 --cache-type-v q8_0 # exact K, FP8-class V (q8_0)
|
||||
--batch-size 4096 --ubatch-size 1024
|
||||
--n-gpu-layers 99
|
||||
--log-verbosity 4 # debug: KV layout, buffers, graph capture
|
||||
--spec-type none # MTP OFF (see below)
|
||||
--mmproj <BF16 projector>
|
||||
--jinja --reasoning auto --reasoning-format deepseek
|
||||
```
|
||||
|
||||
Measured boot at **q8_0 K + q8_0 V, ubatch 2048, MTP off** (2026-10-07
|
||||
morning): **21,890 MiB used / 2,235 MiB free** (`nvidia-smi`). With MTP ON
|
||||
(the assistant loaded) the same shape was **23,806 / 319** — the drafter's
|
||||
context costs ~1.9 GiB because its KV is built against the target's full
|
||||
window ([[Llama Cpp Serving Sizing]]).
|
||||
|
||||
**Deployed shape (2026-10-07 evening, Frank): K = f16, ubatch 1024, verbosity
|
||||
4. Measured boot: 21,868 MiB used / 2,257 MiB free** — i.e. the f16-K raise
|
||||
(+2,161 MiB of KV, see the accounting below) was paid for by the ubatch cut
|
||||
(2048 → 1024 returns ~616 MiB of compute buffer) plus slot-geometry variance.
|
||||
Exact K came essentially free. The verbosity-4 ledger (journal, `llama_kv_cache`
|
||||
lines): model 16,002.91 MiB (CUDA0) + 577.50 MiB (CPU-mapped), **KV 3,001.25
|
||||
MiB (5 full layers) + 306.25 MiB (25 SWA rings)**, compute buffers 926.39 +
|
||||
150.63 MiB, output buffer 1.00 MiB.
|
||||
|
||||
**Measured decode: 139.9 tok/s** (`/completion` timings, temp 0, 256 tokens,
|
||||
deployed shape) — the ~4 B-active MoE is at home on the 3090. The model
|
||||
thinks: a 256-token budget on a short prompt can be consumed entirely by
|
||||
`reasoning_content` with empty `content` and `finish_reason: length`, so
|
||||
harness requests must budget for thinking (same law as the uncensored Swift
|
||||
payload). Vision probe on this build: solid-red 512 px PNG via the OpenAI
|
||||
`image_url` part → `Red`, `prompt_tokens` 151 (the image reached the model).
|
||||
|
||||
Decode is expected ubatch-invariant (Swift-era 3090 law; the 139.9 t/s above
|
||||
is at ubatch 1024 — one rung measured, not a ladder).
|
||||
|
||||
## Architecture facts
|
||||
|
||||
Gemma4 MoE hybrid: 30 layers, 128 experts (8 active), n_embd 2816, 262,144
|
||||
native window, n_vocab 262,144. Attention is hybrid per
|
||||
`gemma4.attention.sliding_window_pattern` (read from the file at load): **25
|
||||
sliding-window layers (window 1024, `n_head_kv = 8`) + 5 full-attention layers
|
||||
(indices 5, 11, 17, 23, 29, `n_head_kv = 2`)**, head_dim 256 for K and V in
|
||||
both classes. The KV pool is TWO caches: the full-context cache scales with
|
||||
the 5 layers at the full 200,704 window; the SWA cache is a fixed 1024-token
|
||||
ring per sliding layer.
|
||||
|
||||
**The rate law (measured, exact):** bytes/token = `n_kv_heads × 256 × 6.125 B`
|
||||
per layer at f16 K + q8_0 V — `attention_k_eq_v` does **NOT** shrink the
|
||||
allocation: `Vcur = Kcur` is a compute-graph mapping, the cache reserves K and
|
||||
V slots and, with `k_eq_v`, reserves the pair **twice** (6.125 = 2 × (2 B f16 +
|
||||
1.0625 B q8_0)). Verified against the boot: 5 × 2 × 256 × 6.125 = 15,680
|
||||
B/token × 200,704 = **3,001.25 MiB** ✓ and 25 × 1024 × 8 × 256 × 6.125 =
|
||||
**306.25 MiB** ✓. At q8_0 K + q8_0 V the rate is 2.125 B/elem: the same shape
|
||||
costs 1,041 + 106 = **1,147 MiB**, so the f16-K raise is **+2,161 MiB**. The
|
||||
`--cache-type-v` flag is honoured, not inert.
|
||||
|
||||
The llama.cpp master build at commit `5fc4f3c` supports the arch
|
||||
(`src/models/gemma4.cpp`, `LLM_ARCH_GEMMA4 = "gemma4"`). Note: `grep -c`
|
||||
arch names on the packed binary returns 0 for EVERY arch (including the
|
||||
`qwen3_5` it serves) — check the source tree, not the binary.
|
||||
|
||||
## MTP law
|
||||
|
||||
MTP for Gemma4 is a SEPARATE file (`mtp-gemma-4-26B-A4B-it.gguf`, arch
|
||||
`gemma4-assistant`), loaded via `--spec-draft-model` beside
|
||||
`--spec-type draft-mtp` — unlike Swift, whose MTP head ships inside the main
|
||||
GGUF. The serve script gained `SPEC_DRAFT_FILE` for this. 2026-10-07 (Frank):
|
||||
**MTP disabled** — `SPEC_TYPE=none` strips both spec flags (flags move as a
|
||||
set). The assistant payload stays on disk; setting `SPEC_TYPE=draft-mtp` +
|
||||
`SPEC_DRAFT_FILE` re-enables it at the cost of ~1.9 GiB (boots at 319 MiB
|
||||
free).
|
||||
|
||||
## Verification
|
||||
|
||||
- `curl http://192.168.0.143:7771/health` → `{"status":"ok"}`
|
||||
- `/v1/models` reports `n_ctx: 200704`, `ftype: Q4_K - Medium`
|
||||
- Vision probe: solid-color PNG ≥ 256 px via OpenAI `image_url` part,
|
||||
expect the exact color word.
|
||||
|
||||
## Rollback
|
||||
|
||||
Revert to the Swift trio: `MODEL_FILE=/home/dank/models/Swift15-UncIQ3XXS/…`,
|
||||
`CTX_SIZE=229376`, `PARALLEL=2`, in-file `SPEC_TYPE=draft-mtp` — the Swift
|
||||
payload remains on disk. See
|
||||
[[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]].
|
||||
@@ -0,0 +1,150 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-secondary on Dankclaw — llama.cpp, Qwen3.5-9B Q6_K, 2 x 200704, f16 KV (ACTIVE)"
|
||||
description: "The ACTIVE frank-secondary runbook (since 2026-10-07) — llama.cpp master (CUDA sm_86) serving HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090: two conversations of 200,704 tokens with exact f16 KV, vision through the publisher's BF16 projector, MTP impossible (no tensors in the file), and the measured geometry that priced the shape."
|
||||
tags:
|
||||
- llm
|
||||
- frank-secondary
|
||||
- dankclaw
|
||||
- llama-cpp
|
||||
- gguf
|
||||
- rtx3090
|
||||
timestamp: 2026-10-07
|
||||
---
|
||||
|
||||
# frank-secondary — llama.cpp Qwen3.5-9B Q6_K, 2 × 200,704, f16 KV
|
||||
|
||||
Live since 2026-10-07, replacing the
|
||||
[[frank-secondary-llama-gemma4-q4km-200704]] era (Q4_K_M on ONE 200,704
|
||||
slot). The swap is a capacity trade: the 9B dense-hybrid rung costs ~7 GiB of
|
||||
weights where the 26B MoE cost ~16.6 GiB, and the freed room buys the second
|
||||
conversation slot and unquantized KV back.
|
||||
|
||||
## Identity
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
|
||||
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-secondary` name), `/v1` OpenAI surface |
|
||||
| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`); `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
|
||||
| Unit | root systemd `frank-secondary.service` ([`scripts/frank-secondary.service`](../../scripts/frank-secondary.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
|
||||
| Serve script | [`scripts/serve-frank-secondary.sh`](../../scripts/serve-frank-secondary.sh), tunables [`scripts/frank-secondary.env`](../../scripts/frank-secondary.env) |
|
||||
| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
|
||||
| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
|
||||
| Fetch | [`scripts/download-frank-secondary.sh`](../../scripts/download-frank-secondary.sh) — `-f`, byte-count + sha256 verified (the hashes are computed locally: the repo is Xet-backed, so the Hub publishes file OIDs, not file sha256), idempotent, `--verify-only` |
|
||||
| Seam | `tests/frank_secondary_unit.bats` (argv assertions against a stub engine) |
|
||||
|
||||
## Payload geometry (read from the GGUF header, not the model card)
|
||||
|
||||
Range-read of the file's own metadata + tensor-info section:
|
||||
|
||||
|Key|Value|
|
||||
|---|---|
|
||||
|`general.architecture`|`qwen35` (dense hybrid: Gated DeltaNet + softmax attention)|
|
||||
|`qwen35.block_count`|32, `full_attention_interval` 4 ⇒ **8 full-attention layers (3, 7, 11, …, 31) + 24 DeltaNet layers**|
|
||||
|attention|16 heads, **4 KV heads**, `key_length` = `value_length` = 256|
|
||||
|`qwen35.ssm.*`|`conv_kernel` 4, `state_size` 128, `group_count` 16, `time_step_rank` 32, `inner_size` 4096|
|
||||
|context / vocab|262,144 native / 248,320|
|
||||
|tensor types|Q6_K × 250 + F32 × 177 — standard types only, so the stock master build loads it (the ROCmFPX-style custom-quant lock-in does not apply)|
|
||||
|MTP|**no `nextn`/`mtp` tensors exist** — in-file MTP is impossible; the card's "MTP support" line describes the base architecture, not this quantized export|
|
||||
|chat template|supports `enable_thinking`, `tools`/`tool_call`, image/video content parts; it has **no `reasoning_effort` variable** (unused Jinja variables are ignored, so the harness's effort map is inert here, not an error source)|
|
||||
|
||||
## Shape
|
||||
|
||||
`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
|
||||
so each conversation gets `n_ctx_slot = 200,704` (Frank's cap; the harness
|
||||
`contextWindow` quotes the per-slot number). `--flash-attn on`,
|
||||
**`--cache-type-k f16 --cache-type-v f16`** (exact KV), batch 4096 /
|
||||
**ubatch 2048**, `--log-verbosity 4`, `--jinja --reasoning auto
|
||||
--reasoning-format deepseek`, `--mmproj`, **speculative decoding OFF**
|
||||
(`SPEC_TYPE=none` strips both spec flags — the file has no MTP tensors).
|
||||
|
||||
**KV rate law.** The pool scales with the 8 full-attention layers: 8 × 4 KV
|
||||
heads × 256 dim = 8,192 elements/token per cache, so f16 K + f16 V bills
|
||||
32,768 B/token = **32 KiB/token** → 12.25 GiB at the 401,408 pool. The 24
|
||||
DeltaNet layers bill a fixed recurrent state **per slot**, not per token —
|
||||
that is what makes two 200k slots affordable where the 26B MoE fit only one.
|
||||
|
||||
**Priced from boots, not formulas.** MiMo-V2.6-Distill-Qwen-9B (identical
|
||||
qwen35 geometry, 7,793,710,624 B, same mmproj class) at 2 × 200,704 f16 K +
|
||||
f16 V measured **21,276 used / 2,849 free** at ubatch 1024 and **21,846 /
|
||||
2,279** at ubatch 2048 — see [[frank-secondary-llama-mimo-q6k-2x200704]].
|
||||
This file is 415 MiB lighter in weights and its projector is 3 MiB heavier,
|
||||
so the projection is ~21,430 used / ~2,700 free.
|
||||
|
||||
Ubatch 2048 is the measured CUDA optimum for this card and arch (+4.6 %
|
||||
prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented
|
||||
fallback rung if a future shape lands under the 2 GiB floor.
|
||||
|
||||
## Measured boot (2026-10-07, deploy `ec6184d`)
|
||||
|
||||
`nvidia-smi`: **21,436 MiB used / 2,689 MiB free** — the MiMo-priced
|
||||
projection (~21,430 / ~2,700) was within 11 MiB. Engine ledger
|
||||
(`--log-verbosity 4`, `journalctl -u frank-secondary`):
|
||||
|
||||
|Line|Value|
|
||||
|---|---|
|
||||
|`llama_context`|`n_ctx = 401408`, `n_ctx_seq = 200704`, `kv_unified = false`|
|
||||
|`llama_kv_cache`|`CUDA0 KV buffer size = 12544.00 MiB` — `size = 12544.00 MiB (200704 cells, 8 layers, 2/2 seqs), K (f16): 6272.00 MiB, V (f16): 6272.00 MiB`|
|
||||
|`sched_reserve`|`CUDA0 compute buffer size = 1136.06 MiB`, `CUDA_Host compute buffer size = 848.07 MiB`|
|
||||
|`clip_model_loader` (mmproj)|`model size: 878.99 MiB` + `reserve_compute_meta: CUDA0 248.10 MiB, CPU 24.93 MiB`|
|
||||
|`srv load_model`|`n_slots = 2, n_ctx_slot = 200704`|
|
||||
|`print_info`|`n_ctx_train = 262144`, `n_head_kv = 4`|
|
||||
|
||||
The `32 KiB/token` law is confirmed exactly: 401,408 × 32 KiB = 12,544 MiB,
|
||||
which is what the engine printed. The `n_ctx_seq (200704) < n_ctx_train
|
||||
(262144)` line is informational (a context **below** the trained window is
|
||||
safe; the Gemma4 era's warning was the opposite case, a drafter running 53 %
|
||||
**past** its window). Weights load in ~6 s from page cache; `/health` answers
|
||||
`{"status":"ok"}` and `/v1/models` serves the alias `frank-zealot`.
|
||||
|
||||
**Wire rename (2026-10-09):** `ALIAS` moved `frank-secondary` → `frank-zealot`,
|
||||
so the served id matches the harness model id and the family-facing name.
|
||||
Consumers re-keyed in the same change: OpenWebUI (`model` row, `access_grant`
|
||||
row, `openai.api_configs` index-2 `model_ids`) and `models.yml` (already
|
||||
renamed 2026-10-08). llama.cpp does NOT validate the requested model name —
|
||||
it answers and echoes the served alias — so a stale client keeps working;
|
||||
the picker and any id-keyed grant is what breaks. See [[OpenWebUI]].
|
||||
|
||||
The deploy's Playwright gate passed (33 tests) and the fleet e2e set is
|
||||
unchanged.
|
||||
|
||||
## Client contract
|
||||
|
||||
llama.cpp surface: model id `frank-secondary`, native tools via `--jinja`,
|
||||
thinking in **`reasoning_content`**, vision via OpenAI `image_url` content
|
||||
parts (probe with a ≥ 256 px image, or the Qwen-VL tower answers "video").
|
||||
The publisher's recipe ships in `models.yml` `extraBody`: thinking mode
|
||||
`temperature 0.6 / top_p 0.95 / top_k 20 / min_p 0` (non-thinking mode is
|
||||
0.7/0.8/20/0). The card's advice "maintain at least 128K context to preserve
|
||||
thinking" is satisfied by the 200,704 slot.
|
||||
|
||||
**Compaction hub**: `models.yml` routes `compactionModel` for all three
|
||||
endpoints here. With `--parallel 2` a compaction request no longer serializes
|
||||
behind a live generation (the Gemma4 era's `--parallel 1` law), and the
|
||||
200,704 slot still fits the worst-case compaction input (~133k tokens).
|
||||
|
||||
## Rollback
|
||||
|
||||
|Target|How|
|
||||
|---|---|
|
||||
|Gemma4-26B-A4B Q4_K_M, 1 × 200,704, f16 K + q8_0 V|revert the swap commit — payload stays at `/home/dank/models/Gemma4-26B-A4B/`; see [[frank-secondary-llama-gemma4-q4km-200704]]|
|
||||
|MiMo-V2.6 Q6_K, 2 × 200,704, f16 KV|`MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` + its mmproj — measured 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048|
|
||||
|Swift-1.5-27B IQ3_XXS 2 × 114,688 + MTP|payload on disk at `/home/dank/models/Swift15-UncIQ3XXS/`|
|
||||
|
||||
## When it misbehaves
|
||||
|
||||
Same decision tree as the predecessors: `No CUDA GPUs are available` ⇒
|
||||
`nvidia-modprobe -u -c=0`; a 400 naming 200,704 ⇒ the conversation outgrew
|
||||
one slot, not the pool; queue behind two long generations ⇒ the slot ceiling
|
||||
is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in
|
||||
`journalctl -u frank-secondary` (KV MiB, compute buffers) before touching
|
||||
flags — the engine prints the exact figures the shape needs.
|
||||
|
||||
**`status=203/EXEC` boot loop**: the rewrite of `serve-frank-secondary.sh`
|
||||
shipped without its `#!/usr/bin/env bash` line, and systemd failed the
|
||||
`ExecStart` before the script ran — 10 s restarts, `Mem peak: 1.9M`, and
|
||||
`nvidia-smi` flat at 1 MiB used (the GPU is never touched). Journal signature:
|
||||
`Failed at step EXEC spawning /home/dank/…/serve-frank-secondary.sh`. A
|
||||
shebang is not decoration on this host: every LLM serve script is a direct
|
||||
`ExecStart`, so `head -1` is part of the deploy check (commit `ec6184d`).
|
||||
@@ -16,10 +16,9 @@ digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
|
||||
ctx, vision ON)** and
|
||||
**llama.cpp CUDA (frank-secondary, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
|
||||
Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
|
||||
IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
|
||||
128k cap (the tightest fit in the endpoint's history — 489 MiB free),
|
||||
**llama.cpp CUDA (frank-secondary, port 7771: HauhauCS Gemma4-26B-A4B QAT
|
||||
Uncensored Q4_K_M GGUF — 26B MoE hybrid (25 sliding-window + 5 full-attention
|
||||
layers), ONE 200,704-token conversation, MTP disabled,
|
||||
q8_0 K + q8_0 V (FP8 class), vision via the publisher's own BF16
|
||||
mmproj, on the RTX 3090)**.
|
||||
|
||||
@@ -52,8 +51,8 @@ frank-large, port 7770]
|
||||
2 x 200192 ctx, MTP always-on, image input]
|
||||
G[llama.cpp master CUDA sm_86
|
||||
frank-secondary, port 7771]
|
||||
H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
|
||||
2 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
|
||||
H[Gemma4-26B-A4B QAT Uncensored Q4_K_M GGUF
|
||||
1 x 200704 ctx, q8_0 K + q8_0 V, MTP off, mmproj vision]
|
||||
E2 --> E --> F
|
||||
E2 --> G --> H
|
||||
end
|
||||
@@ -103,8 +102,24 @@ frank-secondary, port 7771]
|
||||
(`slow`/`advisor` → frank-primary) carried forward. Superseded by the
|
||||
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
|
||||
engine moved with the model
|
||||
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] — **ACTIVE since
|
||||
2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
|
||||
- [[frank-secondary-llama-qwen35-9b-q6k-2x200704]] — **ACTIVE since 2026-10-07**
|
||||
(root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
|
||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090
|
||||
— TWO 200,704-token conversations (`--ctx-size 401408 --parallel 2`), exact
|
||||
f16 K + f16 V, vision via the publisher's BF16 mmproj. MTP is OFF because
|
||||
the file has no MTP tensors at all. Priced from the MiMo Q6_K boots (same
|
||||
qwen35 geometry, same card, same shape). Supersedes the Gemma4 era (below).
|
||||
- [[frank-secondary-llama-gemma4-q4km-200704]] — **RETIRED 2026-10-07** (same
|
||||
day; superseded by qwen35-9b-q6k; payload on disk as rollback): llama.cpp
|
||||
master (CUDA sm_86) serving HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-
|
||||
HauhauCS-Balanced-MTP **Q4_K_M** on the RTX 3090 — ONE 200,704-token
|
||||
conversation, f16 K + q8_0 V, vision via the publisher's mmproj. MTP was
|
||||
OFF (the gemma4-assistant drafter costs ~1.9 GiB: 23,806 used/319 free with
|
||||
it, 21,890/2,235 without).
|
||||
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] — **RETIRED
|
||||
2026-10-07** (superseded by gemma4-q4km; payload on disk as rollback);
|
||||
ACTIVE 2026-10-05 (evening) → 2026-10-07; UNCECERSED RCO-IQ3_XXS with
|
||||
MTP back on since
|
||||
2026-10-06 evening; RENAMED from frank-small and reshaped to 2 ×
|
||||
114,688 / ubatch ladder 512 → 1024 → **2048** late 2026-10-06** (root
|
||||
systemd unit, port
|
||||
@@ -137,6 +152,10 @@ frank-secondary, port 7771]
|
||||
target; payload on disk; owns the measured boot-pair table)**: the same
|
||||
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
||||
104.8 tok/s, 3,571 MiB true free
|
||||
- [[frank-primary-radiance-swift15-w4dn8]] — **STAGED (campaign running)**:
|
||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
||||
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
||||
replaces the FP8-native payload when the gates pass
|
||||
- [[frank-primary-radiance-qwen38-native-fp8]] — **ACTIVE since 2026-10-05**
|
||||
(root systemd unit, port 7770): the StillDeadcode **radiance** engine
|
||||
(C++/HIP, libr4d gfx1201 kernels, digest-pinned 287 MiB image, engine as
|
||||
@@ -473,6 +492,9 @@ frank-secondary, port 7771]
|
||||
One polished reference page per skill-cluster, distilled from the managed
|
||||
skill library into the vault (see [log](../log.md) for the wave plan):
|
||||
|
||||
- [[Quantization Production]] — how we self-make quants per card
|
||||
(imatrix/K_P ladders, radiance recipes, REAP/Heretic/abliteration
|
||||
prior art), and what each runtime decodes natively
|
||||
- [[GGUF Compatibility]] — what a GGUF's tensor histogram and header demand
|
||||
of a loader; how the runtime is chosen by the file
|
||||
- [[Llama Cpp Serving Sizing]] — slot/pool/KV sizing laws, boot-shape
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
# Swift1.5 w4dn8 Campaign — Work Log
|
||||
|
||||
Campaign branch `swift15-radiance-w4dn8`. Goal: convert
|
||||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB) to
|
||||
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (GPTQ gate_up w4nl64 + block-fp8 down +
|
||||
10 protected bf16 experts + int8 trunk + fixed-scale PLE) and swap it into
|
||||
frank-primary on the 1.0.13 pin. Design settled (25-question grill); this log
|
||||
carries execution facts only. The cutover runbook
|
||||
`frank-primary-radiance-swift15-w4dn8.md` replaces this page at cutover.
|
||||
|
||||
## 2026-10-06 evening
|
||||
|
||||
- Purge freed 835.8 GB; Swift1.5 BF16 downloaded 131/131 shards (sha256 ok).
|
||||
- Corpora built: calib 10,014,932 tok / 1,294 prompts; KLD 1,800,816 tok /
|
||||
197 prompts (`/home/dank/corpora/out/`).
|
||||
- Engine verdict: 1.2.0 REJECTED for serving (decode 29.21 vs control 30.08,
|
||||
accept 0.7123 vs 0.7549 deterministic; vram-pair translation ledger-exact).
|
||||
**Serving pin stays 1.0.13** `sha256:4c3f48…c322e`.
|
||||
- bf16 rad-convert done: `/home/dank/models/rad/qwen4exp-swift15-bf16.rad`
|
||||
(335.28 GiB, 248,077 vocab tokens baked).
|
||||
- **Blocker resolved — engine-version probe law:** the calibration tap lives
|
||||
in the architecture plugin `/opt/radiance/share/radiance/architectures/
|
||||
qwen4exp_fp8.so`, NOT the radiance binary, and the image has no
|
||||
`strings(1)` (greps return 0 for everything — positive controls
|
||||
mandatory). 1.0.13 plugin: `RADIANCE_CALIB_DIR` only; 1.2.0 adds
|
||||
`RADIANCE_CALIB_TOKENS`. Both `libquant.so` carry the gptq quantizer
|
||||
(`calib=/damp/act_order/cd`) and the identical `RADGRAM2` format
|
||||
(magic + `gram.r%d.%s.bin` + calib_gram/calib_resolve/calib_stem).
|
||||
**Decision: calibrate on 1.2.0, final convert on 1.0.13** — the serving
|
||||
pin produces the served `.rad`; only the Gram files cross versions.
|
||||
- `run-swift15-bf16.sh` boot-failure trail (all fixed, committed):
|
||||
duplicated `radiance` entrypoint arg (rc=2); ratio-derived KV pool SHORT
|
||||
217 MiB at 32×16k (explicit `--vram-kv-mib`); explicit 4608 starved the
|
||||
disk tier — **weights slab = card − headroom − kv, disk-tier floor
|
||||
9.57 GiB** (4.21 layer buffers + 5.36 fixed) → kv 3072; `KLD_MOUNT`
|
||||
referenced-never-defined emptied the corpus mount.
|
||||
- KLD record launched on 1.0.13 (disk tier, kv 3072, 4×16k, MNBT 32768):
|
||||
measured ~116 tok/s at the 32k step (44 docs/58 min).
|
||||
- Chain driver `kld-then-calib.sh` (pid 2848206, log
|
||||
`/home/dank/logs/swift15-chain.log`): KLD rc=0 → verify kld-ref → boot
|
||||
1.2.0 calib (MNBT 65536/seqs 8 per MOE-W4 930 tok/s law, synced commit
|
||||
45de2fd) → `calib-push.py` parallel 16 → kill → prod restored by trap.
|
||||
- Ready on the branch: `convert-swift15-w4dn8.sh` (1.0.13 convert, gates:
|
||||
≥126,000 MiB), `apply-protected-cast.sh` (derive → recipe PROTECTED
|
||||
section, refuses double-derive), `kld-gate-swift15.sh` (candidate vs
|
||||
reference on 1.0.13, gate mean KL ≤ 0.03 / top-1 ≥ 95 %).
|
||||
- `frank-primary` showing `failed` during the campaign is the 143-on-stop
|
||||
signature of the foreground podman unit — cosmetic, not a crash.
|
||||
|
||||
## 2026-10-07 morning
|
||||
|
||||
- **KLD reference COMPLETE** on 1.0.13: `/home/dank/corpora/kld-ref` 24 GiB,
|
||||
`radiance-kld-1`, `rows: 50235`, 197 docs, top-16 log-probs (f16),
|
||||
corpus fnv1a64 `a9d62b33c18b3d9c`. Record took ~4 h at the 32k step.
|
||||
The wrapper died rc=127 AFTER the engine exited clean: the mid-run scp of
|
||||
commit 45de2fd shifted bash's lazy read offset — script-swap law, third
|
||||
fleet occurrence. Resume path added: `SKIP_KLD=1` + `rows: 50235` receipt
|
||||
gate (cf6c62d). **Never scp a script a running process owns.**
|
||||
- Calibration chain3 live on 1.2.0 (big-step shape MNBT 65536/seqs 8): push
|
||||
started, requests landing at 7–8k tokens.
|
||||
|
||||
## Campaign order after the chain
|
||||
|
||||
1. derive protected experts from Swift's own grams; apply to recipe.
|
||||
2. Delete bf16 `.rad` (335 GiB) to `.AGENTTRASH` staging (Freed space gates the convert).
|
||||
3. `convert-swift15-w4dn8.sh` (~2 h, CPU-only, prod can stay up).
|
||||
4. rad-tune declare TP=2 spec=2; vram-pair boot arms; bench (gate decode ≥ 30.8 t/s).
|
||||
5. `kld-gate-swift15.sh` (fail → escalate gate_up w4→w5nl, pre-authorized).
|
||||
6. Cutover: docs, tests + e2e, deploy, probes. Throughput fallback: w4 down tier (Q35).
|
||||
|
||||
Disk ledger at KLD-record time: 104 GiB free → KLD ref −50 → Grams −20 →
|
||||
bf16 delete +335 → convert +132 written.
|
||||
@@ -159,7 +159,7 @@ skill-cluster. See [log.md](log.md) for the wave plan and results.
|
||||
[[vLLM Porting and Compile Triage]] · [[LLM Bench Windows and Measurement]] ·
|
||||
[[LLM Performance Diagnosis]] · [[Model Swap and Rollback]] ·
|
||||
[[Tcclaviger Fork Ops]] · [[Heterogeneous Split Triage]] ·
|
||||
[[FreeToken Builds]]
|
||||
[[FreeToken Builds]] · [[Quantization Production]]
|
||||
|
||||
**Agent:**
|
||||
|
||||
@@ -182,6 +182,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
### Deployment and Topology
|
||||
|
||||
- [[deployment]] — unified deploy and update process for both servers
|
||||
- [[frank-primary-radiance-swift15-w4dn8]] — STAGED frank-primary runbook:
|
||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
||||
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
||||
activates by flipping `MODEL_FILE` in `scripts/frank-primary.env`, rollback
|
||||
= revert that line (the FP8-native `.rad` stays on disk)
|
||||
- [[frank-primary-radiance-qwen38-native-fp8]] — the ACTIVE frank-primary
|
||||
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
||||
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
||||
@@ -191,6 +196,9 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
7.32 KiB/token, MTP depth 3, 2 × 200,000 first boot; rollback is the
|
||||
FreeToken one-card era (tag `frank-primary-freetoken-pr587-20261005`),
|
||||
then the tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`)
|
||||
- [[swift15-campaign-worklog]] — ACTIVE campaign log: the Swift1.5 →
|
||||
w4dn8 radiance convert and frank-primary swap in flight (branch
|
||||
`swift15-radiance-w4dn8`); the cutover runbook replaces it at completion.
|
||||
- [[frank-primary-freetoken-pr587]] — RETIRED 2026-10-05 (superseded the
|
||||
same day by the radiance cutover above; kept as the rollback runbook):
|
||||
FreeToken built at PR #587 head `77a7201` on the host into
|
||||
@@ -372,8 +380,22 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] — the ACTIVE
|
||||
frank-secondary runbook since 2026-10-05 (evening; **UNCECERSED RCO-IQ3_XXS
|
||||
- [[frank-secondary-llama-qwen35-9b-q6k-2x200704]] — the **ACTIVE** frank-secondary
|
||||
runbook since 2026-10-07 (late): llama.cpp master (CUDA sm_86) serving
|
||||
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** on the RTX 3090 —
|
||||
TWO 200,704-token conversations, exact f16 K + f16 V, MTP **impossible**
|
||||
(the file carries no `nextn`/`mtp` tensors), vision through the publisher's
|
||||
BF16 mmproj; priced from the MiMo Q6_K boots (same qwen35 geometry)
|
||||
- [[frank-secondary-llama-gemma4-q4km-200704]] — RETIRED 2026-10-07 (same day;
|
||||
payload on disk as rollback): llama.cpp master (CUDA sm_86) serving
|
||||
HauhauCS Gemma4-26B-A4B QAT Uncensored **Q4_K_M** on the RTX 3090 — ONE
|
||||
200,704-token conversation, f16 K + q8_0 V, MTP **disabled** (the
|
||||
`gemma4-assistant` drafter costs ~1.9 GiB: boot was 23,806 used/319 free
|
||||
with it, 21,890/2,235 without), vision through the publisher's mmproj
|
||||
- [[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] — RETIRED 2026-10-07
|
||||
(superseded by gemma4; payload on disk as the rollback): the
|
||||
frank-secondary runbook 2026-10-05 (evening) → 2026-10-07
|
||||
(**UNCECERSED RCO-IQ3_XXS
|
||||
with MTP back on since 2026-10-06 evening; renamed from frank-small and
|
||||
reshaped to TWO 114,688-token conversations at Frank's 112k cap with
|
||||
the ubatch ladder climbed to 2048 late 2026-10-06**): llama.cpp master (CUDA
|
||||
|
||||
@@ -1,5 +1,121 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-09
|
||||
- **Live — five authentik human accounts provisioned (Dankclaw, `ak shell`
|
||||
ORM, no repo objects):** kirby (pk 17), ry (18), nic (19), debo (20),
|
||||
freya (21), each `<user>@franksdank.cloud` in the human-user template shape
|
||||
(`is_active=True`, password via `set_password` on stdin only, groups
|
||||
`Non-Admin Users` + `jellyfin_users` + `openwebui_users`). Verified through
|
||||
the exact PasswordStage code path (`authenticate()` accepts each password and
|
||||
rejects a wrong one) and `app_entitlements(open-webui)` → `Open WebUI Users`
|
||||
for all five (identical to the ridley precedent). No `proxysession` rows
|
||||
existed for them — groups were set before any login, so no claim purge was
|
||||
due. Downstream: Jellyfin visibility rides the LDAP outpost cache (expect
|
||||
lag); OpenWebUI mints their accounts as role `pending` on first login.
|
||||
- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user template
|
||||
census extended with pk 17–21.
|
||||
- `/serverAppConfigs-docs/Dankclaw/OpenWebUI.md` — new SSO bullet: first login
|
||||
lands as role `pending` (`ui.default_user_role=pending` in the config DB),
|
||||
the role-flip SQL, and the global `access_grant` (`principal_id='*'`) fact
|
||||
that makes the picker model visible to every `role='user'` account.
|
||||
|
||||
- **Live — the Protoss rename finished on the wire (all three endpoints).**
|
||||
Served model ids moved `frank-primary` → **frank-archon**
|
||||
(`FRANK_PRIMARY_SERVED_NAME`), `frank-large` → **frank-dragoon**
|
||||
(`HALOGEN_MODEL_ID`), `frank-secondary` → **frank-zealot** (`ALIAS`). The
|
||||
services, units, scripts, env files and runbook pages keep the old names —
|
||||
container/unit identity stays stable, only the alias moved. Deploys in order:
|
||||
`dankclaw --service frank-secondary` (e2e gate 33/33 green), `dankclaw
|
||||
--service frank-large` (healthy 30 s), `dankserver --service frank-primary`
|
||||
(healthy 102 s), `dankclaw --service homepage`. Verified on `/v1/models`,
|
||||
not on chat echoes: `frank-zealot` (`n_ctx 200704`, Q6_K), `frank-dragoon`
|
||||
(`context_length 262144`), `frank-archon`.
|
||||
- **Live — OpenWebUI re-keyed in the same window** (snapshot
|
||||
`webui.db.bak-20261009`): `model` row (`frank-secondary` → `frank-zealot`,
|
||||
params + `meta.description` preserved), the `access_grant` row
|
||||
(`model`/`frank-zealot`/`user`/`*`/`read`), and `openai.api_configs`
|
||||
`model_ids` on all three backends; `systemctl --user restart
|
||||
open-webui.service`. Family-account proof, in a real browser as `qa-e2e`:
|
||||
the welcome screen selects `frank-zealot` and
|
||||
`POST /api/v1/chat/completions {"model":"frank-zealot"}` answers
|
||||
`zealous` with `"model":"frank-zealot"`. A `GET /api/v1/models/list`
|
||||
probe returns `{"items":[],"total":0}` for that account while the picker
|
||||
is plainly populated — the API census is not the acceptance test, the UI
|
||||
snapshot is.
|
||||
- **Live — Hermes re-keyed** (`config.yaml` `frank-large` → `frank-dragoon`
|
||||
in the provider block, `model.default` and the three `dankclaw-large`
|
||||
auxiliary routes; backup `config.yaml.bak-20261009`; `hermes.service` +
|
||||
`hermes-webui.service` restarted, gateway log clean, `/health` ok) and the
|
||||
repo baseline `dankclaw/hermes/config.yaml.example` synced to the live
|
||||
shape (context 262144). **Flagged, not fixed:** Hermes still routes its
|
||||
`dankserver-small` auxiliary tasks at the id `frank-medium` on
|
||||
`192.168.0.235:7770`, an endpoint that has served `frank-primary` since
|
||||
2026-09-20 — those routes were dead before this rename and remain dead;
|
||||
they need a routing decision (archon capacity), not a rename.
|
||||
- **Deployer blind spot found by accident:** every
|
||||
`deploy.sh dankserver --service frank-primary` since 2026-10-08 04:09
|
||||
restarted the **staging worktree**, not the deploy checkout — a campaign
|
||||
drop-in `frank-primary.service.d/bench.conf` overrode `Environment=` and
|
||||
`ExecStart`. The health gate went green each time. Verified parity before
|
||||
removing it (serve script + unit byte-identical, env identical except the
|
||||
alias, staging HEAD an ancestor of `main`), moved the drop-in to
|
||||
`/home/dank/.AGENTTRASH/20261009-frank-primary-bench-dropin/`, and
|
||||
restarted: `/v1/models` then answered `frank-archon`. Post-deploy
|
||||
behavioural proof is now part of the runbook.
|
||||
- Repo: seams re-pinned (`frank_secondary`/`frank_large`/`frank_primary`
|
||||
unit bats), fleet harnesses retargeted (`bench-probe`, `deep-probe`,
|
||||
`calib-push`, `fleet-probes-swift15`, `autoresearch*`,
|
||||
`bench-threads-batch-http`, `bench-decode`), `shared/homepage/docs.html`
|
||||
now teaches the ids it ships (curl/opencode/Pi send `frank-zealot`),
|
||||
`AGENTS.md`, [[OMP Harness]], [[Homepage]], [[OpenWebUI]] and the three
|
||||
endpoint runbooks updated. Suites: `./tests/run.sh` 366 green.
|
||||
- **Transcript secret incident:** dumping `podman exec hermes-webui env`
|
||||
printed the Porkbun API key + secret key into this session's transcript —
|
||||
same class as the 2026-09-16 HF-token-in-`ps` leak. Not committed
|
||||
anywhere; rotation is Frank's call.
|
||||
|
||||
## 2026-10-08
|
||||
- **frank-primary (Dankserver): Swift1.5-Qwen3.8-Flash-Next GPTQ w4 —
|
||||
the radiance quantization campaign.** ukisai's BF16 checkpoint (335.3
|
||||
GiB, 397/397 sha256-verified) → `qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||||
(~114 GiB): all 49,152 experts GPTQ w4 (act-order, damp 0.05) calibrated
|
||||
on 10,000,000 live-model tokens, 24 whole bf16-protected experts per
|
||||
layer derived from the calibration grams' energy share, int8 KV, fp8 HC
|
||||
mixtures. Engine pins: **1.0.13** for serve/convert/KLD, **1.2.0** only
|
||||
for calibration (the sole pin exposing `RADIANCE_CALIB_TOKENS`; the
|
||||
RADGRAM2 gram format is byte-identical across pins, so grams cross the
|
||||
boundary by design). Chain scripts in `scripts/`:
|
||||
`make-calib-corpus.py`, `run-swift15-bf16.sh` (bf16 reference: KLD record
|
||||
+ calib tap), `calib-push.py`, `derive-protected-experts.py`,
|
||||
`apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`,
|
||||
`kld-gate-swift15.sh`, `fleet-probes-swift15.sh`.
|
||||
- **Engine laws learned (measured, not inferred).** (1) The qwen4exp_fp8
|
||||
plugin declares **one encoding per expert table**: a recipe may NOT split
|
||||
`ffn_gate_up_exps` (w4) from `ffn_down_exps` (block-fp8) — the serve
|
||||
kernel is a single w4nl64a8h table for the whole expert set; the only
|
||||
legal beside-table content is whole bf16 experts (the PROTECTED section).
|
||||
(2) w4 expert tables stream from the file tier **in place only at
|
||||
MNBT ≤ 2048**; at MNBT 32768 the engine demands a widened form and the
|
||||
disk tier refuses (`DID NOT FIT … stored form is not the container's`).
|
||||
(3) The calibration tap lives in the architecture plugin
|
||||
(`/opt/radiance/share/radiance/architectures/qwen4exp_fp8.so`), not the
|
||||
binary — `grep -ac` the `.so` before declaring a flag missing.
|
||||
(4) Disk-tier boots need ~9.6 GiB of weight buffers + arena charged to
|
||||
the VRAM weights slab; explicit `--vram-weights-mib`/`--vram-kv-mib`
|
||||
pairs replaced the fp8-era EVC ratio.
|
||||
- **Throughput gate PASSED with 4.5× margin** (campaign boot, TP=2, int8
|
||||
KV, MTP 3): decode **136.0 t/s median** (122–173 across probes) at
|
||||
44.5k-token depth vs the 30.8 t/s bar; MTP acceptance 0.725; warm TTFT
|
||||
9.45 s; prefill 6.4k t/s. The fp8-native control on 1.0.13 measured
|
||||
30.08 t/s decode, and the 1.2.0 engine re-baseline was REJECTED
|
||||
(29.21 t/s, acceptance 0.7123 < 0.7549) — serve stays on 1.0.13.
|
||||
- **Deep-output audit:** a 44.5k-token verbatim-recall probe enumerated all
|
||||
48 planted documents in order — the DDR5-corruption signature is absent.
|
||||
- Fleet probes (`scripts/fleet-probes-swift15.sh`): tools, reasoning
|
||||
levels x4, reasoning_content, response_format, vision — all PASS.
|
||||
KLD gate vs the BF16 reference: recorded on 1.0.13 (50,235 positions,
|
||||
197 prompts), scored on the same pin at the w4-legal shape.
|
||||
Verdict: see runbook [[frank-primary-radiance-swift15-w4dn8]].
|
||||
|
||||
## 2026-10-06
|
||||
- **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp,
|
||||
@@ -1446,3 +1562,26 @@ every restart since the cutover.
|
||||
so the honest claim is ~184 s of inductor/AOT build recovered per restart —
|
||||
not a proportional boot cut.
|
||||
|
||||
## 2026-10-09 — OpenWebUI picker: `frank-dragoon` enabled beside `frank-zealot`
|
||||
|
||||
- Per Frank: make `frank-dragoon` (service `frank-large`, halogen
|
||||
flash-server, `host.containers.internal:7770`) selectable in OpenWebUI. The
|
||||
connection (idx 0) already carried `model_ids: ["frank-dragoon"]` from the
|
||||
wire rename; it lacked the other two legs of the trio: `enable: false`, no
|
||||
`model` row, no `access_grant`.
|
||||
- Applied on the live DB (snapshot `webui.db.bak-20261009b`): idx 0
|
||||
`enable: true`; `model` row `frank-dragoon` with
|
||||
`params.context.length = 262144` (per-slot `HALOGEN_CTX`) and the v2
|
||||
abliterated description; `access_grant`
|
||||
`(model, frank-dragoon, user, *, read)`. Then
|
||||
`systemctl --user restart open-webui.service` (drops cached model list;
|
||||
browser JWTs die with it — expect one SSO re-round-trip per tab).
|
||||
- Verified: `/api/models` lists `frank-dragoon` as admin AND as the QA
|
||||
`role='user'` account (picker truth, key `data`); `POST
|
||||
/api/chat/completions` through the app answers `pong` at `max_tokens
|
||||
1500`. `/api/v1/models/list` reads `items: []` for connection-backed
|
||||
models on this 0.6.x image (pre-existing for `frank-zealot` too) — the
|
||||
battery in [[OpenWebUI]] is rewritten to gate on `/api/models`.
|
||||
- Docs: [[OpenWebUI]] Facts/LLM-backends/battery/history updated;
|
||||
`frank-archon` (idx 1, Dankserver) intentionally stays disabled.
|
||||
|
||||
@@ -5,6 +5,9 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>frank — admin</title>
|
||||
<base href="/">
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Rajdhani:wght@500;600;700&family=Share+Tech+Mono&display=swap">
|
||||
<link rel="stylesheet" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
@@ -16,7 +19,7 @@
|
||||
<main id="grid" class="sections"></main>
|
||||
<footer class="foot">
|
||||
<span>last master deployment: <span id="deployed-at"></span></span>
|
||||
<a href="/">home</a>
|
||||
<a class="foot-btn" href="/">home</a>
|
||||
</footer>
|
||||
<script src="deployed.js"></script>
|
||||
<script src="app.js"></script>
|
||||
|
||||
@@ -19,6 +19,7 @@ const BADGE_LABEL = {
|
||||
down: 'DOWN',
|
||||
'wip-open': 'WIP: OPEN',
|
||||
'wip-closed': 'WIP: CLOSED',
|
||||
'wip-down': 'WIP: DOWN',
|
||||
};
|
||||
|
||||
function load(url) {
|
||||
@@ -31,8 +32,8 @@ function esc(s) {
|
||||
});
|
||||
}
|
||||
|
||||
// The description: structured role/spec/use lines when the service defines
|
||||
// them (the LLM cards), else the plain description string.
|
||||
// The plain description for ordinary panels. (Model cards render through
|
||||
// renderModelInfo below, not through descHTML.)
|
||||
function descHTML(svc) {
|
||||
const d = svc.details;
|
||||
if (d && (d.role || d.spec || d.inputs || d.use)) {
|
||||
@@ -46,6 +47,88 @@ function descHTML(svc) {
|
||||
return esc(svc.description || '');
|
||||
}
|
||||
|
||||
// The Model Information component: one panel fusing every fleet model card.
|
||||
// Each model is an informational sub-card (smaller type, Protoss unit art,
|
||||
// unit-coloured flair); the quoted lines are buttons that play the matching
|
||||
// StarCraft unit voice from /sfx/. Audio files are sequential: a click plays
|
||||
// the whole queue (the Dragoon tagline is two lines spliced).
|
||||
let sfxQueue = [];
|
||||
let sfxPlaying = false;
|
||||
|
||||
function playSfx(files, el) {
|
||||
sfxQueue = sfxQueue.concat(files);
|
||||
if (sfxPlaying) return;
|
||||
sfxPlaying = true;
|
||||
el.classList.add('mi-speaking');
|
||||
const next = function () {
|
||||
const f = sfxQueue.shift();
|
||||
if (!f) { sfxPlaying = false; el.classList.remove('mi-speaking'); return; }
|
||||
const a = new Audio('/sfx/' + f);
|
||||
a.volume = 0.5;
|
||||
a.onended = next;
|
||||
a.onerror = next;
|
||||
a.play().then(function () {}, next);
|
||||
};
|
||||
next();
|
||||
}
|
||||
|
||||
function voiceBtn(text, files) {
|
||||
return '<button type="button" class="mi-voice" title="play voice" ' +
|
||||
'data-sfx="' + esc(files.join(',')) + '">▶ ' + esc(text) + '</button>';
|
||||
}
|
||||
|
||||
function miCard(svc, state) {
|
||||
const m = svc.model;
|
||||
const parts = ['<div class="mi-top">' +
|
||||
'<img class="mi-art" src="/img/unit-' + esc(m.unit) + '.jpg" alt="' + esc(m.unit) + '">' +
|
||||
'<div class="mi-head"><div class="mi-id"><span class="badge ' + state + '">' + BADGE_LABEL[state] + '</span>' +
|
||||
'<span class="mi-name">' + esc(svc.name) + '</span></div>' +
|
||||
'<div class="mi-role">' + esc(m.role) + '</div>' +
|
||||
(m.hf ? '<a class="mi-hf" href="' + esc(m.hf) +
|
||||
'" target="_blank" rel="noopener">model on huggingface ↗</a>' : '') +
|
||||
'</div></div>',
|
||||
voiceBtn(m.tagline, m.sfx)
|
||||
];
|
||||
if (m.equiv) parts.push('<div class="mi-equiv">' + esc(m.equiv) + '</div>');
|
||||
if (m.spec) parts.push('<div class="mi-spec">' + esc(m.spec) + '</div>');
|
||||
if (m.inputs && m.inputs.length) parts.push('<div class="mi-inputs">Inputs: ' + esc(m.inputs.join(' · ')) + '</div>');
|
||||
if (m.use) parts.push('<div class="mi-use">Use: ' + esc(m.use) + '</div>');
|
||||
if (m.compaction) parts.push('<div class="mi-comp">⚡ ' + esc(m.compaction) + '</div>');
|
||||
if (m.supervision) parts.push('<div class="mi-super">' + esc(m.supervision) + '</div>');
|
||||
if (m.permission) parts.push('<div class="mi-perm">⚠ ' + esc(m.permission) + '</div>');
|
||||
if (m.warn) parts.push('<details class="mi-warn"><summary>⚠ privacy note</summary>' +
|
||||
'<div class="warn-body">' + esc(m.warn) + '</div></details>');
|
||||
const card = document.createElement('article');
|
||||
card.className = 'mi-card mi-' + esc(m.unit);
|
||||
card.innerHTML = parts.join('');
|
||||
return card;
|
||||
}
|
||||
|
||||
function renderModelInfo(models) {
|
||||
const panel = document.createElement('div');
|
||||
panel.className = 'panel model-info';
|
||||
const head = document.createElement('div');
|
||||
head.className = 'mi-title';
|
||||
const title = document.createElement('span');
|
||||
title.textContent = 'Model Information';
|
||||
head.appendChild(title);
|
||||
const docsLink = document.createElement('a');
|
||||
docsLink.className = 'mi-docs-link';
|
||||
docsLink.href = '/docs#models';
|
||||
docsLink.textContent = 'AI guide \u2192';
|
||||
head.appendChild(docsLink);
|
||||
panel.appendChild(head);
|
||||
const grid = document.createElement('div');
|
||||
grid.className = 'mi-grid';
|
||||
for (const m of models) grid.appendChild(m.el);
|
||||
panel.appendChild(grid);
|
||||
panel.addEventListener('click', function (ev) {
|
||||
const btn = ev.target.closest('.mi-voice');
|
||||
if (btn) playSfx(btn.getAttribute('data-sfx').split(','), btn);
|
||||
});
|
||||
return panel;
|
||||
}
|
||||
|
||||
// Shared panel component: one TUI panel for a service in a given state.
|
||||
function renderPanel(svc, state) {
|
||||
const panel = document.createElement('div');
|
||||
@@ -53,13 +136,25 @@ function renderPanel(svc, state) {
|
||||
const links = [];
|
||||
if (svc.showLink && svc.url) links.push('<a class="open" href="' + esc(svc.url) + '">open</a>');
|
||||
if (svc.docs) links.push('<a class="docs" href="' + esc(svc.docs) + '">docs</a>');
|
||||
const art = svc.art ? '<img class="app-art" src="' + esc(svc.art) + '" alt=""' +
|
||||
(svc.art_bg ? ' style="background:' + esc(svc.art_bg) + ';border-color:' + esc(svc.art_bg) + '"' : '') + '>' : '';
|
||||
const warn = svc.warn
|
||||
? '<details class="svc-warn"><summary>⚠ privacy note</summary>' +
|
||||
'<div class="warn-body">' + esc(svc.warn) + '</div></details>'
|
||||
: '';
|
||||
panel.innerHTML =
|
||||
'<div class="panel-head">' +
|
||||
'<div class="panel-head">' + art +
|
||||
' <span class="badge ' + state + '">' + BADGE_LABEL[state] + '</span>' +
|
||||
' <span class="name">' + esc(svc.name) + '</span>' +
|
||||
'</div>' +
|
||||
'<div class="desc">' + descHTML(svc) + '</div>' +
|
||||
'<div class="links">' + links.join('') + '</div>';
|
||||
'<div class="links">' + links.join('') + '</div>' + warn;
|
||||
if (svc.showLink && svc.url) {
|
||||
panel.classList.add('clickable');
|
||||
panel.addEventListener('click', function (ev) {
|
||||
if (ev.target.closest('a, .svc-warn')) return;
|
||||
});
|
||||
}
|
||||
return panel;
|
||||
}
|
||||
|
||||
@@ -87,15 +182,22 @@ function renderPanels(services, status) {
|
||||
let up = 0, down = 0, wip = 0;
|
||||
|
||||
const bySection = {};
|
||||
const modelsBySection = {};
|
||||
for (const svc of services) {
|
||||
if (svc.category !== CATEGORY) continue;
|
||||
const state = svc.wip ? 'wip-' + svc.wip : (stateByName[svc.name] || 'down');
|
||||
if (state === 'up') up++; else if (state === 'down') down++; else wip++;
|
||||
if (svc.model) {
|
||||
(modelsBySection[svc.section] = modelsBySection[svc.section] || []).push({ svc: svc, el: miCard(svc, state) });
|
||||
continue;
|
||||
}
|
||||
(bySection[svc.section] = bySection[svc.section] || []).push(renderPanel(svc, state));
|
||||
}
|
||||
|
||||
for (const title of SECTION_ORDER[CATEGORY]) {
|
||||
if (bySection[title]) root.appendChild(renderSection(title, bySection[title]));
|
||||
const panels = bySection[title] || [];
|
||||
if (modelsBySection[title]) panels.push(renderModelInfo(modelsBySection[title]));
|
||||
if (panels.length) root.appendChild(renderSection(title, panels));
|
||||
}
|
||||
|
||||
document.getElementById('summary').textContent = 'UP ' + up + ' DOWN ' + down + ' WIP ' + wip;
|
||||
@@ -118,9 +220,25 @@ function fillDeployed() {
|
||||
if (el && window.DEPLOYED_AT) el.textContent = window.DEPLOYED_AT;
|
||||
}
|
||||
|
||||
// Show the admin button only to members of the authentik homepage-admins
|
||||
// group (the same group the /admin* server gate enforces). The data comes
|
||||
// from Caddy's /whoami, which echoes the outpost-injected identity headers.
|
||||
function gateAdminLink() {
|
||||
const link = document.getElementById('admin-link');
|
||||
if (!link) return;
|
||||
fetch('/whoami', { headers: { accept: 'application/json' } })
|
||||
.then(function (r) { return r.ok ? r.json() : null; })
|
||||
.then(function (me) {
|
||||
if (me && /homepage-admins/i.test(me.groups || '')) link.hidden = false;
|
||||
})
|
||||
.catch(function () {});
|
||||
}
|
||||
|
||||
function main() {
|
||||
applyBare();
|
||||
fillDeployed();
|
||||
gateAdminLink();
|
||||
|
||||
Promise.all([load('services.json'), load('status.json')])
|
||||
.then(function (pair) { renderPanels(pair[0], pair[1]); })
|
||||
.catch(function (e) {
|
||||
|
||||
@@ -5,6 +5,9 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>frank — docs</title>
|
||||
<base href="/">
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Rajdhani:wght@500;600;700&family=Share+Tech+Mono&display=swap">
|
||||
<link rel="stylesheet" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
@@ -29,67 +32,6 @@
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="chat">
|
||||
<h1>Talking to the AI</h1>
|
||||
<p>The everyday front door is <a href="https://ai.franksdank.cloud">Open WebUI</a>
|
||||
(it has its own login the first time; use the same email you use everywhere).
|
||||
Pick a model in the chat's model selector — the three models the fleet runs are
|
||||
described <a href="#models">below</a>. Open WebUI keeps your chat history; the raw
|
||||
model endpoints do not.</p>
|
||||
<p>Images go to <code>frank-primary</code> or <code>frank-secondary</code>
|
||||
(both see them). Audio (WAV) goes to <code>frank-secondary</code> only.</p>
|
||||
</section>
|
||||
|
||||
<section id="models">
|
||||
<h1>The running models</h1>
|
||||
<p>Three models serve the family. The homepage status cards show whether
|
||||
each one is up right now.</p>
|
||||
|
||||
<h2>frank-large — deep reasoning</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> Qwen3.8-27B (UD-Q8_K_XL, 31.5 GiB).</li>
|
||||
<li><strong>Capacity:</strong> 262k-token context, exactly one conversation at a time. Text only.</li>
|
||||
<li><strong>Inputs:</strong> text.</li>
|
||||
<li><strong>Best for:</strong> cybersecurity and debugging. <em>Use requires Frank's permission.</em></li>
|
||||
<li><strong>Not for:</strong> images. It cannot see them.</li>
|
||||
</ul>
|
||||
|
||||
<h2>frank-primary — the primary model</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> Qwen3.8-Flash-Next (Q8_0, 175.3 GiB) with MTP + n-gram speculative drafting on both R9700s.</li>
|
||||
<li><strong>Capacity:</strong> 262k-token context, exactly one conversation at a time; sees images.</li>
|
||||
<li><strong>Inputs:</strong> text, image.</li>
|
||||
<li><strong>Best for:</strong> software engineering, system administration, and instruction tasks.</li>
|
||||
</ul>
|
||||
|
||||
<h2>frank-secondary — the fast utility model</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> Gemma4-12B QAT (Balanced, Q4_K_M, 7.4 GiB).</li>
|
||||
<li><strong>Capacity:</strong> one million tokens of pooled context across four simultaneous conversations (up to 262k each); fastest at images.</li>
|
||||
<li><strong>Inputs:</strong> text, image, audio.</li>
|
||||
<li><strong>Best for:</strong> agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture (screenshots, photos, diagrams).</li>
|
||||
</ul>
|
||||
|
||||
<p class="foot-note">Which one to pick? Small utility work, RAG, search, and
|
||||
screenshots: <code>frank-secondary</code>. Software engineering, sysadmin, and
|
||||
instruction tasks: <code>frank-primary</code>. Cybersecurity and debugging:
|
||||
<code>frank-large</code> (with Frank's permission).</p>
|
||||
</section>
|
||||
|
||||
<section id="media">
|
||||
<h1>Media & files</h1>
|
||||
<ul>
|
||||
<li><strong>Movies & TV:</strong> request with <a href="https://seerr.franksdank.cloud">Seerr</a>;
|
||||
watch with <a href="https://plex.franksdank.cloud/web/index.html">Plex</a> or
|
||||
<a href="https://jelly.franksdank.cloud">Jellyfin</a>. Don't ask the *arrs directly —
|
||||
Seerr is the front door, and requests ride the tunnel from there.</li>
|
||||
<li><strong>Books:</strong> <a href="https://books.franksdank.cloud">Calibre-Web</a> — the e-book library, with its own login.</li>
|
||||
<li><strong>Games:</strong> <a href="https://romm.franksdank.cloud">RomM</a> — ROM library and emulator collection.</li>
|
||||
<li><strong>Raw files:</strong> <a href="https://files.franksdank.cloud">Copyparty</a> — direct downloads and shares.</li>
|
||||
<li><strong>Search:</strong> <a href="https://search.franksdank.cloud">SearXNG</a> — metasearch, VPN-routed, no AI snoop.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="troubleshooting">
|
||||
<h1>When something looks broken</h1>
|
||||
<ul>
|
||||
@@ -97,15 +39,171 @@
|
||||
re-probes every few minutes. If it stays down, that service really is unhappy — tell Frank.</li>
|
||||
<li><strong>"status unavailable" on the homepage.</strong> The page couldn't read
|
||||
its status file. Hard-reload (Ctrl/Cmd-Shift-R).</li>
|
||||
<li><strong>Connection timeout on a home device.</strong> Nine times out of ten
|
||||
the device isn't using the house DNS (Pi-hole at 192.168.0.143) and resolved
|
||||
our names to the public address, which doesn't answer. Point the device's DNS at
|
||||
192.168.0.143 and retry.</li>
|
||||
<li><strong>Connection timeout on any device.</strong> Our names only resolve
|
||||
on the home network (or the VPN). If you are on guest Wi-Fi, a phone hotspot,
|
||||
or out of the house, the pages will not load — that is the security working.
|
||||
Connect to the house network and retry.</li>
|
||||
<li><strong>The bot-check spins forever.</strong> Let it finish; heavy JS blocking
|
||||
(privacy extensions) can starve it. Pause the blocker, reload, resume.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="media">
|
||||
<h1>Media & files</h1>
|
||||
<ul>
|
||||
<li><strong>Movies & TV:</strong> request with <a href="https://seerr.franksdank.cloud">Seerr</a>;
|
||||
watch with <a href="https://plex.franksdank.cloud/web/index.html">Plex</a> or
|
||||
<a href="https://jelly.franksdank.cloud">Jellyfin</a>. Seerr is the front door —
|
||||
requests ride the VPN tunnel from there.</li>
|
||||
<li><strong>Books:</strong> <a href="https://books.franksdank.cloud">Calibre-Web</a> — the e-book library, with its own login.</li>
|
||||
<li><strong>Games:</strong> <a href="https://romm.franksdank.cloud">RomM</a> — ROM library and emulator collection.</li>
|
||||
<li><strong>Raw files:</strong> <a href="https://files.franksdank.cloud">Copyparty</a> — direct downloads and shares.</li>
|
||||
<li><strong>Search:</strong> <a href="https://search.franksdank.cloud">SearXNG</a> — metasearch, VPN-routed, no AI snoop. <details class="key-warn"><summary>⚠ privacy note</summary><div class="warn-body">Searches are obfuscated, not anonymous: SearXNG hides you from the upstream engines and the VPN tunnel hides the traffic from the LAN — but the upstream search API keys are tied to Frank’s identity. Don’t treat it as Tor.</div></details></li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="chat">
|
||||
<h1>Talking to the AI</h1>
|
||||
<p>The everyday front door is <a href="https://ai.franksdank.cloud">Open WebUI</a>,
|
||||
the family chat app. Open the address, click <strong>Continue with authentik</strong>,
|
||||
and use the same single sign-on as everything else — no separate password. Your
|
||||
conversations are saved to your account, so history follows you between devices.</p>
|
||||
<img src="img/openwebui-chat.png" alt="Open WebUI chat screen with a question and the AI's answer">
|
||||
<p>Below the text box, the model box on the right chooses which model answers you.
|
||||
Family accounts get <code>frank-zealot</code> — the fleet's fast all-rounder,
|
||||
which also sees images: drag a photo or screenshot straight into the chat box.
|
||||
Frank's account can additionally reach the bigger models.</p>
|
||||
<img src="img/openwebui-model-picker.png" alt="The Open WebUI welcome screen with the family model selected in the model box">
|
||||
<p>Open WebUI keeps your chat history; the raw model endpoints (used by developer
|
||||
tools, see <a href="#api-access">API access</a>) do not.</p>
|
||||
</section>
|
||||
|
||||
<section id="models">
|
||||
<h1>The running models</h1>
|
||||
<p>Three models serve the family, named for the Protoss units of StarCraft.
|
||||
The homepage "Model Information" panel shows whether each one is up right now —
|
||||
and plays the unit's voice if you click its quoted line.</p>
|
||||
|
||||
<p class="foot-note">Naming: the Open WebUI picker and the raw API show
|
||||
exactly the Protoss names used on this page (renamed 2026-10-09). The name you
|
||||
read here is the name you type into a config file.</p>
|
||||
|
||||
<h2>frank-zealot — the fast utility model</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> <a href="https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive" target="_blank" rel="noopener">Qwen3.5-9B-Uncensored</a> (Q6_K) on the RTX 3090 — roughly 95 tokens per second.</li>
|
||||
<li><strong>Equivalent:</strong> last year's frontier level (2025).</li>
|
||||
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
|
||||
<li><strong>Inputs:</strong> text, image.</li>
|
||||
<li><strong>Best for:</strong> code editor autocomplete, agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture. This is the model Open WebUI offers to family accounts.</li>
|
||||
<li><strong>Compaction:</strong> if your agent harness supports setting a compaction model, set this one. A compaction job severely slows the bigger models for far longer than it slows this one.</li>
|
||||
<li><strong>Supervision:</strong> requires tight human supervision.</li>
|
||||
<li><strong>Warning:</strong> <details class="key-warn"><summary>⚠ uncensored model</summary><div class="warn-body">Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.</div></details></li>
|
||||
</ul>
|
||||
|
||||
<h2>frank-dragoon — deep reasoning</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> <a href="https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated" target="_blank" rel="noopener">Qwen3.8-Flash-Next v2 Swift1.5 abliterated</a> (halogen flash-server, ~62 GiB) on the Strix Halo iGPU — an abliterated (censorship-removed) edition that thinks before answering.</li>
|
||||
<li><strong>Equivalent:</strong> Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier intelligence).</li>
|
||||
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
|
||||
<li><strong>Inputs:</strong> text, image.</li>
|
||||
<li><strong>Best for:</strong> complex agentic tasks, programming, system administration, and "claw"-style personal agents.</li>
|
||||
<li><strong>Supervision:</strong> safe for autonomous, low-supervision use.</li>
|
||||
<li><strong>Warning:</strong> <details class="key-warn"><summary>⚠ uncensored model</summary><div class="warn-body">Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.</div></details></li>
|
||||
|
||||
<h2>frank-archon — the big one</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> our own build — a GPTQ-w4 quant of <a href="https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700" target="_blank" rel="noopener">Swift1.5-Qwen3.8-Flash-Next</a>, quantized and tuned in-house (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — the biggest brain on the network and the fastest runtime.</li>
|
||||
<li><strong>Equivalent:</strong> Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).</li>
|
||||
<li><strong>Capacity:</strong> 200k-token context, three submissions at a time; sees images.</li>
|
||||
<li><strong>Inputs:</strong> text, image.</li>
|
||||
<li><strong>Best for:</strong> software engineering, system administration, and instruction tasks. <strong>⚠ Ask Frank for permission before use</strong> — this is his primary model; he uses it more than anything else on the fleet.</li>
|
||||
</ul>
|
||||
|
||||
<p class="foot-note">Which one to pick? Small utility work, autocomplete, RAG,
|
||||
programming, and personal agents: <code>frank-dragoon</code>. Software
|
||||
engineering, sysadmin, and instruction tasks: <code>frank-archon</code> (ask
|
||||
Frank first). Every name on this page is the id the Open WebUI picker and
|
||||
the raw API answer to.</p>
|
||||
</section>
|
||||
|
||||
<section id="api-access">
|
||||
<h1>API access — the AI on your own laptop</h1>
|
||||
<p>Developer assistants like <strong>Opencode</strong> and <strong>Pi</strong> are
|
||||
chat programs that run on your own computer and can read your files, run commands,
|
||||
and edit code while they talk. They are all built to speak the same standard
|
||||
language as the big commercial products (the "OpenAI-compatible" API), and our
|
||||
server speaks it too. You give the program three things — an address, a personal
|
||||
key, and a model name — and your laptop thinks through our hardware.</p>
|
||||
|
||||
<h2>Step 1 — get your personal key</h2>
|
||||
<ol>
|
||||
<li>Open <a href="https://ai.franksdank.cloud">ai.franksdank.cloud</a> and log in.</li>
|
||||
<li>Click your circle (bottom-left corner) → <strong>Settings</strong> → <strong>Account</strong>.</li>
|
||||
<li>Scroll to <strong>API keys</strong> → press <strong>Show</strong> (it flips to
|
||||
<strong>Hide</strong>) — the key appears as a row of dots.</li>
|
||||
<li>Press the <strong>copy icon</strong> beside the key (the eye previews it;
|
||||
the <strong>⋯</strong> menu rotates it later). The key looks like <code>sk-3f2a…</code>.</li>
|
||||
</ol>
|
||||
<img src="img/openwebui-api-keys.png" alt="The Open WebUI Account settings page, scrolled to the API keys section with the masked key, preview, copy and rotate controls">
|
||||
<p class="key-warn">The key is a password. Anyone who has it can use our AI as
|
||||
you. Paste it only into the program's config file — never into a shared chat,
|
||||
a screenshot, or a git repository.</p>
|
||||
|
||||
<h2>Step 2 — the three details every tool asks for</h2>
|
||||
<ul>
|
||||
<li><strong>Base URL:</strong> <code>https://ai.franksdank.cloud/api/v1</code></li>
|
||||
<li><strong>Model:</strong> <code>frank-zealot</code> — the family utility model (this is the id the API expects)</li>
|
||||
<li><strong>API key:</strong> the <code>sk-…</code> from step 1 — sent as the header <code>Authorization: Bearer sk-…</code></li>
|
||||
</ul>
|
||||
<p>Proof that it works, from any terminal (replace the key):</p>
|
||||
<pre><code>curl https://ai.franksdank.cloud/api/v1/chat/completions \
|
||||
-H "Authorization: Bearer PASTE-YOUR-KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "frank-zealot", "messages": [{"role": "user", "content": "Hello!"}]}'</code></pre>
|
||||
|
||||
<h2>Step 3a — Opencode</h2>
|
||||
<p>Install once (<code>npm install -g opencode-ai</code>), then create the file
|
||||
<code>~/.config/opencode/opencode.json</code> with:</p>
|
||||
<pre><code>{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"frank": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"name": "Frank fleet",
|
||||
"options": {
|
||||
"baseURL": "https://ai.franksdank.cloud/api/v1",
|
||||
"apiKey": "PASTE-YOUR-KEY"
|
||||
},
|
||||
"models": {
|
||||
"frank-zealot": { "name": "frank-zealot" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}</code></pre>
|
||||
<p>Start <code>opencode</code> and pick the model with <code>/model</code> —
|
||||
it appears as <code>frank/frank-zealot</code>.</p>
|
||||
|
||||
<h2>Step 3b — Pi</h2>
|
||||
<p>Create the file <code>~/.pi/agent/models.json</code> with:</p>
|
||||
<pre><code>{
|
||||
"providers": {
|
||||
"frank": {
|
||||
"baseUrl": "https://ai.franksdank.cloud/api/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "PASTE-YOUR-KEY",
|
||||
"models": [
|
||||
{ "id": "frank-zealot" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}</code></pre>
|
||||
<p>Then start Pi with <code>pi --model frank/frank-zealot</code>.</p>
|
||||
|
||||
<p class="foot-note">The key works only for the family AI — it is not your
|
||||
Authentik password, and losing it costs nothing beyond asking Frank to rotate it.
|
||||
Want a bigger model (<code>frank-archon</code>, <code>frank-dragoon</code>)? Ask Frank to raise your access.</p>
|
||||
</section>
|
||||
|
||||
</main>
|
||||
<footer class="foot">
|
||||
<span><a href="/">← services</a></span>
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48">
|
||||
<path d="M6 10c6-3 12-3 18 0v28c-6-3-12-3-18 0z" fill="#ffb26b" stroke="#e66d00" stroke-width="2"/>
|
||||
<path d="M42 10c-6-3-12-3-18 0v28c6-3 12-3 18 0z" fill="#ffd9ae" stroke="#e66d00" stroke-width="2"/>
|
||||
<path d="M24 10v28" stroke="#7a3a00" stroke-width="2.5"/>
|
||||
<path d="M10 16h9M10 21h9M10 26h9M29 16h9M29 21h9M29 26h9" stroke="#e66d00" stroke-width="1.6" opacity="0.75"/>
|
||||
<path d="M6 38c6-3 12-3 18 0 6-3 12-3 18 0v3c-6-3-12-3-18 0-6-3-12-3-18 0z" fill="#e66d00"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 542 B |
@@ -0,0 +1,6 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48">
|
||||
<path d="M5 12c0-2 1.5-3.5 3.5-3.5H18l4 4.5h14c2 0 3.5 1.5 3.5 3.5v3H5z" fill="#2c7ca6"/>
|
||||
<path d="M5 17h38v20c0 2-1.5 3.5-3.5 3.5h-31C6.5 40.5 5 39 5 37z" fill="#58b4d8"/>
|
||||
<path d="M24 21v9m0-9l-4 4m4-4l4 4" stroke="#eaf7ff" stroke-width="2.4" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<path d="M18 34h12" stroke="#eaf7ff" stroke-width="2.2" stroke-linecap="round" opacity="0.85"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 478 B |
@@ -0,0 +1,11 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48">
|
||||
<rect x="9" y="4" width="30" height="40" rx="4" ry="4" fill="#9d6bd0" stroke="#5b3a86" stroke-width="2"/>
|
||||
<rect x="14" y="8" width="20" height="13" rx="1.5" fill="#3a2b52"/>
|
||||
<rect x="16" y="10" width="16" height="9" fill="#bfe8a4"/>
|
||||
<path d="M16 12h16M16 14h16M16 16h16" stroke="#8fbf74" stroke-width="0.8" opacity="0.6"/>
|
||||
<path d="M15 27h4v-4h4v4h4v4h-4v4h-4v-4h-4z" fill="#2a1f3d"/>
|
||||
<circle cx="33" cy="30" r="2.6" fill="#e05fa0"/>
|
||||
<circle cx="38" cy="27" r="2.6" fill="#e05fa0"/>
|
||||
<path d="M28 40l6-3" stroke="#2a1f3d" stroke-width="1.6" stroke-linecap="round"/>
|
||||
<path d="M29 41.5l6-3M30 43l6-3" stroke="#2a1f3d" stroke-width="1.2" stroke-linecap="round" opacity="0.8"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 759 B |
@@ -0,0 +1,13 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48">
|
||||
<g fill="#c96f4a" stroke="#5b2a18" stroke-width="1.4">
|
||||
<rect x="5" y="34" width="12" height="8"/>
|
||||
<rect x="18" y="34" width="12" height="8"/>
|
||||
<rect x="31" y="34" width="12" height="8"/>
|
||||
<rect x="5" y="25" width="18" height="8"/>
|
||||
<rect x="24" y="26.5" width="16" height="8" transform="rotate(-4 32 30)"/>
|
||||
<rect x="5" y="16" width="12" height="8"/>
|
||||
<rect x="18" y="17.5" width="13" height="8" transform="rotate(5 24 21)"/>
|
||||
</g>
|
||||
<path d="M36 12l-3 6 4 2-3 6" stroke="#e5534b" stroke-width="2.2" fill="none" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<path d="M38 8l2 3M42 14l2 2" stroke="#c96f4a" stroke-width="2" stroke-linecap="round"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 746 B |
@@ -0,0 +1 @@
|
||||
<svg fill="#1F88C0" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Caddy</title><path d="M11.094.47c-.842 0-1.696.092-2.552.288a11.37 11.37 0 0 0-4.87 2.423 10.632 10.632 0 0 0-2.36 2.826A10.132 10.132 0 0 0 .305 8.582c-.398 1.62-.4 3.336-.043 5.048.085.405.183.809.31 1.212a11.85 11.85 0 0 0 1.662 3.729 3.273 3.273 0 0 0-.086.427 3.323 3.323 0 0 0 2.848 3.71 3.279 3.279 0 0 0 1.947-.346c1.045.51 2.17.864 3.339 1.04a11.66 11.66 0 0 0 4.285-.155 11.566 11.566 0 0 0 4.936-2.485 10.643 10.643 0 0 0 2.352-2.894 11.164 11.164 0 0 0 1.356-4.424 11.214 11.214 0 0 0-.498-4.335c.175-.077.338-.175.486-.293a.444.444 89.992 0 0 .001 0c.402-.322.693-.794.777-1.342a2.146 2.146 0 0 0-1.79-2.434 2.115 2.115 0 0 0-1.205.171c-.038-.043-.078-.086-.113-.13a11.693 11.693 0 0 0-3.476-2.93 13.348 13.348 0 0 0-1.76-.81 13.55 13.55 0 0 0-2.06-.613A12.121 12.121 0 0 0 11.093.47Zm.714.328c.345-.004.688.01 1.028.042a9.892 9.892 0 0 1 2.743.639c.984.39 1.89.958 2.707 1.632.803.662 1.502 1.45 2.091 2.328.026.039.048.08.07.12a2.12 2.12 0 0 0-.435 2.646c-.158.114-.97.692-1.634 1.183-.414.308-.733.557-.733.557l.581.68s.296-.276.665-.638c.572-.562 1.229-1.233 1.395-1.403a2.122 2.122 0 0 0 1.907.677 11.229 11.229 0 0 1-.013 4.046 11.41 11.41 0 0 1-1.475 3.897 12.343 12.343 0 0 1-2.079 2.587c-1.19 1.125-2.633 2.022-4.306 2.531a10.826 10.826 0 0 1-3.973.484 11.04 11.04 0 0 1-3.057-.652 3.304 3.304 0 0 0 1.417-2.294 3.275 3.275 0 0 0-.294-1.842c.18-.162.403-.363.656-.6 1.015-.955 2.353-2.303 2.353-2.303l-.47-.599s-1.63.972-2.801 1.728c-.307.198-.573.378-.777.517a3.273 3.273 0 0 0-1.516-.611c-1.507-.198-2.927.672-3.487 2.017a10.323 10.323 0 0 1-.695-1.078A10.92 10.92 0 0 1 .728 14.8a10.35 10.35 0 0 1-.2-1.212c-.164-1.653.103-3.258.629-4.754a12.95 12.95 0 0 1 1.087-2.288c.57-.968 1.248-1.872 2.069-2.656A11.013 11.013 0 0 1 11.808.797Zm-.147 3.257a3.838 3.838 0 0 0-3.82 3.82v2.36h-.94c-.751 0-1.377.625-1.377 1.377v3.8h1.46v-3.718h9.354v6.264H10.02v1.46h6.4c.751 0 1.377-.625 1.377-1.377v-6.43c0-.751-.626-1.377-1.377-1.377h-.94v-2.36a3.838 3.838 0 0 0-3.82-3.819zm0 1.46a2.371 2.371 0 0 1 2.36 2.36v2.36H9.3v-2.36a2.372 2.372 0 0 1 2.36-2.36zm10.141.392a1.253 1.253 0 0 1 1.296 1.434c-.049.319-.217.59-.453.78-.266.213-.61.318-.968.264a1.253 1.253 0 0 1-1.045-1.42 1.255 1.255 0 0 1 1.17-1.058zM5.384 17.425a2.02 2.02 0 0 1 1.917 1.298c.116.3.159.628.114.967a2.015 2.015 0 0 1-2.249 1.728 2.016 2.016 0 0 1-1.727-2.25 2.017 2.017 0 0 1 1.945-1.743z"/></svg>
|
||||
|
After Width: | Height: | Size: 2.4 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#609926" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Gitea</title><path d="M4.209 4.603c-.247 0-.525.02-.84.088-.333.07-1.28.283-2.054 1.027C-.403 7.25.035 9.685.089 10.052c.065.446.263 1.687 1.21 2.768 1.749 2.141 5.513 2.092 5.513 2.092s.462 1.103 1.168 2.119c.955 1.263 1.936 2.248 2.89 2.367 2.406 0 7.212-.004 7.212-.004s.458.004 1.08-.394c.535-.324 1.013-.893 1.013-.893s.492-.527 1.18-1.73c.21-.37.385-.729.538-1.068 0 0 2.107-4.471 2.107-8.823-.042-1.318-.367-1.55-.443-1.627-.156-.156-.366-.153-.366-.153s-4.475.252-6.792.306c-.508.011-1.012.023-1.512.027v4.474l-.634-.301c0-1.39-.004-4.17-.004-4.17-1.107.016-3.405-.084-3.405-.084s-5.399-.27-5.987-.324c-.187-.011-.401-.032-.648-.032zm.354 1.832h.111s.271 2.269.6 3.597C5.549 11.147 6.22 13 6.22 13s-.996-.119-1.641-.348c-.99-.324-1.409-.714-1.409-.714s-.73-.511-1.096-1.52C1.444 8.73 2.021 7.7 2.021 7.7s.32-.859 1.47-1.145c.395-.106.863-.12 1.072-.12zm8.33 2.554c.26.003.509.127.509.127l.868.422-.529 1.075a.686.686 0 0 0-.614.359.685.685 0 0 0 .072.756l-.939 1.924a.69.69 0 0 0-.66.527.687.687 0 0 0 .347.763.686.686 0 0 0 .867-.206.688.688 0 0 0-.069-.882l.916-1.874a.667.667 0 0 0 .237-.02.657.657 0 0 0 .271-.137 8.826 8.826 0 0 1 1.016.512.761.761 0 0 1 .286.282c.073.21-.073.569-.073.569-.087.29-.702 1.55-.702 1.55a.692.692 0 0 0-.676.477.681.681 0 1 0 1.157-.252c.073-.141.141-.282.214-.431.19-.397.515-1.16.515-1.16.035-.066.218-.394.103-.814-.095-.435-.48-.638-.48-.638-.467-.301-1.116-.58-1.116-.58s0-.156-.042-.27a.688.688 0 0 0-.148-.241l.516-1.062 2.89 1.401s.48.218.583.619c.073.282-.019.534-.069.657-.24.587-2.1 4.317-2.1 4.317s-.232.554-.748.588a1.065 1.065 0 0 1-.393-.045l-.202-.08-4.31-2.1s-.417-.218-.49-.596c-.083-.31.104-.691.104-.691l2.073-4.272s.183-.37.466-.497a.855.855 0 0 1 .35-.077z"/></svg>
|
||||
|
After Width: | Height: | Size: 1.8 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#F46800" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Grafana</title><path d="M23.02 10.59a8.578 8.578 0 0 0-.862-3.034 8.911 8.911 0 0 0-1.789-2.445c.337-1.342-.413-2.505-.413-2.505-1.292-.08-2.113.4-2.416.62-.052-.02-.102-.044-.154-.064-.22-.089-.446-.172-.677-.247-.231-.073-.47-.14-.711-.197a9.867 9.867 0 0 0-.875-.161C14.557.753 12.94 0 12.94 0c-1.804 1.145-2.147 2.744-2.147 2.744l-.018.093c-.098.029-.2.057-.298.088-.138.042-.275.094-.413.143-.138.055-.275.107-.41.166a8.869 8.869 0 0 0-1.557.87l-.063-.029c-2.497-.955-4.716.195-4.716.195-.203 2.658.996 4.33 1.235 4.636a11.608 11.608 0 0 0-.607 2.635C1.636 12.677.953 15.014.953 15.014c1.926 2.214 4.171 2.351 4.171 2.351.003-.002.006-.002.006-.005.285.509.615.994.986 1.446.156.19.32.371.488.548-.704 2.009.099 3.68.099 3.68 2.144.08 3.553-.937 3.849-1.173a9.784 9.784 0 0 0 3.164.501h.08l.055-.003.107-.002.103-.005.003.002c1.01 1.44 2.788 1.646 2.788 1.646 1.264-1.332 1.337-2.653 1.337-2.94v-.058c0-.02-.003-.039-.003-.06.265-.187.52-.387.758-.6a7.875 7.875 0 0 0 1.415-1.7c1.43.083 2.437-.885 2.437-.885-.236-1.49-1.085-2.216-1.264-2.354l-.018-.013-.016-.013a.217.217 0 0 1-.031-.02c.008-.092.016-.18.02-.27.011-.162.016-.323.016-.48v-.253l-.005-.098-.008-.135a1.891 1.891 0 0 0-.01-.13c-.003-.042-.008-.083-.013-.125l-.016-.124-.018-.122a6.215 6.215 0 0 0-2.032-3.73 6.015 6.015 0 0 0-3.222-1.46 6.292 6.292 0 0 0-.85-.048l-.107.002h-.063l-.044.003-.104.008a4.777 4.777 0 0 0-3.335 1.695c-.332.4-.592.84-.768 1.297a4.594 4.594 0 0 0-.312 1.817l.003.091c.005.055.007.11.013.164a3.615 3.615 0 0 0 .698 1.82 3.53 3.53 0 0 0 1.827 1.282c.33.098.66.14.971.137.039 0 .078 0 .114-.002l.063-.003c.02 0 .041-.003.062-.003.034-.002.065-.007.099-.01.007 0 .018-.003.028-.003l.031-.005.06-.008a1.18 1.18 0 0 0 .112-.02c.036-.008.072-.013.109-.024a2.634 2.634 0 0 0 .914-.415c.028-.02.056-.041.085-.065a.248.248 0 0 0 .039-.35.244.244 0 0 0-.309-.06l-.078.042c-.09.044-.184.083-.283.116a2.476 2.476 0 0 1-.475.096c-.028.003-.054.006-.083.006l-.083.002c-.026 0-.054 0-.08-.002l-.102-.006h-.012l-.024.006c-.016-.003-.031-.003-.044-.006-.031-.002-.06-.007-.091-.01a2.59 2.59 0 0 1-.724-.213 2.557 2.557 0 0 1-.667-.438 2.52 2.52 0 0 1-.805-1.475 2.306 2.306 0 0 1-.029-.444l.006-.122v-.023l.002-.031c.003-.021.003-.04.005-.06a3.163 3.163 0 0 1 1.352-2.29 3.12 3.12 0 0 1 .937-.43 2.946 2.946 0 0 1 .776-.101h.06l.07.002.045.003h.026l.07.005a4.041 4.041 0 0 1 1.635.49 3.94 3.94 0 0 1 1.602 1.662 3.77 3.77 0 0 1 .397 1.414l.005.076.003.075c.002.026.002.05.002.075 0 .024.003.052 0 .07v.065l-.002.073-.008.174a6.195 6.195 0 0 1-.08.639 5.1 5.1 0 0 1-.267.927 5.31 5.31 0 0 1-.624 1.13 5.052 5.052 0 0 1-3.237 2.014 4.82 4.82 0 0 1-.649.066l-.039.003h-.287a6.607 6.607 0 0 1-1.716-.265 6.776 6.776 0 0 1-3.4-2.274 6.75 6.75 0 0 1-.746-1.15 6.616 6.616 0 0 1-.714-2.596l-.005-.083-.002-.02v-.056l-.003-.073v-.096l-.003-.104v-.07l.003-.163c.008-.22.026-.45.054-.678a8.707 8.707 0 0 1 .28-1.355c.128-.444.286-.872.473-1.277a7.04 7.04 0 0 1 1.456-2.1 5.925 5.925 0 0 1 .953-.763c.169-.111.343-.213.524-.306.089-.05.182-.091.273-.135.047-.02.093-.042.138-.062a7.177 7.177 0 0 1 .714-.267l.145-.045c.049-.015.098-.026.148-.041.098-.029.197-.052.296-.076.049-.013.1-.02.15-.033l.15-.032.151-.028.076-.013.075-.01.153-.024c.057-.01.114-.013.171-.023l.169-.021c.036-.003.073-.008.106-.01l.073-.008.036-.003.042-.002c.057-.003.114-.008.171-.01l.086-.006h.023l.037-.003.145-.007a7.999 7.999 0 0 1 1.708.125 7.917 7.917 0 0 1 2.048.68 8.253 8.253 0 0 1 1.672 1.09l.09.077.089.078c.06.052.114.107.171.159.057.052.112.106.166.16.052.055.107.107.159.164a8.671 8.671 0 0 1 1.41 1.978c.012.026.028.052.04.078l.04.078.075.156c.023.051.05.1.07.153l.065.15a8.848 8.848 0 0 1 .45 1.34.19.19 0 0 0 .201.142.186.186 0 0 0 .172-.184c.01-.246.002-.532-.024-.856z"/></svg>
|
||||
|
After Width: | Height: | Size: 3.7 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#00A4DC" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Jellyfin</title><path d="M12 .002C8.826.002-1.398 18.537.16 21.666c1.56 3.129 22.14 3.094 23.682 0C25.384 18.573 15.177 0 12 0zm7.76 18.949c-1.008 2.028-14.493 2.05-15.514 0C3.224 16.9 9.92 4.755 12.003 4.755c2.081 0 8.77 12.166 7.759 14.196zM12 9.198c-1.054 0-4.446 6.15-3.93 7.189.518 1.04 7.348 1.027 7.86 0 .511-1.027-2.874-7.19-3.93-7.19z"/></svg>
|
||||
|
After Width: | Height: | Size: 445 B |
@@ -0,0 +1 @@
|
||||
<svg fill="#003545" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>MariaDB</title><path d="M23.157 4.412c-.676.284-.79.31-1.673.372-.65.045-.757.057-1.212.209-.75.246-1.395.75-2.02 1.59-.296.398-1.249 1.913-1.249 1.988 0 .057-.65.998-.915 1.32-.574.713-1.08 1.079-2.14 1.59-.77.36-1.224.524-4.102 1.477-1.073.353-2.133.738-2.367.864-.852.449-1.515 1.036-2.203 1.938-1.003 1.32-.972 1.313-3.042.947a12.264 12.264 0 00-.675-.063c-.644-.05-1.023.044-1.332.334L0 17.193l.177.088c.094.05.353.234.561.398.215.17.461.347.55.391.088.044.17.088.183.101.012.013-.089.17-.228.353-.435.581-.593.871-.574 1.048.019.164.032.17.43.17.517-.006.826-.056 1.261-.208.65-.233 2.058-.94 2.784-1.4.776-.5 1.717-.998 1.956-1.042.082-.02.354-.07.594-.114.58-.107 1.464-.095 2.587.05.108.013.373.045.6.064.227.025.43.057.454.076.026.012.474.037.998.056.934.026 1.104.007 1.3-.189.126-.133.385-.631.498-.985.209-.643.417-.921.366-.492-.113.966-.322 1.692-.713 2.411-.259.499-.663 1.092-.934 1.395-.322.347-.315.36.088.315.619-.063 1.471-.397 2.096-.82.827-.562 1.647-1.691 2.19-3.03.107-.27.22-.22.183.083-.013.094-.038.315-.057.498l-.031.328.353-.202c.833-.48 1.414-1.262 2.127-2.884.227-.518.877-2.922 1.073-3.976a9.64 9.64 0 01.271-1.042c.127-.429.196-.555.48-.858.183-.19.625-.555.978-.808.72-.505.953-.75 1.187-1.205.208-.417.284-1.13.132-1.357-.132-.202-.284-.196-.763.006Z"/></svg>
|
||||
|
After Width: | Height: | Size: 1.4 KiB |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#e5534b" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#e5534b" text-anchor="middle">AM</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#e5534b" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#e5534b" text-anchor="middle">AY</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ffc94d" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ffc94d" text-anchor="middle">CP</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#d946ef" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#d946ef" text-anchor="middle">CU</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#b56cff" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#b56cff" text-anchor="middle">CW</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#3fc1c9" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#3fc1c9" text-anchor="middle">GT</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#4fd6ff" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#4fd6ff" text-anchor="middle">HD</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#4fd6ff" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#4fd6ff" text-anchor="middle">HM</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ff7f00" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ff7f00" text-anchor="middle">LD</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#b96bff" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#b96bff" text-anchor="middle">LK</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#4fb4ff" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#4fb4ff" text-anchor="middle">MR</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#e5534b" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#e5534b" text-anchor="middle">NE</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ffb000" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ffb000" text-anchor="middle">OW</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ff7f00" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ff7f00" text-anchor="middle">PW</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ff7f00" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ff7f00" text-anchor="middle">RA</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1,11 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96">
|
||||
<polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#b56cff" stroke-width="3"/>
|
||||
<rect x="30" y="22" width="36" height="52" rx="4" fill="#b56cff"/>
|
||||
<rect x="35" y="27" width="26" height="18" rx="2" fill="#1c1300"/>
|
||||
<rect x="38" y="30" width="20" height="3" fill="#ffb000"/>
|
||||
<rect x="38" y="36" width="14" height="3" fill="#ffb000"/>
|
||||
<rect x="34" y="52" width="4" height="14" fill="#1c1300"/>
|
||||
<rect x="29" y="57" width="14" height="4" fill="#1c1300"/>
|
||||
<circle cx="57" cy="63" r="3.2" fill="#1c1300"/>
|
||||
<circle cx="64" cy="59" r="3.2" fill="#1c1300"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 639 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ffb000" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ffb000" text-anchor="middle">SR</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#3fc1c9" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#3fc1c9" text-anchor="middle">TT</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#ff3d45" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#ff3d45" text-anchor="middle">VK</text></svg>
|
||||
|
After Width: | Height: | Size: 292 B |
|
After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#96060C" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Pi-hole</title><path d="M4.344 0c.238 4.792 3.256 7.056 6.252 7.376.165-1.692-4.319-5.6-4.319-5.6-.008-.011.009-.025.019-.014 0 0 4.648 4.01 5.423 5.645 2.762-.15 5.196-1.947 5-4.912 0 0-4.12-.613-5 4.618C11.48 2.753 8.993 0 4.344 0zM12 7.682v.002a3.68 3.68 0 0 0-2.591 1.077L4.94 13.227a3.683 3.683 0 0 0-.86 1.356 3.31 3.31 0 0 0-.237 1.255A3.681 3.681 0 0 0 4.92 18.45l4.464 4.466a3.69 3.69 0 0 0 2.251 1.06l.002.001c.093.01.187.015.28.017l-.1-.008c.06.003.117.009.177.009l-.077-.001L12 24l-.004-.005a3.68 3.68 0 0 0 2.61-1.077l4.469-4.465a3.683 3.683 0 0 0 1.006-1.888l.012-.063a3.682 3.682 0 0 0 .057-.541l.003-.061c0-.017.003-.05.004-.06h-.002a3.683 3.683 0 0 0-1.077-2.607l-4.466-4.468a3.694 3.694 0 0 0-1.564-.927l-.07-.02a3.43 3.43 0 0 0-.946-.133L12 7.682zm3.165 3.357c.023 1.748-1.33 3.078-1.33 4.806.164 2.227 1.733 3.207 3.266 3.146-.035.003-.068.007-.104.009-1.847.135-3.209-1.326-5.002-1.326-2.23.164-3.21 1.736-3.147 3.27l-.008-.104c-.133-1.847 1.328-3.21 1.328-5.002-.173-2.32-1.867-3.284-3.46-3.132.1-.011.203-.021.31-.027 1.847-.133 3.209 1.328 5.002 1.328 2.082-.155 3.074-1.536 3.145-2.968zM4.344 0c.238 4.792 3.256 7.056 6.252 7.376.165-1.692-4.319-5.6-4.319-5.6-.008-.011.009-.025.019-.014 0 0 4.648 4.01 5.423 5.645 2.762-.15 5.196-1.947 5-4.912 0 0-4.12-.613-5 4.618C11.48 2.753 8.993 0 4.344 0zM12 7.682v.002a3.68 3.68 0 0 0-2.591 1.077L4.94 13.227a3.683 3.683 0 0 0-.86 1.356 3.31 3.31 0 0 0-.237 1.255A3.681 3.681 0 0 0 4.92 18.45l4.464 4.466a3.69 3.69 0 0 0 2.251 1.06l.002.001c.093.01.187.015.28.017l-.1-.008c.06.003.117.009.177.009l-.077-.001L12 24l-.004-.005a3.68 3.68 0 0 0 2.61-1.077l4.469-4.465a3.683 3.683 0 0 0 1.006-1.888l.012-.063a3.682 3.682 0 0 0 .057-.541l.003-.061c0-.017.003-.05.004-.06h-.002a3.683 3.683 0 0 0-1.077-2.607l-4.466-4.468a3.694 3.694 0 0 0-1.564-.927l-.07-.02a3.43 3.43 0 0 0-.946-.133L12 7.682zm3.165 3.357c.023 1.748-1.33 3.078-1.33 4.806.164 2.227 1.733 3.207 3.266 3.146-.035.003-.068.007-.104.009-1.847.135-3.209-1.326-5.002-1.326-2.23.164-3.21 1.736-3.147 3.27l-.008-.104c-.133-1.847 1.328-3.21 1.328-5.002-.173-2.32-1.867-3.284-3.46-3.132.1-.011.203-.021.31-.027 1.847-.133 3.209 1.328 5.002 1.328 2.082-.155 3.074-1.536 3.145-2.968z"/></svg>
|
||||
|
After Width: | Height: | Size: 2.2 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#EBAF00" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Plex</title><path d="M3.987 8.409c-.96 0-1.587.28-2.12.933v-.72H0v8.88s.038.018.127.037c.138.03.821.187 1.331-.249.441-.377.542-.814.542-1.318v-1.283c.533.573 1.147.813 2 .813 1.84 0 3.253-1.493 3.253-3.48 0-2.12-1.36-3.613-3.266-3.613Zm16.748 5.595.406.591c.391.614.894.906 1.492.908.621-.012 1.064-.562 1.226-.755 0 0-.307-.27-.686-.72-.517-.614-1.214-1.755-1.24-1.803l-1.198 1.779Zm-3.205-1.955c0-2.08-1.52-3.64-3.52-3.64s-3.467 1.587-3.467 3.573a3.48 3.48 0 0 0 3.507 3.52c1.413 0 2.626-.84 3.253-2.293h-2.04l-.093.093c-.427.4-.72.533-1.227.533-.787 0-1.373-.506-1.453-1.266h4.986c.04-.214.054-.307.054-.52Zm-7.671-.219c0 .769.11 1.701.868 2.722l.056.069c-.306.526-.742.88-1.248.88-.399 0-.814-.211-1.138-.579a2.177 2.177 0 0 1-.538-1.441V6.409H9.86l-.001 5.421Zm9.283 3.46h-2.39l2.247-3.332-2.247-3.335h2.39l2.248 3.335-2.248 3.332Zm1.593-1.286Zm-17.162-.342c-.933 0-1.68-.773-1.68-1.72s.76-1.666 1.68-1.666c.92 0 1.68.733 1.68 1.68 0 .946-.733 1.706-1.68 1.706Zm18.361-1.974L24 8.622h-2.391l-.87 1.293 1.195 1.773Zm-9.404-.466c.16-.706.72-1.133 1.493-1.133.773 0 1.373.467 1.507 1.133h-3Z"/></svg>
|
||||
|
After Width: | Height: | Size: 1.2 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#4169E1" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>PostgreSQL</title><path d="M23.5594 14.7228a.5269.5269 0 0 0-.0563-.1191c-.139-.2632-.4768-.3418-1.0074-.2321-1.6533.3411-2.2935.1312-2.5256-.0191 1.342-2.0482 2.445-4.522 3.0411-6.8297.2714-1.0507.7982-3.5237.1222-4.7316a1.5641 1.5641 0 0 0-.1509-.235C21.6931.9086 19.8007.0248 17.5099.0005c-1.4947-.0158-2.7705.3461-3.1161.4794a9.449 9.449 0 0 0-.5159-.0816 8.044 8.044 0 0 0-1.3114-.1278c-1.1822-.0184-2.2038.2642-3.0498.8406-.8573-.3211-4.7888-1.645-7.2219.0788C.9359 2.1526.3086 3.8733.4302 6.3043c.0409.818.5069 3.334 1.2423 5.7436.4598 1.5065.9387 2.7019 1.4334 3.582.553.9942 1.1259 1.5933 1.7143 1.7895.4474.1491 1.1327.1441 1.8581-.7279.8012-.9635 1.5903-1.8258 1.9446-2.2069.4351.2355.9064.3625 1.39.3772a.0569.0569 0 0 0 .0004.0041 11.0312 11.0312 0 0 0-.2472.3054c-.3389.4302-.4094.5197-1.5002.7443-.3102.064-1.1344.2339-1.1464.8115-.0025.1224.0329.2309.0919.3268.2269.4231.9216.6097 1.015.6331 1.3345.3335 2.5044.092 3.3714-.6787-.017 2.231.0775 4.4174.3454 5.0874.2212.5529.7618 1.9045 2.4692 1.9043.2505 0 .5263-.0291.8296-.0941 1.7819-.3821 2.5557-1.1696 2.855-2.9059.1503-.8707.4016-2.8753.5388-4.1012.0169-.0703.0357-.1207.057-.1362.0007-.0005.0697-.0471.4272.0307a.3673.3673 0 0 0 .0443.0068l.2539.0223.0149.001c.8468.0384 1.9114-.1426 2.5312-.4308.6438-.2988 1.8057-1.0323 1.5951-1.6698zM2.371 11.8765c-.7435-2.4358-1.1779-4.8851-1.2123-5.5719-.1086-2.1714.4171-3.6829 1.5623-4.4927 1.8367-1.2986 4.8398-.5408 6.108-.13-.0032.0032-.0066.0061-.0098.0094-2.0238 2.044-1.9758 5.536-1.9708 5.7495-.0002.0823.0066.1989.0162.3593.0348.5873.0996 1.6804-.0735 2.9184-.1609 1.1504.1937 2.2764.9728 3.0892.0806.0841.1648.1631.2518.2374-.3468.3714-1.1004 1.1926-1.9025 2.1576-.5677.6825-.9597.5517-1.0886.5087-.3919-.1307-.813-.5871-1.2381-1.3223-.4796-.839-.9635-2.0317-1.4155-3.5126zm6.0072 5.0871c-.1711-.0428-.3271-.1132-.4322-.1772.0889-.0394.2374-.0902.4833-.1409 1.2833-.2641 1.4815-.4506 1.9143-1.0002.0992-.126.2116-.2687.3673-.4426a.3549.3549 0 0 0 .0737-.1298c.1708-.1513.2724-.1099.4369-.0417.156.0646.3078.26.3695.4752.0291.1016.0619.2945-.0452.4444-.9043 1.2658-2.2216 1.2494-3.1676 1.0128zm2.094-3.988-.0525.141c-.133.3566-.2567.6881-.3334 1.003-.6674-.0021-1.3168-.2872-1.8105-.8024-.6279-.6551-.9131-1.5664-.7825-2.5004.1828-1.3079.1153-2.4468.079-3.0586-.005-.0857-.0095-.1607-.0122-.2199.2957-.2621 1.6659-.9962 2.6429-.7724.4459.1022.7176.4057.8305.928.5846 2.7038.0774 3.8307-.3302 4.7363-.084.1866-.1633.3629-.2311.5454zm7.3637 4.5725c-.0169.1768-.0358.376-.0618.5959l-.146.4383a.3547.3547 0 0 0-.0182.1077c-.0059.4747-.054.6489-.115.8693-.0634.2292-.1353.4891-.1794 1.0575-.11 1.4143-.8782 2.2267-2.4172 2.5565-1.5155.3251-1.7843-.4968-2.0212-1.2217a6.5824 6.5824 0 0 0-.0769-.2266c-.2154-.5858-.1911-1.4119-.1574-2.5551.0165-.5612-.0249-1.9013-.3302-2.6462.0044-.2932.0106-.5909.019-.8918a.3529.3529 0 0 0-.0153-.1126 1.4927 1.4927 0 0 0-.0439-.208c-.1226-.4283-.4213-.7866-.7797-.9351-.1424-.059-.4038-.1672-.7178-.0869.067-.276.1831-.5875.309-.9249l.0529-.142c.0595-.16.134-.3257.213-.5012.4265-.9476 1.0106-2.2453.3766-5.1772-.2374-1.0981-1.0304-1.6343-2.2324-1.5098-.7207.0746-1.3799.3654-1.7088.5321a5.6716 5.6716 0 0 0-.1958.1041c.0918-1.1064.4386-3.1741 1.7357-4.4823a4.0306 4.0306 0 0 1 .3033-.276.3532.3532 0 0 0 .1447-.0644c.7524-.5706 1.6945-.8506 2.802-.8325.4091.0067.8017.0339 1.1742.081 1.939.3544 3.2439 1.4468 4.0359 2.3827.8143.9623 1.2552 1.9315 1.4312 2.4543-1.3232-.1346-2.2234.1268-2.6797.779-.9926 1.4189.543 4.1729 1.2811 5.4964.1353.2426.2522.4522.2889.5413.2403.5825.5515.9713.7787 1.2552.0696.087.1372.1714.1885.245-.4008.1155-1.1208.3825-1.0552 1.717-.0123.1563-.0423.4469-.0834.8148-.0461.2077-.0702.4603-.0994.7662zm.8905-1.6211c-.0405-.8316.2691-.9185.5967-1.0105a2.8566 2.8566 0 0 0 .135-.0406 1.202 1.202 0 0 0 .1342.103c.5703.3765 1.5823.4213 3.0068.1344-.2016.1769-.5189.3994-.9533.6011-.4098.1903-1.0957.333-1.7473.3636-.7197.0336-1.0859-.0807-1.1721-.151zm.5695-9.2712c-.0059.3508-.0542.6692-.1054 1.0017-.055.3576-.112.7274-.1264 1.1762-.0142.4368.0404.8909.0932 1.3301.1066.887.216 1.8003-.2075 2.7014a3.5272 3.5272 0 0 1-.1876-.3856c-.0527-.1276-.1669-.3326-.3251-.6162-.6156-1.1041-2.0574-3.6896-1.3193-4.7446.3795-.5427 1.3408-.5661 2.1781-.463zm.2284 7.0137a12.3762 12.3762 0 0 0-.0853-.1074l-.0355-.0444c.7262-1.1995.5842-2.3862.4578-3.4385-.0519-.4318-.1009-.8396-.0885-1.2226.0129-.4061.0666-.7543.1185-1.0911.0639-.415.1288-.8443.1109-1.3505.0134-.0531.0188-.1158.0118-.1902-.0457-.4855-.5999-1.938-1.7294-3.253-.6076-.7073-1.4896-1.4972-2.6889-2.0395.5251-.1066 1.2328-.2035 2.0244-.1859 2.0515.0456 3.6746.8135 4.8242 2.2824a.908.908 0 0 1 .0667.1002c.7231 1.3556-.2762 6.2751-2.9867 10.5405zm-8.8166-6.1162c-.025.1794-.3089.4225-.6211.4225a.5821.5821 0 0 1-.0809-.0056c-.1873-.026-.3765-.144-.5059-.3156-.0458-.0605-.1203-.178-.1055-.2844.0055-.0401.0261-.0985.0925-.1488.1182-.0894.3518-.1226.6096-.0867.3163.0441.6426.19Line truncated
|
||||
|
After Width: | Height: | Size: 5.1 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#E6522C" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Prometheus</title><path d="M12 0C5.373 0 0 5.372 0 12c0 6.627 5.373 12 12 12s12-5.373 12-12c0-6.628-5.373-12-12-12zm0 22.46c-1.885 0-3.414-1.26-3.414-2.814h6.828c0 1.553-1.528 2.813-3.414 2.813zm5.64-3.745H6.36v-2.046h11.28v2.046zm-.04-3.098H6.391c-.037-.043-.075-.086-.111-.13-1.155-1.401-1.427-2.133-1.69-2.879-.005-.025 1.4.287 2.395.511 0 0 .513.119 1.262.255-.72-.843-1.147-1.915-1.147-3.01 0-2.406 1.845-4.508 1.18-6.207.648.053 1.34 1.367 1.387 3.422.689-.951.977-2.69.977-3.755 0-1.103.727-2.385 1.454-2.429-.648 1.069.168 1.984.894 4.256.272.854.237 2.29.447 3.201.07-1.892.395-4.652 1.595-5.605-.529 1.2.079 2.702.494 3.424.671 1.164 1.078 2.047 1.078 3.716a4.642 4.642 0 01-1.11 2.996c.792-.149 1.34-.283 1.34-.283l2.573-.502s-.374 1.538-1.81 3.019z"/></svg>
|
||||
|
After Width: | Height: | Size: 862 B |
@@ -0,0 +1 @@
|
||||
<svg fill="#2F67BA" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>qbittorrent</title><path d="M11.911 23.994c-1.31 0-2.605-.232-3.831-.705-3.4-1.024-6.2-3.865-7.433-7.58-1.23-3.708-.685-7.654 1.459-10.554C4.062 2.038 7.677.094 11.742.008c4.064-.079 7.758 1.703 9.882 4.785a12.066 12.066 0 0 1 2.369 7.145c.138 3.733-1.75 7.368-5.052 9.728-2.147 1.535-4.61 2.328-7.03 2.328zm.11-22.314c-.081 0-.162 0-.244.002-3.5.074-6.599 1.725-8.29 4.415-1.856 2.516-2.31 5.893-1.25 9.086 1.06 3.197 3.448 5.636 6.386 6.523 3.025 1.165 6.496.633 9.345-1.402 2.847-2.035 4.473-5.144 4.351-8.318v-.032c0-2.214-.73-4.41-2.055-6.185-1.78-2.58-4.84-4.09-8.243-4.09zM9.406 20.246v-4.578a2.663 2.663 0 0 1-.952.863 2.573 2.573 0 0 1-1.29.344c-1.016 0-1.893-.444-2.63-1.33-.731-.887-1.097-2.102-1.097-3.646 0-.939.148-1.781.444-2.527.301-.746.734-1.309 1.299-1.69A3.26 3.26 0 0 1 7.052 7.1c1.058 0 1.891.487 2.5 1.46v-1.25h1.306v12.935H9.406zm-4.477-8.285c0 1.203.232 2.108.694 2.711.463.6 1.016.9 1.662.9.619 0 1.15-.286 1.597-.855.446-.576.67-1.447.67-2.615 0-1.245-.237-2.18-.71-2.81-.468-.627-1.02-.941-1.654-.941-.63 0-1.164.293-1.605.88-.435.581-.654 1.491-.654 2.73m9.55 4.702h-1.346V3.755h1.452v4.604c.613-.84 1.395-1.258 2.347-1.258.526 0 1.024.117 1.492.351.464.222.864.558 1.161.978.307.416.546.922.718 1.514.172.593.258 1.227.258 1.902 0 1.603-.363 2.841-1.088 3.716-.727.874-1.598 1.312-2.614 1.312-1.011 0-1.804-.46-2.379-1.382v1.17m-.016-4.746c0 1.122.14 1.932.42 2.432.456.815 1.074 1.223 1.854 1.223.635 0 1.183-.3 1.646-.898.462-.604.693-1.503.693-2.695 0-1.22-.224-2.122-.67-2.703-.44-.58-.975-.872-1.605-.872-.634 0-1.182.303-1.645.907-.463.6-.694 1.468-.694 2.607"/></svg>
|
||||
|
After Width: | Height: | Size: 1.7 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#FFCB3D" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>radarr</title><path d="M5.274 0C3.189.039 1.19 1.547 1.19 4.705l.184 14.518c0 1.47 1.103 2.205 2.573 2.021L3.764 3.786c0-1.654.919-1.838 2.022-1.103l14.7 8.27c1.103.734 1.655 1.47 1.838 2.756.92-1.654.552-4.043-1.286-5.33L7.991.846A4.559 4.559 0 0 0 5.274.001zm1.982 6.91-.184 10.107 9.004-5.146Zm13.598 6.064-15.068 8.82c-.92.552-2.022.736-3.124.368.918 1.47 3.307 2.389 5.145 1.47l12.68-7.35c1.102-.736 1.286-2.022.367-3.308z"/></svg>
|
||||
|
After Width: | Height: | Size: 529 B |
@@ -0,0 +1 @@
|
||||
<svg fill="#3050FF" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>SearXNG</title><path d="m13.716 17.261 6.873 6.582L24 20.282l-6.824-6.536a9.11 9.11 0 0 0 1.143-4.43c0-5.055-4.105-9.159-9.16-9.159S0 4.261 0 9.316c0 5.055 4.104 9.159 9.159 9.159a9.11 9.11 0 0 0 4.557-1.214ZM9.159 2.773a6.546 6.546 0 0 1 6.543 6.543 6.545 6.545 0 0 1-6.543 6.542 6.545 6.545 0 0 1-6.542-6.542 6.545 6.545 0 0 1 6.542-6.543ZM7.26 5.713a4.065 4.065 0 0 1 4.744.747 4.064 4.064 0 0 1 .707 4.749l1.157.611a5.376 5.376 0 0 0-.935-6.282 5.377 5.377 0 0 0-6.274-.987l.601 1.162Z"/></svg>
|
||||
|
After Width: | Height: | Size: 591 B |
@@ -0,0 +1,16 @@
|
||||
<svg width="96" height="96" viewBox="0 0 96 96" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<circle cx="52" cy="52" r="28" fill="#131928"/>
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M48 96C74.5097 96 96 74.5097 96 48C96 21.4903 74.5097 0 48 0C21.4903 0 0 21.4903 0 48C0 74.5097 21.4903 96 48 96ZM80.0001 52C80.0001 67.464 67.4641 80 52.0001 80C36.5361 80 24.0001 67.464 24.0001 52C24.0001 49.1303 24.4318 46.3615 25.2338 43.7548C27.4288 48.6165 32.3194 52 38.0001 52C45.7321 52 52.0001 45.732 52.0001 38C52.0001 32.3192 48.6166 27.4287 43.755 25.2337C46.3616 24.4317 49.1304 24 52.0001 24C67.4641 24 80.0001 36.536 80.0001 52Z" fill="url(#paint0_linear)"/>
|
||||
<path opacity="0.2" fill-rule="evenodd" clip-rule="evenodd" d="M80.0002 52C80.0002 67.464 67.4642 80 52.0002 80C36.864 80 24.5329 67.9897 24.017 52.9791C24.0057 53.318 24 53.6583 24 54C24 70.5685 37.4315 84 54 84C70.5685 84 84 70.5685 84 54C84 37.4315 70.5685 24 54 24C53.6597 24 53.3207 24.0057 52.9831 24.0169C67.9919 24.5347 80.0002 36.865 80.0002 52Z" fill="#131928"/>
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M48 12C28.1177 12 12 28.1177 12 48C12 50.2091 10.2091 52 8 52C5.79086 52 4 50.2091 4 48C4 23.6995 23.6995 4 48 4C50.2091 4 52 5.79086 52 8C52 10.2091 50.2091 12 48 12Z" fill="url(#paint1_linear)"/>
|
||||
<defs>
|
||||
<linearGradient id="paint0_linear" x1="48" y1="-2.07126e-06" x2="117.5" y2="69.5" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="#C395FC"/>
|
||||
<stop offset="1" stop-color="#4F65F5"/>
|
||||
</linearGradient>
|
||||
<linearGradient id="paint1_linear" x1="28" y1="8" x2="28" y2="48" gradientUnits="userSpaceOnUse">
|
||||
<stop stop-color="white" stop-opacity="0.4"/>
|
||||
<stop offset="1" stop-color="white" stop-opacity="0"/>
|
||||
</linearGradient>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.7 KiB |
@@ -0,0 +1 @@
|
||||
<svg fill="#2596BE" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>sonarr</title><path d="M21.212 4.282c1.851 2.204 2.777 4.776 2.777 7.718 0 2.848-.867 5.344-2.602 7.489a934.355 934.355 0 0 1-2.101-2.095c-1.477-1.477-1.792-3.293-1.792-5.278 0-2.224.127-3.486 1.577-4.935l2.478-2.478a13.209 13.209 0 0 0-.337-.421Zm-17.7 16.193C1.708 18.678.6 16.59.188 14.213A11.84 11.84 0 0 1 .011 12c0-.28.006-.548.017-.802 0-.026.007-.052.022-.078.153-2.601 1.076-4.889 2.767-6.865-.108.127-.214.256-.316.387 0 0 1.351 1.346 2.329 2.323 1.408 1.409 1.726 3.215 1.726 5.151 0 1.985-.249 3.762-1.781 5.295-1.035 1.035-2.119 2.124-2.119 2.124.112.136.229.271.349.404.029-.027 1.297-1.348 2.123-2.175 1.638-1.637 1.928-3.528 1.928-5.648 0-2.072-.365-3.997-1.873-5.504a620.045 620.045 0 0 0-2.366-2.357c.168-.196.342-.388.523-.576l3.117 3.106-.194.195 1.903 1.898.547-.549L6.81 6.432l-.196.196L3.495 3.52c.01-.009.436-.416.643-.597.009.011 2.28 2.283 2.28 2.283 1.538 1.537 3.5 1.955 5.621 1.955 2.18 0 4.134-.442 5.731-2.038.907-.908 2.153-2.149 2.162-2.16.17.151.491.461.56.528l.013.013-3.111 3.028-.001.002-.197-.194-1.876 1.903.552.543 1.875-1.903-.197-.194 3.109-3.026c.193.203.377.41.553.619-.03.025-2.495 2.546-2.495 2.546-1.556 1.556-1.723 2.9-1.723 5.288 0 2.121.361 4.054 1.939 5.632a576.91 576.91 0 0 0 2.133 2.124c-.183.208-.599.645-.613.66l-3.066-3.174.195-.196-1.995-1.986-.546.549 1.995 1.986.195-.196 3.065 3.172c-.021.019-.385.362-.552.506-.01-.013-1.974-1.978-1.974-1.978-1.842-1.842-3.299-2.039-5.731-2.039-2.338 0-3.92.239-5.632 1.95-.944.944-2.078 2.085-2.089 2.099-.275-.23-.649-.594-.649-.594l3.019-3.024.199.192 1.854-1.925-.558-.538-1.854 1.926.199.191-3.016 3.022ZM12 8.672A3.33 3.33 0 0 0 8.672 12 3.33 3.33 0 0 0 12 15.328 3.33 3.33 0 0 0 15.328 12 3.33 3.33 0 0 0 12 8.672ZM4.52 2.6C6.665.867 9.162 0 12.011 0c2.88 0 5.394.88 7.541 2.639 0 0-1.215 1.209-2.136 2.13-1.496 1.496-3.334 1.892-5.377 1.892-1.985 0-3.829-.37-5.267-1.809L4.52 2.6Zm14.837 18.909a9.507 9.507 0 0 1-.342.256C16.994 23.255 14.659 24 12.011 24c-2.652 0-4.983-.745-6.993-2.235-.104-.074-.208-.15-.31-.227 0 0 1.096-1.101 2.053-2.058 1.602-1.602 3.09-1.804 5.278-1.804 2.28 0 3.651.166 5.377 1.892l1.941 1.941Z"/></svg>
|
||||
|
After Width: | Height: | Size: 2.2 KiB |
|
After Width: | Height: | Size: 102 KiB |
|
After Width: | Height: | Size: 43 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 93 KiB |
|
After Width: | Height: | Size: 128 KiB |
|
After Width: | Height: | Size: 163 KiB |
@@ -5,20 +5,25 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>frank — services</title>
|
||||
<base href="/">
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Rajdhani:wght@500;600;700&family=Share+Tech+Mono&display=swap">
|
||||
<link rel="stylesheet" href="style.css">
|
||||
</head>
|
||||
<body>
|
||||
<header class="top">
|
||||
<span class="title">frank@home:~$ services</span>
|
||||
<a id="bare-toggle" class="toggle" href="?bare=1">bare</a>
|
||||
<nav class="top-links">
|
||||
<a class="foot-btn" href="/docs">docs</a>
|
||||
<a class="foot-btn" id="admin-link" href="/admin/" hidden>admin</a>
|
||||
<a class="foot-btn" href="https://auth.franksdank.cloud/if/flow/default-user-settings-flow/" target="_blank" rel="noopener">account settings</a>
|
||||
<a id="bare-toggle" class="toggle" href="?bare=1">bare</a>
|
||||
</nav>
|
||||
</header>
|
||||
<div id="summary" class="summary"></div>
|
||||
<main id="grid" class="sections"></main>
|
||||
<footer class="foot">
|
||||
<span>last master deployment: <span id="deployed-at"></span></span>
|
||||
<a href="/docs">docs</a>
|
||||
<a href="/admin/">admin</a>
|
||||
<a href="https://auth.franksdank.cloud/if/flow/default-user-settings-flow/" target="_blank" rel="noopener">account</a>
|
||||
</footer>
|
||||
<script src="deployed.js"></script>
|
||||
<script src="app.js"></script>
|
||||
|
||||
@@ -9,7 +9,9 @@
|
||||
"probe": "127.0.0.1:8081",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"warn": "Searches are obfuscated, not anonymous: SearXNG hides you from the upstream engines and the VPN tunnel hides the traffic from the LAN — but the upstream search API keys are tied to Frank’s identity. Don’t treat it as Tor.",
|
||||
"art": "/img/apps/searxng.svg"
|
||||
},
|
||||
{
|
||||
"name": "OpenWebUI",
|
||||
@@ -21,91 +23,101 @@
|
||||
"probe": "127.0.0.1:3005",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/openwebui.png",
|
||||
"art_bg": "#f5f5f5",
|
||||
"warn": "Chats stay on Frank's server, but OpenWebUI's web search rides the same SearXNG path as the search app: obfuscated, not anonymous. The upstream search API keys are tied to Frank’s identity."
|
||||
},
|
||||
{
|
||||
"name": "Gitea",
|
||||
"category": "user",
|
||||
"section": "Web Apps",
|
||||
"description": "Git code hosting and review platform.",
|
||||
"description": "Git code hosting and review platform. Needs its own login, separate from the dankcloud single sign-on — use the sign-up flow in the app to get an account; no need to ask Frank.",
|
||||
"url": "http://gitea.franksdank.cloud",
|
||||
"docs": "",
|
||||
"probe": "127.0.0.1:3000",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/gitea.svg"
|
||||
},
|
||||
{
|
||||
"name": "Calibre-Web",
|
||||
"category": "user",
|
||||
"section": "Media Library Apps",
|
||||
"description": "E-book library.",
|
||||
"url": "http://books.franksdank.cloud",
|
||||
"description": "E-book library. Work in progress: not open to family users yet.",
|
||||
"url": "",
|
||||
"docs": "",
|
||||
"probe": "192.168.0.235:8083",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"wip": "down",
|
||||
"showLink": false,
|
||||
"art": "/img/apps/app-calibre.svg"
|
||||
},
|
||||
{
|
||||
"name": "RomM",
|
||||
"category": "user",
|
||||
"section": "Media Library Apps",
|
||||
"description": "ROM and emulator library.",
|
||||
"url": "http://romm.franksdank.cloud",
|
||||
"description": "ROM and emulator library. Work in progress: not open to family users yet.",
|
||||
"url": "",
|
||||
"docs": "",
|
||||
"probe": "192.168.0.235:8085",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"wip": "down",
|
||||
"showLink": false,
|
||||
"art": "/img/apps/app-gameboy.svg"
|
||||
},
|
||||
{
|
||||
"name": "Copyparty",
|
||||
"category": "user",
|
||||
"section": "Web Apps",
|
||||
"description": "Direct downloads from Frank's server.",
|
||||
"description": "Direct downloads from Frank's server. Everything the other media apps hold — including the work-in-progress ones (Calibre-Web, RomM, Seerr) — is available here.",
|
||||
"url": "http://files.franksdank.cloud",
|
||||
"docs": "",
|
||||
"probe": "192.168.0.235:3923",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/app-folder.svg"
|
||||
},
|
||||
{
|
||||
"name": "Plex",
|
||||
"category": "user",
|
||||
"section": "Media Library Apps",
|
||||
"description": "Closed source media server with apps for everything.",
|
||||
"description": "Closed source media server with apps for everything. Needs its own Plex login, separate from the dankcloud single sign-on (ask Frank).",
|
||||
"url": "http://plex.franksdank.cloud/web/index.html",
|
||||
"docs": "",
|
||||
"probe": "192.168.0.235:32400",
|
||||
"method": "tcp",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/plex.svg"
|
||||
},
|
||||
{
|
||||
"name": "Jellyfin",
|
||||
"category": "user",
|
||||
"section": "Media Library Apps",
|
||||
"description": "Open source media server.",
|
||||
"url": "http://jelly.franksdank.cloud",
|
||||
"description": "Open source media server. Work in progress: not open to family users yet.",
|
||||
"url": "",
|
||||
"docs": "",
|
||||
"probe": "192.168.0.235:8096",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"wip": "down",
|
||||
"showLink": false,
|
||||
"art": "/img/apps/jellyfin.svg"
|
||||
},
|
||||
{
|
||||
"name": "Seerr",
|
||||
"category": "user",
|
||||
"section": "Media Library Apps",
|
||||
"description": "Download Requests",
|
||||
"url": "http://seerr.franksdank.cloud",
|
||||
"description": "Download requests. Work in progress: not open to family users yet.",
|
||||
"url": "",
|
||||
"docs": "",
|
||||
"probe": "127.0.0.1:5055",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"wip": "down",
|
||||
"showLink": false,
|
||||
"art": "/img/apps/seerr.svg"
|
||||
},
|
||||
{
|
||||
"name": "Marreta",
|
||||
@@ -117,7 +129,8 @@
|
||||
"probe": "127.0.0.1:8082",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/app-wall.svg"
|
||||
},
|
||||
{
|
||||
"name": "Sonarr",
|
||||
@@ -129,7 +142,8 @@
|
||||
"probe": "192.168.0.112:8989",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/sonarr.svg"
|
||||
},
|
||||
{
|
||||
"name": "Hermes",
|
||||
@@ -141,7 +155,8 @@
|
||||
"probe": "127.0.0.1:8787",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-hm.svg"
|
||||
},
|
||||
{
|
||||
"name": "Hermes dashboard",
|
||||
@@ -153,7 +168,8 @@
|
||||
"probe": "192.168.0.143:9119",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-hd.svg"
|
||||
},
|
||||
{
|
||||
"name": "Grafana",
|
||||
@@ -165,7 +181,8 @@
|
||||
"probe": "127.0.0.1:3002",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/grafana.svg"
|
||||
},
|
||||
{
|
||||
"name": "Prometheus",
|
||||
@@ -177,7 +194,8 @@
|
||||
"probe": "127.0.0.1:9090",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/prometheus.svg"
|
||||
},
|
||||
{
|
||||
"name": "Loki",
|
||||
@@ -189,7 +207,8 @@
|
||||
"probe": "127.0.0.1:3100",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-lk.svg"
|
||||
},
|
||||
{
|
||||
"name": "node-exporter",
|
||||
@@ -201,7 +220,8 @@
|
||||
"probe": "127.0.0.1:9100",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mono-ne.svg"
|
||||
},
|
||||
{
|
||||
"name": "alloy",
|
||||
@@ -213,7 +233,8 @@
|
||||
"probe": "127.0.0.1:12345",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mono-ay.svg"
|
||||
},
|
||||
{
|
||||
"name": "alertmanager",
|
||||
@@ -225,7 +246,8 @@
|
||||
"probe": "127.0.0.1:9093",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mono-am.svg"
|
||||
},
|
||||
{
|
||||
"name": "Caddy",
|
||||
@@ -237,7 +259,8 @@
|
||||
"probe": "127.0.0.1:80",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/caddy.svg"
|
||||
},
|
||||
{
|
||||
"name": "Pi-hole",
|
||||
@@ -249,7 +272,8 @@
|
||||
"probe": "192.168.0.143:8083",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/pihole.svg"
|
||||
},
|
||||
{
|
||||
"name": "Gluetun",
|
||||
@@ -261,17 +285,72 @@
|
||||
"probe": "127.0.0.1:51820",
|
||||
"method": "tcp",
|
||||
"wip": "closed",
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mono-gt.svg"
|
||||
},
|
||||
{
|
||||
"name": "frank-zealot",
|
||||
"category": "user",
|
||||
"section": "AI",
|
||||
"model": {
|
||||
"unit": "zealot",
|
||||
"role": "Frank's utility model",
|
||||
"hf": "https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive",
|
||||
"tagline": "HONOR GUIDE ME!",
|
||||
"sfx": [
|
||||
"zealot-honor.ogg"
|
||||
],
|
||||
"equiv": "Equivalent to last year's frontier level (2025).",
|
||||
"spec": "Qwen3.5-9B-Uncensored (Q6_K) on an NVIDIA RTX 3090 — ~95 tokens/s; 200k-token context, two submissions at a time; sees images.",
|
||||
"inputs": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"use": "Code editor autocomplete, agentic search, RAG, research subagents, translation, roleplay, and summarization.",
|
||||
"voice": {
|
||||
"text": "I LONG FOR COMBAT!",
|
||||
"sfx": [
|
||||
"zealot-combat.ogg"
|
||||
]
|
||||
},
|
||||
"supervision": "Requires tight human supervision.",
|
||||
"warn": "Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.",
|
||||
"compaction": "If your agent harness lets you set a compaction model, set this one: a compaction job on the bigger models blocks them for far longer."
|
||||
},
|
||||
"url": "",
|
||||
"docs": "/docs#models",
|
||||
"probe": "127.0.0.1:7771",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
},
|
||||
{
|
||||
"name": "frank-large",
|
||||
"name": "frank-dragoon",
|
||||
"category": "user",
|
||||
"section": "AI",
|
||||
"details": {
|
||||
"role": "Deep reasoning model",
|
||||
"spec": "Qwen3.8-27B (UD-Q8_K_XL, 31.5 GiB) — 262k-token context, one conversation at a time, text only.",
|
||||
"inputs": ["text"],
|
||||
"use": "Cybersecurity and debugging. Requires Frank's permission."
|
||||
"model": {
|
||||
"unit": "dragoon",
|
||||
"role": "Frank's deep reasoning model",
|
||||
"hf": "https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated",
|
||||
"tagline": "I have returned — make use of me.",
|
||||
"sfx": [
|
||||
"dragoon-tagline.ogg"
|
||||
],
|
||||
"equiv": "Equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier intelligence).",
|
||||
"spec": "Qwen3.8-Flash-Next v2 Swift1.5 abliterated (halogen flash-server, ~62 GiB) on the Strix Halo iGPU — 200k-token context, two submissions at a time; sees images.",
|
||||
"inputs": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"use": "Complex agentic tasks, programming, system administration, and \"claw\"-style personal agents.",
|
||||
"voice": {
|
||||
"text": "Awaiting instructions.",
|
||||
"sfx": [
|
||||
"dragoon-awaiting.ogg"
|
||||
]
|
||||
},
|
||||
"supervision": "Safe for autonomous, low-supervision use.",
|
||||
"warn": "Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning “sensitive offensive cybersecurity topics”, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody."
|
||||
},
|
||||
"url": "",
|
||||
"docs": "/docs#models",
|
||||
@@ -281,14 +360,32 @@
|
||||
"showLink": false
|
||||
},
|
||||
{
|
||||
"name": "frank-primary",
|
||||
"name": "frank-archon",
|
||||
"category": "user",
|
||||
"section": "AI",
|
||||
"details": {
|
||||
"role": "Primary model",
|
||||
"spec": "Qwen3.8-Flash-Next (Q8_0, 175.3 GiB) — full 262k-token context, one conversation; sees images. Spec-drafted (MTP + n-gram) on both R9700s.",
|
||||
"inputs": ["text", "image"],
|
||||
"use": "Software engineering, system administration, and instruction tasks."
|
||||
"model": {
|
||||
"unit": "archon",
|
||||
"role": "Frank's strongest and fastest model runtime",
|
||||
"hf": "https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700",
|
||||
"tagline": "...Power Overwhelming...",
|
||||
"sfx": [
|
||||
"archon-power.ogg"
|
||||
],
|
||||
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
|
||||
"spec": "Our own build: a GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next, quantized and tuned in-house (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
|
||||
"inputs": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"use": "Software engineering, system administration, and instruction tasks.",
|
||||
"voice": {
|
||||
"text": "...we BURN.",
|
||||
"sfx": [
|
||||
"archon-burn.ogg"
|
||||
]
|
||||
},
|
||||
"supervision": "Safe for autonomous, low-supervision use.",
|
||||
"permission": "Ask Frank for permission before use — this is his primary model; he uses it more than anything else on the fleet."
|
||||
},
|
||||
"url": "",
|
||||
"docs": "/docs#models",
|
||||
@@ -297,23 +394,6 @@
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
},
|
||||
{
|
||||
"name": "frank-secondary",
|
||||
"category": "user",
|
||||
"section": "AI",
|
||||
"details": {
|
||||
"role": "Fast utility model",
|
||||
"spec": "MiMo-V2.6-Distill-Qwen-9B (INT8 W8A16, 12 GiB) on SGLang — 606k-token pool across four 151k-token conversations; 84 tok/s decode; sees images.",
|
||||
"inputs": ["text", "image"],
|
||||
"use": "Agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence."
|
||||
},
|
||||
"url": "",
|
||||
"docs": "/docs#models",
|
||||
"probe": "127.0.0.1:7771",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": false
|
||||
},
|
||||
{
|
||||
"name": "mariadb",
|
||||
"category": "admin",
|
||||
@@ -324,7 +404,8 @@
|
||||
"probe": "192.168.0.235:3306",
|
||||
"method": "tcp",
|
||||
"wip": "closed",
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mariadb.svg"
|
||||
},
|
||||
{
|
||||
"name": "valkey",
|
||||
@@ -336,7 +417,8 @@
|
||||
"probe": "127.0.0.1:6379",
|
||||
"method": "tcp",
|
||||
"wip": "closed",
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/mono-vk.svg"
|
||||
},
|
||||
{
|
||||
"name": "Prowlarr",
|
||||
@@ -348,7 +430,8 @@
|
||||
"probe": "192.168.0.112:9696",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-pw.svg"
|
||||
},
|
||||
{
|
||||
"name": "Radarr",
|
||||
@@ -360,7 +443,8 @@
|
||||
"probe": "192.168.0.112:7878",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/radarr.svg"
|
||||
},
|
||||
{
|
||||
"name": "Readarr",
|
||||
@@ -372,7 +456,8 @@
|
||||
"probe": "192.168.0.112:8787",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-ra.svg"
|
||||
},
|
||||
{
|
||||
"name": "Lidarr",
|
||||
@@ -384,7 +469,8 @@
|
||||
"probe": "192.168.0.112:8686",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-ld.svg"
|
||||
},
|
||||
{
|
||||
"name": "qBittorrent",
|
||||
@@ -396,7 +482,8 @@
|
||||
"probe": "192.168.0.112:8080",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/qbittorrent.svg"
|
||||
},
|
||||
{
|
||||
"name": "Tautulli",
|
||||
@@ -408,7 +495,8 @@
|
||||
"probe": "192.168.0.235:8181",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-tt.svg"
|
||||
},
|
||||
{
|
||||
"name": "Postgres",
|
||||
@@ -420,7 +508,8 @@
|
||||
"probe": "127.0.0.1:5432",
|
||||
"method": "tcp",
|
||||
"wip": "closed",
|
||||
"showLink": false
|
||||
"showLink": false,
|
||||
"art": "/img/apps/postgresql.svg"
|
||||
},
|
||||
{
|
||||
"name": "ComfyUI",
|
||||
@@ -432,6 +521,7 @@
|
||||
"probe": "127.0.0.1:8188",
|
||||
"method": "http",
|
||||
"wip": false,
|
||||
"showLink": true
|
||||
"showLink": true,
|
||||
"art": "/img/apps/mono-cu.svg"
|
||||
}
|
||||
]
|
||||
@@ -1,17 +1,25 @@
|
||||
/* The homepage theme: terminal TUI, green-on-black, 1995 phosphor palette.
|
||||
A ?bare=1 toggle strips the page to bare, unstyled HTML (Q27). The page
|
||||
never sets a custom or hidden cursor, so the user's cursor is untouched (Q28). */
|
||||
/* The homepage theme: StarCraft Protoss console — flat deep-space navy,
|
||||
warp-blue/cyan accents used sparingly, Share Tech Mono body with
|
||||
Rajdhani headings (Google Fonts CDN), soft-radius panels on crisp
|
||||
1px steel borders, quiet section headers. Motion is reserved for
|
||||
state (speaking voice). A ?bare=1 toggle strips the page to bare,
|
||||
unstyled HTML (Q27). The page never sets a custom or hidden cursor,
|
||||
so the user's cursor is untouched (Q28). */
|
||||
|
||||
:root {
|
||||
--bg: #050805;
|
||||
--fg: #33ff66;
|
||||
--dim: #2f8f52;
|
||||
--down: #ff5555;
|
||||
--wip: #ffcc33;
|
||||
--link: #33ccff;
|
||||
--head: #aaffcc;
|
||||
--border: #1a4d2e;
|
||||
--panel: #0a120a;
|
||||
--bg: #050d16;
|
||||
--fg: #bfe3ff;
|
||||
--dim: #5d87a8;
|
||||
--down: #ff4d5e;
|
||||
--wip: #ffd24d;
|
||||
--link: #4fd8ff;
|
||||
--head: #eaf7ff;
|
||||
--border: #1d4a6e;
|
||||
--panel: #0a1826;
|
||||
--proto-blue: #3fa9ff; /* warp-blade blue */
|
||||
--proto-cyan: #7fe7ff; /* psychic-light cyan */
|
||||
--font-mono: "Share Tech Mono", "DejaVu Sans Mono", "Consolas", monospace;
|
||||
--font-head: "Rajdhani", "Share Tech Mono", sans-serif;
|
||||
}
|
||||
|
||||
* { box-sizing: border-box; }
|
||||
@@ -21,32 +29,32 @@ html { background: var(--bg); }
|
||||
body {
|
||||
background: var(--bg);
|
||||
color: var(--fg);
|
||||
font-family: "Terminus", "Fira Code", "DejaVu Sans Mono", "Consolas", monospace;
|
||||
font-size: 15px;
|
||||
line-height: 1.55;
|
||||
font-family: var(--font-mono);
|
||||
font-size: 18px;
|
||||
line-height: 1.45;
|
||||
margin: 0;
|
||||
padding: 24px;
|
||||
text-shadow: 0 0 4px rgba(51, 255, 102, 0.30);
|
||||
}
|
||||
|
||||
/* Subtle scanlines. */
|
||||
body::before {
|
||||
/* One quiet pylon glow above the horizon; the navy stays flat. */
|
||||
body::after {
|
||||
content: "";
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
pointer-events: none;
|
||||
z-index: 9999;
|
||||
background: repeating-linear-gradient(0deg, rgba(0, 0, 0, 0) 0 2px, rgba(0, 0, 0, 0.14) 2px 4px);
|
||||
z-index: -1;
|
||||
background: radial-gradient(1200px 500px at 50% -10%,
|
||||
rgba(63, 169, 255, 0.07), transparent 65%);
|
||||
}
|
||||
|
||||
.top {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: baseline;
|
||||
border-bottom: 1px solid var(--border);
|
||||
border-bottom: 2px solid var(--proto-blue);
|
||||
padding-bottom: 8px;
|
||||
}
|
||||
.title { color: var(--head); }
|
||||
.title { color: var(--head); text-shadow: 0 0 10px color-mix(in srgb, var(--proto-blue) 30%, transparent); }
|
||||
.summary { color: var(--dim); margin: 12px 0; letter-spacing: 0.05em; }
|
||||
|
||||
/* Sections stack vertically; each section's panels fill a horizontal
|
||||
@@ -54,14 +62,17 @@ body::before {
|
||||
.section { margin-bottom: 20px; }
|
||||
.section-title {
|
||||
color: var(--head);
|
||||
font-size: 1rem;
|
||||
font-weight: normal;
|
||||
letter-spacing: 0.08em;
|
||||
font-size: 1.05rem;
|
||||
font-weight: 600;
|
||||
letter-spacing: 0.16em;
|
||||
text-transform: uppercase;
|
||||
border-bottom: 1px solid var(--border);
|
||||
margin: 0 0 12px;
|
||||
padding-bottom: 4px;
|
||||
padding: 0 0 6px 24px;
|
||||
position: relative;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.section-title::before { content: "\25C7"; display: inline-block; width: 18px;
|
||||
margin-left: -24px; color: var(--proto-cyan); }
|
||||
|
||||
.grid {
|
||||
display: grid;
|
||||
@@ -69,20 +80,69 @@ body::before {
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.panel { border: 1px solid var(--border); background: var(--panel); padding: 12px 14px; border-radius: 2px; }
|
||||
.panel-head { display: flex; justify-content: space-between; align-items: center; }
|
||||
.panel { border: 1px solid var(--border); background: var(--panel); padding: 12px 14px;
|
||||
border-radius: 6px; display: flex; flex-direction: column; }
|
||||
.panel.clickable { cursor: pointer; transition: border-color 0.15s, filter 0.15s, transform 0.15s; }
|
||||
.panel.clickable:hover {
|
||||
border-color: var(--proto-cyan);
|
||||
transform: translateY(-2px);
|
||||
filter: drop-shadow(0 0 6px color-mix(in srgb, var(--proto-blue) 45%, transparent));
|
||||
}
|
||||
.panel.down {
|
||||
border-color: color-mix(in srgb, var(--down) 70%, var(--border));
|
||||
background: linear-gradient(180deg,
|
||||
color-mix(in srgb, var(--down) 10%, var(--panel)), var(--panel));
|
||||
}
|
||||
.mi-card:has(.badge.down) {
|
||||
border-color: color-mix(in srgb, var(--down) 70%, var(--mi));
|
||||
background: linear-gradient(180deg,
|
||||
color-mix(in srgb, var(--down) 10%, var(--panel)), var(--panel));
|
||||
}
|
||||
.panel-head { display: flex; align-items: center; gap: 8px; }
|
||||
.panel-head .name { flex: 1; }
|
||||
.app-art {
|
||||
width: 40px;
|
||||
height: 40px;
|
||||
flex: 0 0 auto;
|
||||
object-fit: contain;
|
||||
padding: 4px;
|
||||
clip-path: polygon(25% 3%, 75% 3%, 100% 50%, 75% 97%, 25% 97%, 0% 50%);
|
||||
border: 1px solid var(--dim);
|
||||
background: color-mix(in srgb, var(--fg) 8%, transparent);
|
||||
filter: saturate(1.15) brightness(1.05);
|
||||
}
|
||||
.name { color: var(--head); }
|
||||
.badge { border: 1px solid currentColor; padding: 0 6px; border-radius: 2px; font-size: 12px; }
|
||||
.badge { border: 1px solid currentColor; padding: 0 6px; font-size: 12px;
|
||||
border-radius: 3px; }
|
||||
.badge.up { color: var(--fg); }
|
||||
.badge.down { color: var(--down); }
|
||||
.badge.wip-open { color: var(--wip); }
|
||||
.badge.wip-closed { color: var(--dim); }
|
||||
/* Closed WIP items grey out: darker type, desaturated art. */
|
||||
.panel:has(.badge.wip-closed) { border-color: color-mix(in srgb, var(--dim) 45%, var(--border)); }
|
||||
.panel:has(.badge.wip-closed) .name,
|
||||
.panel:has(.badge.wip-closed) .desc,
|
||||
.panel:has(.badge.wip-closed) .desc-role,
|
||||
.panel:has(.badge.wip-closed) .desc-spec,
|
||||
.panel:has(.badge.wip-closed) .desc-use { color: var(--dim); }
|
||||
.panel:has(.badge.wip-closed) .app-art { filter: grayscale(0.75) brightness(0.7); }
|
||||
.badge.wip-down { color: var(--down); }
|
||||
.desc { color: var(--fg); margin: 8px 0; }
|
||||
.desc-role { color: var(--dim); font-weight: bold; }
|
||||
.desc-spec { color: var(--fg); }
|
||||
.desc-equiv { color: var(--head); font-style: italic; }
|
||||
.desc-inputs { color: var(--dim); }
|
||||
.desc-use { color: var(--fg); margin-top: 4px; }
|
||||
.links a { color: var(--link); text-decoration: underline; margin-right: 12px; }
|
||||
/* Privacy disclosures: collapsed chip pinned to the panel bottom. */
|
||||
.svc-warn { margin-top: auto; padding-top: 8px; color: var(--down); }
|
||||
.svc-warn summary, .mi-warn summary { cursor: pointer; color: var(--down);
|
||||
list-style: none; user-select: none; }
|
||||
.svc-warn summary::-webkit-details-marker,
|
||||
.mi-warn summary::-webkit-details-marker { display: none; }
|
||||
.svc-warn summary::after, .mi-warn summary::after { content: " \25B8"; }
|
||||
.svc-warn[open] summary::after, .mi-warn[open] summary::after { content: " \25BE"; }
|
||||
.warn-body { margin-top: 6px; color: var(--down); }
|
||||
|
||||
.foot {
|
||||
margin-top: 24px;
|
||||
@@ -93,22 +153,183 @@ body::before {
|
||||
justify-content: space-between;
|
||||
}
|
||||
.foot a { color: var(--link); text-decoration: none; }
|
||||
.top-links { display: flex; align-items: center; gap: 10px; flex-wrap: wrap; }
|
||||
.top .foot-btn { color: var(--link); text-decoration: none; }
|
||||
.foot-links { display: flex; gap: 10px; flex-wrap: wrap; }
|
||||
.foot-btn {
|
||||
border: 1px solid var(--link);
|
||||
padding: 2px 14px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 1px;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.foot-btn:hover {
|
||||
background: color-mix(in srgb, var(--link) 14%, transparent);
|
||||
}
|
||||
|
||||
/* The user docs page (static prose). */
|
||||
.docs { max-width: 780px; }
|
||||
.docs h1 { color: var(--head); font-size: 1.25rem; border-bottom: 1px solid var(--border); padding-bottom: 4px; }
|
||||
.docs h2 { color: var(--head); font-size: 1.05rem; margin-top: 20px; }
|
||||
.docs h1 { color: var(--head); font-size: 1.7rem; border-bottom: 1px solid var(--border); padding-bottom: 4px; }
|
||||
.docs h2 { color: var(--head); font-size: 1.35rem; margin-top: 20px; }
|
||||
.docs a { color: var(--link); }
|
||||
.docs code { color: var(--head); }
|
||||
.docs pre { background: var(--panel); border: 1px solid var(--border); border-radius: 2px; padding: 10px 12px; overflow-x: auto; }
|
||||
.docs pre code { color: var(--fg); }
|
||||
.docs img { display: block; max-width: 100%; border: 1px solid var(--border); border-radius: 2px; margin: 10px 0; }
|
||||
.docs .key-warn { color: var(--down); }
|
||||
.docs details.key-warn summary { cursor: pointer; list-style: none; user-select: none; }
|
||||
.docs details.key-warn summary::-webkit-details-marker { display: none; }
|
||||
.docs details.key-warn summary::after { content: " \25B8"; }
|
||||
.docs details.key-warn[open] summary::after { content: " \25BE"; }
|
||||
.docs details.key-warn .warn-body { margin-top: 4px; }
|
||||
.docs .foot-note { color: var(--dim); }
|
||||
html[data-bare="1"] .docs h1, html[data-bare="1"] .docs h2 { color: #000; border-color: #000; }
|
||||
html[data-bare="1"] .docs a, html[data-bare="1"] .docs code { color: #000; }
|
||||
html[data-bare="1"] .docs pre { background: none; border-color: #000; }
|
||||
html[data-bare="1"] .docs pre code { color: #000; }
|
||||
html[data-bare="1"] .docs img { border-color: #000; }
|
||||
html[data-bare="1"] .docs .key-warn { color: #000; }
|
||||
|
||||
/* Bare mode (Q27): strip to default browser rendering. */
|
||||
html[data-bare="1"] body { background: #fff; color: #000; font-family: serif; text-shadow: none; font-size: 16px; }
|
||||
html[data-bare="1"] body::before { display: none; }
|
||||
html[data-bare="1"] .panel, html[data-bare="1"] .top, html[data-bare="1"] .foot { border-color: #000; background: none; }
|
||||
html[data-bare="1"] .badge, html[data-bare="1"] .name, html[data-bare="1"] .desc, html[data-bare="1"] .summary, html[data-bare="1"] .section-title { color: #000; }
|
||||
html[data-bare="1"] .desc-role, html[data-bare="1"] .desc-spec, html[data-bare="1"] .desc-inputs, html[data-bare="1"] .desc-use { color: #000; font-weight: normal; }
|
||||
html[data-bare="1"] .section-title { border-color: #000; }
|
||||
html[data-bare="1"] .desc-role, html[data-bare="1"] .desc-spec, html[data-bare="1"] .desc-inputs, html[data-bare="1"] .desc-use, html[data-bare="1"] .desc-equiv { color: #000; font-weight: normal; }
|
||||
html[data-bare="1"] .section-title { border-color: #000; padding: 0 0 6px 2px; }
|
||||
html[data-bare="1"] .links a, html[data-bare="1"] .toggle, html[data-bare="1"] .foot a { color: #000; }
|
||||
html[data-bare="1"] .foot-btn { border-color: #000; text-shadow: none; }
|
||||
html[data-bare="1"] .panel, html[data-bare="1"] .mi-card, html[data-bare="1"] .badge, html[data-bare="1"] .foot-btn, html[data-bare="1"] .model-info { clip-path: none; border-radius: 0; filter: none; animation: none; box-shadow: none; }
|
||||
html[data-bare="1"] body::after { display: none; }
|
||||
html[data-bare="1"] .section-title::before, html[data-bare="1"] .section-title::after, html[data-bare="1"] .title::after { content: none; }
|
||||
html[data-bare="1"] .top { border-image: none; }
|
||||
|
||||
/* Model Information: one fused panel for the fleet models. Informational
|
||||
panels, so the type is smaller than the app panels; each model wears its
|
||||
StarCraft Protoss unit's colours and art. */
|
||||
.model-info {
|
||||
grid-column: 1 / -1;
|
||||
background: var(--panel);
|
||||
box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--proto-cyan) 25%, transparent);
|
||||
border-radius: 8px;
|
||||
}
|
||||
.mi-title {
|
||||
color: var(--proto-cyan);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.14em;
|
||||
border-bottom: 1px solid var(--border);
|
||||
margin-bottom: 10px;
|
||||
padding-bottom: 4px;
|
||||
display: flex;
|
||||
align-items: baseline;
|
||||
justify-content: space-between;
|
||||
gap: 12px;
|
||||
}
|
||||
.mi-docs-link {
|
||||
color: var(--link);
|
||||
text-decoration: none;
|
||||
text-transform: none;
|
||||
letter-spacing: 0.06em;
|
||||
font-size: 13px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.mi-docs-link:hover { text-decoration: underline; }
|
||||
.mi-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
|
||||
gap: 12px;
|
||||
}
|
||||
.mi-card {
|
||||
font-size: 15px;
|
||||
line-height: 1.35;
|
||||
border: 1px solid var(--mi);
|
||||
background: var(--mi-bg);
|
||||
padding: 10px 12px;
|
||||
border-radius: 6px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
.mi-zealot { --mi: #4fb4ff; } /* Zealot: Khai's blue blades */
|
||||
.mi-archon { --mi: #a8ecff; } /* Archon: pure psychic light */
|
||||
.mi-dragoon { --mi: #5fe0d0; } /* Dragoon: phase-crystal teal */
|
||||
.mi-top { display: flex; align-items: center; gap: 10px; }
|
||||
.mi-art {
|
||||
width: 56px;
|
||||
height: 56px;
|
||||
object-fit: cover;
|
||||
flex: 0 0 auto;
|
||||
clip-path: polygon(25% 3%, 75% 3%, 100% 50%, 75% 97%, 25% 97%, 0% 50%);
|
||||
border: 1px solid var(--mi);
|
||||
filter: saturate(0.9) brightness(0.95);
|
||||
}
|
||||
.mi-id { display: flex; align-items: baseline; gap: 8px; }
|
||||
.mi-name { color: var(--mi); font-size: 19px; }
|
||||
.mi-role { color: var(--dim); margin: 6px 0 2px; }
|
||||
.mi-hf { display: inline-block; margin: 1px 0 2px; font-size: 12px; color: var(--proto-cyan); text-decoration: none; }
|
||||
.mi-hf:hover { text-decoration: underline; }
|
||||
.mi-voice {
|
||||
display: block;
|
||||
appearance: none;
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 2px 0;
|
||||
margin: 4px 0;
|
||||
text-align: left;
|
||||
font: inherit;
|
||||
color: var(--mi);
|
||||
cursor: pointer;
|
||||
text-decoration: underline dotted color-mix(in srgb, var(--mi) 60%, transparent);
|
||||
}
|
||||
.mi-voice:hover { text-shadow: 0 0 8px var(--mi); }
|
||||
.mi-voice.mi-speaking { animation: mi-pulse 0.7s ease-in-out infinite; }
|
||||
@keyframes mi-pulse { 50% { text-shadow: 0 0 12px var(--mi); } }
|
||||
.mi-equiv { color: var(--head); font-style: italic; margin: 4px 0; }
|
||||
.mi-spec { color: var(--fg); margin: 4px 0; }
|
||||
.mi-inputs { color: var(--dim); }
|
||||
.mi-use { color: var(--fg); margin-top: 4px; }
|
||||
.mi-super { color: var(--link); margin-top: 6px; }
|
||||
.mi-warn { margin-top: auto; padding-top: 8px; }
|
||||
.mi-perm {
|
||||
margin-top: 8px;
|
||||
padding: 6px 8px;
|
||||
border: 1px solid var(--down);
|
||||
color: var(--down);
|
||||
font-weight: bold;
|
||||
border-radius: 4px;
|
||||
}
|
||||
.mi-comp {
|
||||
margin-top: 8px;
|
||||
padding: 6px 8px;
|
||||
border: 1px solid var(--head);
|
||||
color: var(--head);
|
||||
background: color-mix(in srgb, var(--head) 8%, transparent);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
/* Bare mode strips the model component to plain prose too. */
|
||||
html[data-bare="1"] .mi-card { border-color: #000; box-shadow: none; background: none; }
|
||||
html[data-bare="1"] .mi-art { display: none; }
|
||||
html[data-bare="1"] .mi-name { color: #000; text-shadow: none; }
|
||||
html[data-bare="1"] .mi-voice { color: #000; text-decoration: underline; }
|
||||
html[data-bare="1"] .mi-role, html[data-bare="1"] .mi-spec, html[data-bare="1"] .mi-use, html[data-bare="1"] .mi-comp,
|
||||
html[data-bare="1"] .mi-inputs, html[data-bare="1"] .mi-equiv, html[data-bare="1"] .mi-super,
|
||||
html[data-bare="1"] .mi-warn, html[data-bare="1"] .mi-hf { color: #000; }
|
||||
html[data-bare="1"] .svc-warn summary, html[data-bare="1"] .mi-warn summary,
|
||||
html[data-bare="1"] .warn-body { color: #000; }
|
||||
html[data-bare="1"] .app-art { display: none; }
|
||||
html[data-bare="1"] .svc-warn, html[data-bare="1"] .mi-perm, html[data-bare="1"] .mi-comp { color: #000; border: none; text-shadow: none; padding: 0; }
|
||||
html[data-bare="1"] .mi-perm, html[data-bare="1"] .mi-comp { border: 1px solid #000; }
|
||||
html[data-bare="1"] .mi-docs-link { color: #000; }
|
||||
|
||||
/* Protoss layer: selection + scrollbar. */
|
||||
::selection { background: color-mix(in srgb, var(--proto-blue) 45%, transparent); color: #fff; }
|
||||
::-webkit-scrollbar { width: 10px; }
|
||||
::-webkit-scrollbar-track { background: var(--bg); }
|
||||
::-webkit-scrollbar-thumb { background: var(--dim); border: 1px solid var(--border); }
|
||||
::-webkit-scrollbar-thumb:hover { background: var(--fg); }
|
||||
|
||||
/* Headings wear the console face. */
|
||||
.title, .section-title, .mi-title, .name, .mi-name, .docs h1, .docs h2 {
|
||||
font-family: var(--font-head);
|
||||
font-weight: 600;
|
||||
}
|
||||