Compare commits

..
Author SHA1 Message Date
Frank c238e32d6f fix(quant): a tap is only a tap when its rig is running
Four gates, all bought with the 2026-10-29 incident: the rig's calibration boot died of an OOM,
the chain's health loop expired without failing, and the pusher fed four rounds of 10M tokens
(86 minutes of prefill) into RESTORED PRODUCTION, recording nothing.

- run-qwen38-bf16.sh: --ngram-placement becomes a knob whose calib default is `disk` (the 95.37
  GiB table pin is what starved that boot: 230.22 GiB pinned of 246, 8.06 GiB available) while
  kldref and score keep `ram`. calib writes .rig-boot into the tap dir and retires it on a
  non-zero exit, so a dead boot cannot leave the tap looking rig-authorized.
- calib-push.py: refuses before the first prompt unless the marker exists, names this container,
  and that container is running. With the driver's existing tap-progress gate, a push can no
  longer spend hours feeding an endpoint that is not the rig.
- qwen38-calib-window.sh: the calibration window as a gated chain — stop on a failed phase
  instead of falling through the health wait, bank the 336 GiB reference container into trash
  only when both phases exit 0 and the tap holds grams, and print the whole plan on --dry-run.
- kld-gate-qwen38.sh: two-report mode (candidate vs the served payload) for the disk where the
  172.8 GiB FP8 anchor cannot live, plus the laws the report always carried and the gate never
  read: a p99.9 tail ceiling and every by_source slice passing mean, tail and top-1.
- frank-archon.md: the pipeline laws (marker, n-gram tier, three-metric acceptance) and a Files
  section that names the live campaign instead of the retired swift15 era.

Tests: 587/0 — 13 new behavioral cases (dry-run argv, marker refusals, two-report gate math).
2026-10-29 16:14:16 -07:00
42 changed files with 603 additions and 196 deletions

No files matched your search

+8 -15
View File
@@ -26,8 +26,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Payload | Port | | Target | Service | Runtime | Payload | Port |
|--------|---------|---------|---------|------| |--------|---------|---------|---------|------|
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 | | Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.18.20, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 | | Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 12118, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 | | Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
Payload identity, shape, eras, laws and rollback live in ONE page each: Payload identity, shape, eras, laws and rollback live in ONE page each:
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env` [[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
@@ -293,8 +293,8 @@ means off or NIC dead; a merely wedged kernel usually still answers ARP.
2. Bench against the service itself. No scratch ports on an LLM host — one 2. Bench against the service itself. No scratch ports on an LLM host — one
process owns the card. Use the runtime's own harness (`llama-bench`; a process owns the card. Use the runtime's own harness (`llama-bench`; a
live llama.cpp server also reports honest `timings` per request, including live llama.cpp server also reports honest `timings` per request, including
`draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork that object `draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build
only reaches you on **`/v1/chat/completions`** (measured on build 10777): a greedy 10777 that object only reaches you on **`/v1/chat/completions`**: a greedy
`ignore_eos` run on the raw `/completion` path there answers **500 "The `ignore_eos` run on the raw `/completion` path there answers **500 "The
model produced output that does not match the expected Content-only model produced output that does not match the expected Content-only
format"** (measured 2026-10-18). halogen has no `/completion` path at all, format"** (measured 2026-10-18). halogen has no `/completion` path at all,
@@ -439,16 +439,16 @@ disagrees with it is the bug.
### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771 ### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
- Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`, - Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
`scripts/frank-zealot.service`; fork build `llama-prism-0244744` (build 12118) `scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY
is the ONLY loader — `llama-prism-b10777` stays on disk as its rollback (the payload carries the fork's custom PQ2_0 tensor type plus loader (the payload carries the fork's custom PQ2_0 tensor type plus
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a `prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
loader ignoring that metadata dequantizes wrong); payload loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16 `Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests) + the rank-1 tower can crash the server with concurrent image requests) + the rank-1
refusal-direction adapter applied at load by `--lora-scaled …:1`. The refusal-direction adapter applied at load by `--lora-scaled …:1`. The
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
— and the fork BUILD is also the gate that lets it load: anything older than — and build 10777 is also the gate that lets it load: older forks refuse it
b10777 refuses it with `unsupported prism.hadamard.version: 2`. with `unsupported prism.hadamard.version: 2`.
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at - Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088 200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024, B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
@@ -741,13 +741,6 @@ linked runbooks.
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic `/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
says identity mode costs nothing, so the XDNA2 NPU returned for free. says identity mode costs nothing, so the XDNA2 NPU returned for free.
[[dankclaw-iommu-translation-off]] [[dankclaw-iommu-translation-off]]
- **A deploy handler that only `start`s a quadlet no-ops an image bump.**
`systemctl --user start` on an already-active unit does nothing, so installing
a new digest under a running service keeps the OLD container serving while the
deploy, its health gate and the Playwright suite all pass — the authentik
handler did exactly that on 2026-10-30 (2026.8.3 → 2026.8.5 moved only the
outpost, the one handler that restarted). A pin change is applied by
`restart`, never `start`.
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT - **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing. rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
@@ -23,7 +23,7 @@ Network=dankclaw_default
# Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses # Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
# the tag's nested manifest list, so pin the single-platform digest). # the tag's nested manifest list, so pin the single-platform digest).
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
Image=ghcr.io/goauthentik/ldap:2026.8.5@sha256:4c75bd27918418d2d6fbbfe4581bbd5a199fe38942d24559c4483edd64a1ea87 Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
Environment=AUTHENTIK_INSECURE=false Environment=AUTHENTIK_INSECURE=false
Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN} Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}
@@ -20,7 +20,7 @@ Wants=authentik-server.service
[Container] [Container]
ContainerName=authentik-outpost ContainerName=authentik-outpost
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5 Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
Exec=worker Exec=worker
Network=dankclaw_default Network=dankclaw_default
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -14,7 +14,7 @@ After=network.target
[Container] [Container]
ContainerName=authentik-postgres ContainerName=authentik-postgres
Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab Image=docker.io/library/postgres:16-alpine@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea
Network=dankclaw_default Network=dankclaw_default
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
Environment=POSTGRES_DB=authentik Environment=POSTGRES_DB=authentik
@@ -19,7 +19,7 @@ Wants=authentik-postgres.service
[Container] [Container]
ContainerName=authentik-server ContainerName=authentik-server
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5 Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
Exec=server Exec=server
Network=dankclaw_default Network=dankclaw_default
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -16,7 +16,7 @@ Wants=authentik-server.service
[Container] [Container]
ContainerName=authentik-worker ContainerName=authentik-worker
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5 Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
Exec=worker Exec=worker
Network=dankclaw_default Network=dankclaw_default
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
@@ -15,7 +15,7 @@ After=network.target
[Container] [Container]
ContainerName=gitea-postgres ContainerName=gitea-postgres
Network=gitea.network Network=gitea.network
Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab Image=docker.io/library/postgres:16-alpine@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea
EnvironmentFile=/home/dank/Containers/gitea/env EnvironmentFile=/home/dank/Containers/gitea/env
Environment=POSTGRES_USER=gitea Environment=POSTGRES_USER=gitea
Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD} Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD}
@@ -25,7 +25,7 @@ Description=Chrome DevTools Protocol daemon (headless-shell) for hermes
[Container] [Container]
ContainerName=chrome-devtools-protocol ContainerName=chrome-devtools-protocol
Network=host Network=host
Image=docker.io/chromedp/headless-shell@sha256:7bee201478b2f6266aae01243740ee85c16281e32f42d3aed9fdf04f4eace7a1 Image=docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04
Entrypoint=/headless-shell/headless-shell Entrypoint=/headless-shell/headless-shell
Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222 Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222
ShmSize=1073741824 ShmSize=1073741824
@@ -23,7 +23,7 @@ After=gluetun.service network.target
[Container] [Container]
ContainerName=marreta_browser ContainerName=marreta_browser
Network=gluetun.container Network=gluetun.container
Image=docker.io/lightpanda/browser:nightly@sha256:e72730e364d1fbf352c1149f29ffa4b278f3b1a55ff0dec3262298ebe624ec7a Image=docker.io/lightpanda/browser:nightly@sha256:979c36ed869f1016b8dcf22939a0633c262f2c30e892a03a9f720eba2ff1931e
[Service] [Service]
TimeoutStartSec=300 TimeoutStartSec=300
@@ -33,7 +33,7 @@ Wants=network-online.target
[Container] [Container]
ContainerName=alertmanager ContainerName=alertmanager
Image=docker.io/prom/alertmanager:latest@sha256:39d8bb3966c694a124428a3a88fbcffd40bd069129ecd47c059f8a3f17c55f3b Image=docker.io/prom/alertmanager:latest@sha256:e9733bafb1bdef9b00e25a21f8f99dc26a22224bf16641ad754d1649f4c3357a
Network=dankclaw_default Network=dankclaw_default
User=0:0 User=0:0
Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
@@ -34,7 +34,7 @@ Wants=network-online.target prometheus.service
[Container] [Container]
ContainerName=grafana ContainerName=grafana
Image=docker.io/grafana/grafana:latest@sha256:488984b716c0862639f91a698adbe2862677a5a07bad42da5f03f3030883b205 Image=docker.io/grafana/grafana:latest@sha256:dbbc38ea21103cefb50396340cd88afec779c7e070b0a7bf5a75b4ce46651d3f
Network=dankclaw_default Network=dankclaw_default
User=0:0 User=0:0
PublishPort=127.0.0.1:3002:3000 PublishPort=127.0.0.1:3002:3000
@@ -27,7 +27,7 @@ After=network.target
[Container] [Container]
ContainerName=open-webui ContainerName=open-webui
Network=open-webui.network Network=open-webui.network
Image=ghcr.io/open-webui/open-webui:main@sha256:0b399a8ba34558e181d316750455f2a95376a38cd60a61b9c942e138898944bd Image=ghcr.io/open-webui/open-webui:main@sha256:4860ba005ae8f19f08d1e12b0868855b7b339ea0dad9927f196820fc0de04575
PublishPort=3005:8080 PublishPort=3005:8080
Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770 Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
Environment=ANONYMIZED_TELEMETRY=False Environment=ANONYMIZED_TELEMETRY=False
+1 -1
View File
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
[Container] [Container]
ContainerName=gluetun ContainerName=gluetun
Network=searxng.network Network=searxng.network
Image=docker.io/qmcgaw/gluetun:latest@sha256:4af0c804003481e100180e1f8d4d5e9c48163fedc0f88d061881b615aef9d183 Image=docker.io/qmcgaw/gluetun:latest@sha256:c2ee4191762fca6b6a3c3359a731198a95d9c9af4db32d466e223bf3a84e779b
AddCapability=NET_ADMIN AddCapability=NET_ADMIN
AddDevice=/dev/net/tun:/dev/net/tun AddDevice=/dev/net/tun:/dev/net/tun
Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER} Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
ContainerName=searxng-valkey ContainerName=searxng-valkey
Network=searxng.network Network=searxng.network
NetworkAlias=valkey NetworkAlias=valkey
Image=docker.io/valkey/valkey:9-alpine@sha256:bf09d58e96d6b4524a15396b65b242bbe18beebbcc15a04d8ee75ddac62fae0b Image=docker.io/valkey/valkey:9-alpine@sha256:48332870af354a799964c0012ae1194a0bf2bf894eb508f945810596dc2d8d11
Volume=/opt/searxng/valkey:/data Volume=/opt/searxng/valkey:/data
[Service] [Service]
+1 -1
View File
@@ -16,7 +16,7 @@ After=gluetun.service searxng-valkey.service network.target
[Container] [Container]
ContainerName=searxng ContainerName=searxng
Network=gluetun.container Network=gluetun.container
Image=ghcr.io/searxng/searxng:latest@sha256:f4aa660c54d062dedb268be84e821ea24e39f48c7a496af8dd02cc9eedb93847 Image=ghcr.io/searxng/searxng:latest@sha256:5ed0323b5f5f8701264ef65e239a2bd83cb57098f2b971fa30bf79c06eeae271
Environment=SEARXNG_SECRET=${SEARXNG_SECRET} Environment=SEARXNG_SECRET=${SEARXNG_SECRET}
Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL} Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL}
Volume=/opt/searxng/config:/etc/searxng Volume=/opt/searxng/config:/etc/searxng
+1 -1
View File
@@ -29,7 +29,7 @@ Wants=network-online.target
[Container] [Container]
ContainerName=seerr ContainerName=seerr
Image=ghcr.io/seerr-team/seerr:v3.5.1@sha256:d8fab675b55ad293ded98fd71efcfd46054cbad9746534ae1cc4794a88c0b3be Image=ghcr.io/seerr-team/seerr:v3.4.1@sha256:f4768de5f616248d723e05891f3345a1402123775d03bf0890dbfedc0831bda1
Network=dankclaw_default Network=dankclaw_default
PublishPort=127.0.0.1:5055:5055 PublishPort=127.0.0.1:5055:5055
Volume=/home/dank/Containers/seerr/data:/app/config Volume=/home/dank/Containers/seerr/data:/app/config
@@ -43,7 +43,7 @@ After=network.target
[Container] [Container]
ContainerName=calibre-server ContainerName=calibre-server
Image=lscr.io/linuxserver/calibre:9.17.0@sha256:b198fc03a6abc6be7e251a9958e50e74634b00e447c7b25e92c191c8077ba378 Image=lscr.io/linuxserver/calibre:9.15.0@sha256:59e4b0e036e62f228f8bcfae1ade42f47047719885fe454ddbad38e4597b987b
Network=pasta Network=pasta
EnvironmentFile=/home/dank/Containers/calibre-server/env EnvironmentFile=/home/dank/Containers/calibre-server/env
Environment=PUID=1000 Environment=PUID=1000
@@ -16,7 +16,7 @@ After=network.target
[Container] [Container]
ContainerName=calibre-web ContainerName=calibre-web
Image=lscr.io/linuxserver/calibre-web:0.6.27-ls407@sha256:470b4ed9a7d6ea1475fef17dbcf957cb60880910296d226bd0dca652c05f68ca Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402@sha256:802729966faed5e32d671cd4f92ba6196e9e754a6c0f740ce1ee9acd86b10e55
Network=pasta Network=pasta
Environment=DOCKER_MODS=linuxserver/mods:universal-calibre Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
Environment=LANG=en_US.UTF-8 Environment=LANG=en_US.UTF-8
@@ -16,7 +16,7 @@ After=network.target
[Container] [Container]
ContainerName=jellyfin ContainerName=jellyfin
Image=ghcr.io/jellyfin/jellyfin:latest@sha256:fea863d0cd9ec445985d9c75fc6ed64e38a1d2766cf0c250a33d553f4fc1b771 Image=ghcr.io/jellyfin/jellyfin:latest@sha256:c3efa69a22f2691fc88d6bb035e87a8560e395a6047f062fb381c61c3185e8f5
Network=dankserver_appnet Network=dankserver_appnet
Environment=DEBIAN_FRONTEND=noninteractive Environment=DEBIAN_FRONTEND=noninteractive
Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
+1 -1
View File
@@ -4,7 +4,7 @@ After=network-online.target
Wants=network-online.target Wants=network-online.target
[Container] [Container]
Image=lscr.io/linuxserver/plex:1.43.5.11079-b3b13e506-ls330@sha256:01a53d9c7adcbb3701fbb0a4230c0a37c85453759b6b05a67356c3c7a09de08d Image=lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329@sha256:8f655bad7dcbc5398a78f70a5af12600b4ed3424f901566ed1eaa6710e9e2334
ContainerName=plex ContainerName=plex
Network=dankserver_appnet:alias=plex Network=dankserver_appnet:alias=plex
Environment=PUID=0 Environment=PUID=0
@@ -14,7 +14,7 @@ After=network.target
[Container] [Container]
ContainerName=romm-mariadb ContainerName=romm-mariadb
Image=docker.io/library/mariadb:11@sha256:c4124d1afa875e47f2892edf26b482fa54f796e2f342d3717a04ca091a22a9bd Image=docker.io/library/mariadb:11@sha256:f16d7a344413e519dff518f27bdb221388f84b0d274c97c4fb5f2cb0dc1b34fa
Network=dankserver_appnet:alias=mariadb Network=dankserver_appnet:alias=mariadb
EnvironmentFile=/home/dank/Containers/romm/env EnvironmentFile=/home/dank/Containers/romm/env
Environment=MARIADB_DATABASE=romm Environment=MARIADB_DATABASE=romm
+1 -1
View File
@@ -12,7 +12,7 @@ After=network.target
[Container] [Container]
ContainerName=romm-redis ContainerName=romm-redis
Image=docker.io/library/redis:7-alpine@sha256:8da19c7399dca900c6822a1cf30f9d2285585e2c4b457ff4095b0f4500054d3f Image=docker.io/library/redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499
Network=dankserver_appnet:alias=redis Network=dankserver_appnet:alias=redis
Volume=/home/dank/Containers/romm/redis:/data Volume=/home/dank/Containers/romm/redis:/data
HealthCmd=redis-cli ping HealthCmd=redis-cli ping
+1 -1
View File
@@ -18,7 +18,7 @@ After=romm-mariadb.service romm-redis.service network.target
[Container] [Container]
ContainerName=romm ContainerName=romm
Image=ghcr.io/rommapp/romm:latest@sha256:8677fb7b81b1d3db20b849e989a1dadc2a11159e51eb34a2acdcb01102274edf Image=ghcr.io/rommapp/romm:latest@sha256:03295f95a6290e752641f9553ac4198b0e5b71b879d57f7b2371b98f608feb54
Network=dankserver_appnet Network=dankserver_appnet
EnvironmentFile=/home/dank/Containers/romm/env EnvironmentFile=/home/dank/Containers/romm/env
Environment=DB_HOST=mariadb Environment=DB_HOST=mariadb
@@ -4,7 +4,7 @@ After=network-online.target
Wants=network-online.target Wants=network-online.target
[Container] [Container]
Image=lscr.io/linuxserver/tautulli:v2.18.2-ls250@sha256:529c7eabcc40554c1dc48f33f1f3cc3b1438462d73e0ec2dac8ac0b66aeeb8d4 Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245@sha256:c66d3afa8f7a701a48dd1b1f57b22eca0483d895d670fe11ecff783f6a9bbd73
ContainerName=tautulli ContainerName=tautulli
Network=dankserver_appnet:alias=tautulli Network=dankserver_appnet:alias=tautulli
Environment=PUID=0 Environment=PUID=0
+42
View File
@@ -25,6 +25,11 @@ Mechanics notes:
- a round that seals no tap bytes is a FAILURE, not a retry: the engine - a round that seals no tap bytes is a FAILURE, not a retry: the engine
answering 200 while the tap directory is wrong, full or read-only is answering 200 while the tap directory is wrong, full or read-only is
how a corpus gets pushed four times for nothing. how a corpus gets pushed four times for nothing.
- before any prompt, the tap must carry the rig's own marker
(`--tap-marker`, written by run-qwen38-bf16.sh calib) naming THIS
container, and that container must be running. Measured 2026-10-29:
with neither check, four rounds of 10M tokens went into restored
production after the rig's boot had died, and recorded nothing.
Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench] Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
[--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4] [--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4]
@@ -107,6 +112,40 @@ def tap_bytes(path):
return total return total
def require_rig(a):
"""Refuse unless the rig that opened this tap is up and serving right now.
Three facts, in order: the runner wrote its marker into the tap dir, the
marker names the container this pusher targets, and that container is
running. A marker without a live container is a boot that died; a live
container without a marker is some other engine on the port — on 2026-10-29
it was restored production, which took four rounds of 10M tokens before the
old driver admitted it had recorded nothing.
"""
marker = os.path.join(a.tap_dir, a.tap_marker)
if not os.path.isfile(marker):
qc.refuse(f"no rig marker at {marker}: this tap was not opened by "
"run-qwen38-bf16.sh calib, so a push here would feed whatever "
"answers the port and record nothing")
try:
lines = open(marker, encoding="utf-8", errors="replace").read().splitlines()
except OSError as exc:
qc.refuse(f"cannot read the rig marker {marker}: {exc}")
meta = dict(line.split("=", 1) for line in lines if "=" in line)
who = meta.get("container", "")
if who != a.container:
qc.refuse(f"the rig marker names container {who!r} and this pusher targets "
f"{a.container!r} — wrong tap, or a stale marker")
try:
p = subprocess.run(["podman", "inspect", "-f", "{{.State.Running}}", a.container],
capture_output=True, text=True, timeout=30)
except Exception as exc:
qc.refuse(f"cannot inspect the rig container {a.container}: {exc}")
if p.returncode != 0 or p.stdout.strip() != "true":
qc.refuse(f"the rig container {a.container} is not running — whatever is "
"answering the port, it is not the calibration rig")
def parse_args(): def parse_args():
ap = argparse.ArgumentParser() ap = argparse.ArgumentParser()
ap.add_argument("--port", type=int, default=7770) ap.add_argument("--port", type=int, default=7770)
@@ -114,6 +153,8 @@ def parse_args():
ap.add_argument("--container", default="arbench") ap.add_argument("--container", default="arbench")
ap.add_argument("--model", default="frank-archon") ap.add_argument("--model", default="frank-archon")
ap.add_argument("--tap-dir", default="/home/dank/campaign/qwen38-w5nl64/calib") ap.add_argument("--tap-dir", default="/home/dank/campaign/qwen38-w5nl64/calib")
ap.add_argument("--tap-marker", default=".rig-boot",
help="the file run-qwen38-bf16.sh calib writes into the tap dir; without it the pusher refuses, because an unmarked tap is not a rig")
ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl") ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl")
ap.add_argument("--max-rounds", type=int, default=4) ap.add_argument("--max-rounds", type=int, default=4)
return ap.parse_args() return ap.parse_args()
@@ -136,6 +177,7 @@ def main():
a = parse_args() a = parse_args()
url = f"http://127.0.0.1:{a.port}/v1/completions" url = f"http://127.0.0.1:{a.port}/v1/completions"
prompts = load_prompts(a) prompts = load_prompts(a)
require_rig(a)
chars = sum(len(p) for p in prompts) chars = sum(len(p) for p in prompts)
print(f"corpus: {len(prompts)} prompts, ~{chars // 4} tokens (chars/4)", print(f"corpus: {len(prompts)} prompts, ~{chars // 4} tokens (chars/4)",
flush=True) flush=True)
+1 -1
View File
@@ -21,7 +21,7 @@
# deployed knob has since dropped to 262144. # deployed knob has since dropped to 262144.
set -euo pipefail set -euo pipefail
IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0 IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
CUR=/home/dank/models/halogen-flashnext-v2 CUR=/home/dank/models/halogen-flashnext-v2
NEW=/home/dank/models/halogen-v2-abliterated NEW=/home/dank/models/halogen-v2-abliterated
CUR_CK=qwen38-flash-next-v2-swift15-abliterated.hgn CUR_CK=qwen38-flash-next-v2-swift15-abliterated.hgn
+4 -10
View File
@@ -1362,17 +1362,11 @@ deploy_authentik() {
# generated". Boot persistence comes from [Install] WantedBy=default.target # generated". Boot persistence comes from [Install] WantedBy=default.target
# inside each .container file, which the generator honors at user-manager # inside each .container file, which the generator honors at user-manager
# start (linger must stay on for the deploy user). # start (linger must stay on for the deploy user).
# RESTART, not start: `start` on an already-active unit is a no-op, so an systemctl --user start authentik-postgres.service
# image bump would install the new digest and keep serving the old
# container while every gate stayed green (measured 2026-10-30: server and
# worker stayed on the previous release beside an outpost that moved,
# because only the outpost handler restarts). The generator has already
# re-rendered the unit from the .container, so a restart is what applies it.
systemctl --user restart authentik-postgres.service
wait_container_healthy authentik-postgres 120 || return 1 wait_container_healthy authentik-postgres 120 || return 1
systemctl --user restart authentik-server.service systemctl --user start authentik-server.service
wait_container_healthy authentik-server 300 || return 1 wait_container_healthy authentik-server 300 || return 1
systemctl --user restart authentik-worker.service systemctl --user start authentik-worker.service
} }
deploy_authentik_outpost() { deploy_authentik_outpost() {
@@ -1488,7 +1482,7 @@ deploy_anubis() {
# artifact would leave a restart-looping unit and a dark endpoint. The # artifact would leave a restart-looping unit and a dark endpoint. The
# port gate is the same as every cutover since 2026-09-16: nothing # port gate is the same as every cutover since 2026-09-16: nothing
# starts until 7770 is released. # starts until 7770 is released.
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0}" LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
# The payload dir is its own twin because the NPU lane's containment gate below # The payload dir is its own twin because the NPU lane's containment gate below
# reads it; every artifact path under it stays a full literal as usual. # reads it; every artifact path under it stays a full literal as usual.
LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}" LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}"
+16 -33
View File
@@ -8,10 +8,8 @@
# (/opt/halogen/npu/models.txt inside the image) and refuses to load a model # (/opt/halogen/npu/models.txt inside the image) and refuses to load a model
# whose files do not match. # whose files do not match.
# #
# The manifest below IS that catalog block (dumped from the pinned 0.18.20 # The manifest below IS that catalog block (dumped from the pinned 0.16.2
# image on 2026-10-30; the 0.16.2 block this replaced named different device # image on 2026-10-19), so the files land in the layout the entrypoint
# programs — 10 `u*.elf` and a 5,086,496 B consts.hnpb — so a version bump
# re-fetches the set or the entrypoint refuses the boot), so the files land in the layout the entrypoint
# expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and # expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and
# <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep # <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep
# /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD # /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD
@@ -59,25 +57,21 @@ manifest() {
case "$1" in case "$1" in
qwen3-embedding-0.6b) qwen3-embedding-0.6b)
NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b" NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b"
NPU_REV="8fd44c9f18861618a509bea75c51ff8a37e3bd5d" NPU_REV="4960fd9d4423dd15373ad21c83db48dbdb3b2798"
FILES=( FILES=(
"devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91" "devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91"
"devices/consts.hnpb 8495512 5af7c1f2f24d45a9054872bba882461cf7f6ffda832e785b700643d6467ad88b" "devices/consts.hnpb 5086496 ae430e41545a0b1012938cda373b0c450364d3be631111cca3a8c43679f93336"
"devices/devices.hnpm 35465 85d870f6b900de1829659955f0dc701d57984580f5e5ec251add5d6969f03e3d" "devices/devices.hnpm 24047 4e9bca9d95d084fb89ce5b99f579a00666c8ca1efae2613c53e3a4878cc9f174"
"devices/u0.elf 2975008 f74df12eb8c79e4e90cbedcc1dd8f142219772d400e3a142fde8a95378b105eb" "devices/u0.elf 4931968 b38f165886cb037b7d3951d9cb878473eaba58a432acd45416ff85fd88aa4e46"
"devices/u1.elf 3266848 ac5a3f89f934951b315b62ab53e1d0a74dbe6396134192641c405d08da332754" "devices/u1.elf 5456000 b1811feee851b8af833b37721da92099d89772110815bbbe2c3fa1c276f0ff42"
"devices/u2.elf 4015328 5e4f0d3342f262816289866d7e8c966f56a9f938c2e12e7341718cc7312aa480" "devices/u2.elf 6505552 406ea9a95b470628a7382e376f4a72cdf4d7aa4c59b4ddf0a4bdd139d63d6ff2"
"devices/u3.elf 4326976 94cf5d269fa286d91c26bf4654cedf101c047d23c4495b74c9c129cdc33b210f" "devices/u3.elf 6817216 bc205d51b523d770151a0c61968f7c70bf624412827580bb6fe0aaf4650dc2ee"
"devices/u4.elf 4953792 cbd2796050d4c675f5cb3da117111087d510173e26f262f1b36aab66dd49dc91" "devices/u4.elf 7444032 a5a91338a2603acdcd5c11df532033fda2d3f5479ea44ae87f98fafbdff9a62d"
"devices/u5.elf 4940112 7bf6a4470546bb888da02af395c07d92a24fb6d43aef27eb2151b3a9eda19e4d" "devices/u5.elf 7432416 6c2bf41835569e0eba25ded7c05c7264ae69c6700bdeb4791057d884f2a4c126"
"devices/u6.elf 6174544 10b9d50b635993e1f9c23775c0b94901d922c1e433cd10910e9f97174235f5ac" "devices/u6.elf 4941232 919d0038ca088eafddf195f3466ff38db27d3635be1cffa5d4d4b5140902a68e"
"devices/u7.elf 2983776 e0b06063b7379d05c85b3f435b819d96fda1223d8025d6d15958854b5711ace9" "devices/u7.elf 5488304 d904cf8afae9428dd167960549e18cce5441745f18c62f8c50975dffc077d3c2"
"devices/u8.elf 3298656 3bad204f782a68b97e27c1172530b13d7fbb0b495fa95f9504530a1048560644" "devices/u8.elf 4941232 db3aef69b17825854ee0dd2df475125a3f5c872f95c686793d6af1ee83596fa7"
"devices/u9.elf 2983776 875e313280358ce116e2a51a29a2c42794b5a9fadb3d48db702c90d18d838c2c" "devices/u9.elf 5488304 d182727e551084db8fb9600441d8aa373c7f09a3d267ae0a5e921ef92326db1c"
"devices/u10.elf 3298656 ef2e0f399f149a5476955bcf233c6fa2e896fcef1f86ecb0d8113c952c41a8b2"
"devices/u11.elf 3298656 68a2519eee2d6a61ccc4ca2007e3186d44c9eb4e81162d742fe2f1756ec5a528"
"devices/u12.elf 3298656 2bdc6abfa58cac1d5795a3ef52fe8a9ab4e7c2a4792822d615499a913401f728"
"devices/u13.elf 3298656 4a3ce14f7ba15987b9bcbb4a010083e8aef68cd87c20cbe43403a98263179fe2"
"qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1" "qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1"
"tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a" "tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a"
) ;; ) ;;
@@ -105,18 +99,7 @@ for MODEL in ${MODELS//,/ }; do
# 62 GiB fetch hit this on 2026-10-18). # 62 GiB fetch hit this on 2026-10-18).
url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \ url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \
|| { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; } || { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; }
# Resume ONLY into a genuinely partial file. A staged file of a curl -fL --retry 5 -C - -o "$path" "$url"
# DIFFERENT size is an artifact of another revision (a version
# bump re-cuts the device programs), and `-C -` against it
# appends the tail to the old bytes: the result is the right
# BYTE COUNT with the wrong content, which reads as a corrupt
# program, not as a fetch fault (measured 2026-10-30 on
# consts.hnpb). Fetch those from zero.
resume=()
if [ -f "$path" ] && [ "$(stat -c %s "$path")" -lt "$want" ]; then
resume=(-C -)
fi
curl -fL --retry 5 "${resume[@]+"${resume[@]}"}" -o "$path" "$url"
fi fi
fi fi
[ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; } [ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
+2 -5
View File
@@ -10,11 +10,8 @@
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the # The .hgn format is the lock: it loads ONLY in the flash-server line, so the
# engine moves with the payload, never ahead of it. # engine moves with the payload, never ahead of it.
# Image digest-pinned (a tag pull silently re-points; the digest is what was # Image digest-pinned (a tag pull silently re-points; the digest is what was
# qualified). 0.18.20 since 2026-10-30 (was 0.16.2, the model card's quickstart # qualified — 0.16.2 is the version the model card names as its quickstart pin).
# pin) — the vendor has moved two minor versions and the payload lock is what FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# the deploy's health gate plus the boot ledger decide; rollback is the
# previous digest, 0c61bf84…, on this same line.
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0
# Container identity kept from the 27B era: the unit's ExecStop names it. # Container identity kept from the 27B era: the unit's ExecStop names it.
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
# Predecessor identity the serve script clears if it is still holding the GPUs. # Predecessor identity the serve script clears if it is still holding the GPUs.
+4 -5
View File
@@ -1,12 +1,12 @@
# frank-zealot — llama.cpp (PrismML fork build 12118 = 0.7.0-dev, commit # frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
# 024474448, CUDA) on the RTX 3090, :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus # :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
# the refusal-direction adapter applied at load. # the refusal-direction adapter applied at load.
# #
# ONE OWNER: shape, payload identity, eras, laws and every measured number # ONE OWNER: shape, payload identity, eras, laws and every measured number
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the # live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
# knobs and only the reason a reader editing THIS line cannot recover # knobs and only the reason a reader editing THIS line cannot recover
# elsewhere. Where the two disagree, this file wins on values. # elsewhere. Where the two disagree, this file wins on values.
LLAMA_BIN=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.* # The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard # metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
# metadata dequantizes wrong, silently. # metadata dequantizes wrong, silently.
@@ -64,8 +64,7 @@ CACHE_TYPE_V=q8_0
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of # weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for # draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V # by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
# beside MTP needs fork build >= b10776; LLAMA_BIN is 12118 (b10777 is kept on # beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
# disk as the rollback path).
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and # SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
# every --spec-draft-*); half a set has segfaulted at load. # every --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=draft-mtp SPEC_TYPE=draft-mtp
+108 -44
View File
@@ -1,44 +1,53 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# kld-gate-qwen38.sh — RUNS ON DANKSERVER (or anywhere the reports are). The campaign's # kld-gate-qwen38.sh — RUNS ON DANKSERVER (or anywhere the reports are). The campaign's
# acceptance gate: three KLD reports against ONE reference recording, one verdict. # acceptance gate: one verdict from the KLD reports of the containers that ride one reference
# recording.
# #
# The reference is the BF16 container built from the vendor checkpoint (run-qwen38-bf16.sh # The reference is the BF16 container built from the vendor checkpoint (run-qwen38-bf16.sh
# kldref, in the same drained window that records the calibration grams). Every scored # kldref, in the same drained window that records the calibration grams). Every scored
# container is measured against that one recording, so the KL scale runs from 0 = the # container is measured against that one recording, so the KL scale runs from 0 = the
# checkpoint itself downward in faithfulness. Three containers ride it: # checkpoint itself downward in faithfulness.
# #
# fp8.json the published FP8-native release — the achievable end of the interval # TWO MODES, because the disk decides how many anchors can exist at once:
# iq4r.json the payload frank-archon serves — the bar to beat
# cand.json the arm under test — the subject
# #
# The FP8 number is what makes the interval absolute instead of relative: "half the gap" # gate IQ4R.json CAND.json candidate vs the served payload
# is half of the distance between the 4-bit payload and the near-lossless release, not half # gate FP8.json IQ4R.json CAND.json the same, inside the absolute interval
# of an invented distance to the checkpoint. That is also why the FP8 payload is FETCHED and
# scored, never converted: it is a second opinion about what this checkpoint tolerates, not a
# step of the campaign. It lands LAST (disk sequencing) — by then the 335 GiB bf16 container
# is gone, so its 172.8 GiB fits.
# #
# The law, in order of severity, on the mean KL of one slice (--slice all | a corpus source): # iq4r.json is the payload frank-archon serves — the bar to beat. When the FP8-native release
# 1. KL(fp8) < KL(iq4r) the anchors are ordered. If the near-lossless # (172.8 GiB, fetched last because of the peak-disk law) is on disk, pass its report first:
# release does NOT sit closer to the checkpoint than # "half the gap" is then half of the distance between the 4-bit payload and the near-lossless
# the 4-bit payload, the corpus, the tokenizer or the # release, not half of an invented distance to the checkpoint. Without it the gate still
# boot shape is wrong and no arm number means anything. # answers the campaign's question — does the candidate beat what is served — and says so.
# 2. KL(fp8) < KL(cand) < KL(iq4r) the candidate lies strictly inside the interval — #
# outside it, something is mis-measured, not better # The law, on the summary of one slice (--slice all | a corpus source):
# 3. KL(cand) <= KL(iq4r) - R x gap it closes at least R (--recover, default 0.5) of the #
# gap = KL(iq4r) - KL(fp8): at least half the way from # candidate vs served (2 reports)
# what is served toward the release (agreed 2026-10-27) # 1. mean KL(cand) < mean KL(iq4r) the served payload is beaten on the corpus
# 4. top-1(cand) >= top-1(iq4r) the agreement the harness keys on does not fall # 2. p99.9(cand) <= p99.9(iq4r) and the tail is no worse — a mean can hide a few
# positions taking the whole hit
# 3. top-1(cand) >= top-1(iq4r) the agreement the harness keys on does not fall
# 4. every source slice passes 1-3 a win bought by regressing one distribution
# (code, prose, tools) is not a win
#
# inside the interval (3 reports), laws 1-4 above (1-3 renumbered to the interval and the
# recovery fraction, as agreed 2026-10-27) plus:
# 5. p99.9(cand) <= p99.9(iq4r) the tail law
# 6. every source slice passes the slice law
# #
# Booting the containers is NOT this script's job: `run-qwen38-bf16.sh score RAD OUT.json` # Booting the containers is NOT this script's job: `run-qwen38-bf16.sh score RAD OUT.json`
# writes the reports and owns the single engine argv (same pin, --tp 2, --kv-cache-dtype fp8, # writes the reports and owns the single engine argv (same pin, --tp 2, --kv-cache-dtype fp8,
# same step size, --deterministic, no speculation) that makes the three numbers comparable. # same step size, --deterministic, no speculation) that makes the numbers comparable.
# This script is arithmetic, so it re-runs against old reports and is testable without a GPU. # This script is arithmetic, so it re-runs against old reports and is testable without a GPU.
# #
# Usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F] # Usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]
# kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]
set -uo pipefail set -uo pipefail
[ "$#" -ge 3 ] || { echo "usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]" >&2; exit 2; } if [ "$#" -lt 2 ]; then
echo "usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]" >&2
echo " kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]" >&2
exit 2
fi
python3 - "$@" <<'PY' python3 - "$@" <<'PY'
import json, sys import json, sys
@@ -52,13 +61,15 @@ while i < len(a):
else: else:
files.append(a[i]) files.append(a[i])
i += 1 i += 1
if len(files) != 3: if len(files) not in (2, 3):
sys.exit("usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]") sys.exit("usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]\n"
" kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]")
o = dict(zip(opts[0::2], opts[1::2])) o = dict(zip(opts[0::2], opts[1::2]))
slice_ = o.get("--slice", "all") slice_ = o.get("--slice", "all")
recover = float(o.get("--recover", 0.5)) recover = float(o.get("--recover", 0.5))
if not 0.0 < recover <= 1.0: if not 0.0 < recover <= 1.0:
sys.exit(f"--recover must be in (0,1], got {recover}") sys.exit(f"--recover must be in (0,1], got {recover}")
interval = len(files) == 3
def die(msg): def die(msg):
@@ -66,11 +77,15 @@ def die(msg):
raise SystemExit(2) raise SystemExit(2)
def read(path): def load(path):
try: try:
d = json.load(open(path)) return json.load(open(path))
except Exception as e: except Exception as e:
die(f"cannot read the KLD report {path}: {e}") die(f"cannot read the KLD report {path}: {e}")
def summary(path, d):
"""The slice this gate grades: 'all', or one corpus source."""
if slice_ == "all": if slice_ == "all":
g = d.get("all") g = d.get("all")
else: else:
@@ -80,23 +95,25 @@ def read(path):
f"{', '.join(sorted(d.get('by_source') or {})) or '(none)'}") f"{', '.join(sorted(d.get('by_source') or {})) or '(none)'}")
if g is None or "kld" not in g or "top1_agreement" not in g: if g is None or "kld" not in g or "top1_agreement" not in g:
die(f"{path} has no {slice_!r} summary — wrong report version?") die(f"{path} has no {slice_!r} summary — wrong report version?")
return d, g return g
docs = {label: load(path) for label, path in
zip(("fp8", "iq4r", "cand") if interval else ("iq4r", "cand"), files)}
rows = [] rows = []
for label, path in zip(("fp8", "iq4r", "cand"), files): for label, path in zip(("fp8", "iq4r", "cand") if interval else ("iq4r", "cand"), files):
d, g = read(path) g = summary(path, docs[label])
rows.append((label, path, d.get("model", "?"), g["kld"]["mean"], g["kld"]["p99_9"], rows.append((label, path, docs[label].get("model", "?"), g["kld"]["mean"],
g["top1_agreement"], g["positions"])) g["kld"]["p99_9"], g["top1_agreement"], g["positions"]))
print(f"reference: BF16 container; slice: {slice_}") print(f"reference: BF16 container; slice: {slice_}; "
+ ("inside the FP8 interval" if interval else "candidate vs served (no FP8 anchor)"))
print(f"{'':6} {'model':38} {'KL mean':>10} {'KL p99.9':>10} {'top-1':>8} positions") print(f"{'':6} {'model':38} {'KL mean':>10} {'KL p99.9':>10} {'top-1':>8} positions")
for label, path, model, kl, p999, top1, n in rows: for label, path, model, kl, p999, top1, n in rows:
print(f"{label:6} {str(model)[:38]:38} {kl:10.6f} {p999:10.4f} {100*top1:7.2f}% {n}") print(f"{label:6} {str(model)[:38]:38} {kl:10.6f} {p999:10.4f} {100*top1:7.2f}% {n}")
r = {label: (kl, top1) for label, _p, _m, kl, _q, top1, _n in rows} r = {label: (kl, p999, top1) for label, _p, _m, kl, p999, top1, _n in rows}
kl_fp8, kl_iq4r, kl_cand = r["fp8"][0], r["iq4r"][0], r["cand"][0] kl_cand, p_cand, t_cand = r["cand"]
t_iq4r, t_cand = r["iq4r"][1], r["cand"][1] kl_iq4r, p_iq4r, t_iq4r = r["iq4r"]
gap = kl_iq4r - kl_fp8
ok = True ok = True
@@ -106,17 +123,64 @@ def law(n, text, passed):
print(f" [{'PASS' if passed else 'FAIL'}] {n}. {text}") print(f" [{'PASS' if passed else 'FAIL'}] {n}. {text}")
law(1, f"anchors ordered: fp8 {kl_fp8:.6f} < iq4r {kl_iq4r:.6f} (gap {gap:.6f})", def tail_law(n):
law(n, f"candidate KL p99.9 {p_cand:.4f} <= served iq4r {p_iq4r:.4f}",
p_cand <= p_iq4r)
# --- the slice law: every corpus source must pass the metric laws -------------
def slice_law(n):
"""A win that regresses one source class is not a win: check them all."""
mine = docs["cand"].get("by_source") or {}
theirs = docs["iq4r"].get("by_source") or {}
if not mine or not theirs:
law(n, "no per-source summaries to check", True)
return
missing = sorted(set(theirs) - set(mine))
extra = sorted(set(mine) - set(theirs))
if missing or extra:
law(n, "same source slices in both reports"
+ (f" (missing from cand: {', '.join(missing)})" if missing else "")
+ (f" (only in cand: {', '.join(extra)})" if extra else ""), False)
return
bad = []
for src in sorted(theirs):
c, q = mine[src], theirs[src]
worse_mean = c["kld"]["mean"] >= q["kld"]["mean"]
worse_tail = c["kld"]["p99_9"] > q["kld"]["p99_9"]
worse_top1 = c["top1_agreement"] < q["top1_agreement"]
if worse_mean or worse_tail or worse_top1:
why = ",".join(m for m, w in (("mean", worse_mean), ("p99.9", worse_tail),
("top-1", worse_top1)) if w)
bad.append(f"{src} ({why})")
law(n, "every source slice passes (mean, p99.9, top-1)"
+ (f" — regressed: {'; '.join(bad)}" if bad else ""), not bad)
if interval:
kl_fp8 = r["fp8"][0]
gap = kl_iq4r - kl_fp8
law(1, f"anchors ordered: fp8 {kl_fp8:.6f} < iq4r {kl_iq4r:.6f} (gap {gap:.6f})",
gap > 0) gap > 0)
law(2, f"candidate inside the interval: {kl_fp8:.6f} < {kl_cand:.6f} < {kl_iq4r:.6f}", law(2, f"candidate inside the interval: {kl_fp8:.6f} < {kl_cand:.6f} < {kl_iq4r:.6f}",
kl_fp8 < kl_cand < kl_iq4r) kl_fp8 < kl_cand < kl_iq4r)
need = kl_iq4r - recover * gap need = kl_iq4r - recover * gap
law(3, f"candidate closes {100*max((kl_iq4r - kl_cand)/gap, 0):.1f}% of the " law(3, f"candidate closes {100*max((kl_iq4r - kl_cand)/gap, 0):.1f}% of the "
f"{gap:.6f} gap (needs {100*recover:.0f}%, i.e. KL <= {need:.6f})" f"{gap:.6f} gap (needs {100*recover:.0f}%, i.e. KL <= {need:.6f})"
if gap > 0 else "gap is not positive; cannot score recovery", if gap > 0 else "gap is not positive; cannot score recovery",
gap > 0 and kl_cand <= need) gap > 0 and kl_cand <= need)
law(4, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%", law(4, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%",
t_cand >= t_iq4r) t_cand >= t_iq4r)
tail_law(5)
slice_law(6)
else:
law(1, f"candidate beats the served payload: KL {kl_cand:.6f} < iq4r {kl_iq4r:.6f}",
kl_cand < kl_iq4r)
tail_law(2)
law(3, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%",
t_cand >= t_iq4r)
slice_law(4)
print("KLD GATE " + ("PASS" if ok else "FAIL")) print("KLD GATE " + ("PASS" if ok else "FAIL"))
sys.exit(0 if ok else 1) sys.exit(0 if ok else 1)
PY PY
+122
View File
@@ -0,0 +1,122 @@
#!/usr/bin/env bash
# qwen38-calib-window.sh — RUNS ON DANKSERVER. The calibration window, and nothing else:
# boot the BF16 rig, wait until it is actually serving, push the corpus until the engine says
# the tap is full, stop the rig, restore frank-archon, and bank the reference container.
#
# The reference recording is already taken (kld-ref/kld.json exists), so this window is the
# calibration tap alone — the one job that needs both cards after it. The kldref phase is NOT
# re-run here: the chained launcher that used to do that moved the banked kld-ref into trash
# first, which is how a 16-hour phase gets paid for twice.
#
# Every phase is gated on the previous one, because the last time this ran unguarded
# (2026-10-29) the rig's boot died of an OOM, the health loop quietly fell through, and the
# pusher fed four rounds of 10M tokens into RESTORED PRODUCTION with nothing recorded:
#
# 1. the runner must still be alive and its container must answer /health, or this script
# kills it, restores the service and exits non-zero — no push, ever;
# 2. the pusher refuses on its own unless the rig wrote its marker into the tap dir, the
# marker names this container, and the container is running (scripts/calib-push.py
# --tap-marker), so a second lock survives a mistake in this file;
# 3. the reference container is banked into trash only when BOTH the serve and the push
# exited 0 and the tap dir holds grams. The checkpoint that reproduces it stays.
#
# knobs: Q38_WINDOW_CALIB_CORPUS (calib-7168.jsonl — the re-chunked file the rig's pool can
# hold), Q38_WINDOW_HEALTH_TIMEOUT (seconds to wait for /health), Q38_WINDOW_ROUNDS
# (pusher rounds; one is 10M tokens), Q38_WINDOW_KEEP_RAD=1 to leave the .rad in place.
# Usage: scripts/qwen38-calib-window.sh [--dry-run]
set -uo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 9
DRY=0
[ "${1:-}" = "--dry-run" ] && DRY=1
say() { printf '%s %s\n' "$(date -Is)" "$*"; }
MODELS="${FRANK_QWEN38_MODELS:-/home/dank/models}"
CAMPAIGN="${FRANK_QWEN38_CAMPAIGN:-/home/dank/campaign/qwen38-w5nl64}"
RAD="${FRANK_QWEN38_RAD:-$MODELS/rad/qwen38-bf16.rad}"
CORPUS="${Q38_WINDOW_CALIB_CORPUS:-/home/dank/corpora/out/calib-7168.jsonl}"
KLDREF="$CAMPAIGN/kld-ref/kld.json"
CALIB="$CAMPAIGN/calib"
HEALTH_TIMEOUT="${Q38_WINDOW_HEALTH_TIMEOUT:-3600}"
ROUNDS="${Q38_WINDOW_ROUNDS:-6}"
RIG=qwen38-bf16
SERVICE=frank-archon
PY="${PY:-python3}"
# ---- preflight: everything this window needs, before a card is touched -------
[ -f "$KLDREF" ] || { echo "REFUSED: no reference recording at $KLDREF — record it first (scripts/run-qwen38-bf16.sh kldref); this window only fills the calibration tap" >&2; exit 2; }
[ -f "$RAD" ] || { echo "REFUSED: no BF16 reference container at $RAD — nothing to serve; scripts/build-qwen38-bf16-rad.sh" >&2; exit 2; }
[ -f "$CORPUS" ] || { echo "REFUSED: no calibration corpus at $CORPUS" >&2; exit 2; }
[ -d "$CALIB" ] || { echo "REFUSED: no tap dir at $CALIB" >&2; exit 2; }
[ -w "$CALIB" ] || { echo "REFUSED: tap dir $CALIB is not writable by $(id -un)" >&2; exit 2; }
! [[ "$HEALTH_TIMEOUT" =~ ^[0-9]+$ ]] && { echo "REFUSED: Q38_WINDOW_HEALTH_TIMEOUT must be seconds (got: $HEALTH_TIMEOUT)" >&2; exit 2; }
! [[ "$ROUNDS" =~ ^[0-9]+$ ]] && { echo "REFUSED: Q38_WINDOW_ROUNDS must be a round count (got: $ROUNDS)" >&2; exit 2; }
if [ "$DRY" = 1 ]; then
echo "dry run: calib window — the service is left running, nothing is mounted or written"
echo "would run: bash scripts/run-qwen38-bf16.sh calib (rig boots, writes $CALIB/.rig-boot)"
echo "would wait: /health up within ${HEALTH_TIMEOUT}s, else kill the rig, restore $SERVICE, exit 1"
echo "would run: $PY scripts/calib-push.py --container $RIG --corpus $CORPUS --max-rounds $ROUNDS"
echo "would bank: $RAD -> ~/.AGENTTRASH/$(date +%Y%m%d)-qwen38-bf16-rad/ (only if both phases exit 0)"
exit 0
fi
say "calib window start pid=$$ corpus=$CORPUS rounds=$ROUNDS"
[ -n "$(ls -A "$CALIB" 2>/dev/null)" ] && say "tap dir $CALIB is not empty; the engine's token floor is pooled, so an earlier partial tap still counts"
# ---- phase 1: the rig -------------------------------------------------------
say "phase=calib serve begin"
bash scripts/run-qwen38-bf16.sh calib & spid=$!
rig_up() {
podman inspect -f '{{.State.Running}}' "$RIG" 2>/dev/null | grep -qx true \
&& curl -sf --max-time 10 -o /dev/null http://127.0.0.1:7770/health
}
waited=0
until rig_up; do
if ! kill -0 "$spid" 2>/dev/null; then
wait "$spid"; crc=$?
say "phase=calib serve end rc=$crc (the rig exited before it ever answered /health)"
echo "FAIL: the rig boot died (rc=$crc) — nothing was pushed; read /home/dank/logs/qwen38_bf16_calib.log" >&2
exit 1
fi
if [ "$waited" -ge "$HEALTH_TIMEOUT" ]; then
say "phase=calib health never came after ${HEALTH_TIMEOUT}s; killing the rig"
podman stop "$RIG" >/dev/null 2>&1 || true
wait "$spid"
echo "FAIL: the rig never answered /health within ${HEALTH_TIMEOUT}s — nothing was pushed" >&2
exit 1
fi
sleep 15
waited=$((waited + 15))
done
say "phase=calib health ok waited=${waited}s"
# ---- phase 2: the tap -------------------------------------------------------
say "phase=calib push begin"
"$PY" scripts/calib-push.py --port 7770 --parallel 8 --container "$RIG" \
--tap-dir "$CALIB" --corpus "$CORPUS" --max-rounds "$ROUNDS"
prc=$?
say "phase=calib push end rc=$prc"
podman stop "$RIG" >/dev/null 2>&1 || true
wait "$spid"; crc=$?
say "phase=calib serve end rc=$crc"
if [ "$prc" -ne 0 ] || [ "$crc" -ne 0 ]; then
echo "FAIL: serve rc=$crc push rc=$prc — the tap is NOT banked and $RAD stays: $(basename "$RAD")" >&2
exit 1
fi
[ -n "$(ls -A "$CALIB" 2>/dev/null)" ] || { echo "FAIL: both phases exited 0 but the tap dir $CALIB is empty" >&2; exit 1; }
say "TAP BANKED: $(ls -1 "$CALIB" | wc -l) file(s) in $CALIB"
# ---- phase 3: bank the reference container ----------------------------------
if [ "${Q38_WINDOW_KEEP_RAD:-0}" = "1" ]; then
say "reference .rad kept (Q38_WINDOW_KEEP_RAD=1)"
else
T="$HOME/.AGENTTRASH/$(date +%Y%m%d)-qwen38-bf16-rad"
mkdir -p "$T" || { echo "FAIL: cannot create trash dir $T" >&2; exit 1; }
mv "$RAD" "$T/" && say "reference .rad banked in $T (the checkpoint that rebuilds it stays)"
fi
say "calib window end"
+38 -1
View File
@@ -193,6 +193,25 @@ if ! [[ "$MAX_MODEL_LEN" =~ ^[0-9]+$ ]]; then
exit 2 exit 2
fi fi
# Where the n-gram table lives. On the bf16 payload it is 95.37 GiB and `ram`
# PINS it: the calibration boot died at "startup stopped: out of memory" with
# 230.22 GiB pinned of 246.07 GiB and 8.06 GiB available (measured 2026-10-29),
# while the reference recording and every score boot had already succeeded with
# the same pool and slots. The difference that matters is the job: calib only
# needs the routed weights prefilled, so it reads the table from the container's
# own file tier and the pin never happens; kldref and score keep production's
# placement, because their numbers are only comparable when every other term of
# the shape is identical. FRANK_QWEN38_NGRAM_PLACEMENT overrides either default.
case "$MODE" in
calib) NGRAM_PLACEMENT="${FRANK_QWEN38_NGRAM_PLACEMENT:-disk}" ;;
*) NGRAM_PLACEMENT="${FRANK_QWEN38_NGRAM_PLACEMENT:-ram}" ;;
esac
case "$NGRAM_PLACEMENT" in
auto|vram|ram|disk) ;;
*) echo "REFUSED: FRANK_QWEN38_NGRAM_PLACEMENT must be auto|vram|ram|disk (got: $NGRAM_PLACEMENT)" >&2
exit 2 ;;
esac
# Every container this campaign boots is read through the ro /models mount, so # Every container this campaign boots is read through the ro /models mount, so
# the host path must live under it; the container-side spelling is what goes # the host path must live under it; the container-side spelling is what goes
# into the argv. Same rule for the container a score pass opens. # into the argv. Same rule for the container a score pass opens.
@@ -230,7 +249,7 @@ ARGS=(--tp 2 --placement expert_tiered --weights-disk-tier
--host-pool-mib "$HOST_POOL_MIB" --gpu-headroom-mib "$GPU_HEADROOM_MIB" --host-pool-mib "$HOST_POOL_MIB" --gpu-headroom-mib "$GPU_HEADROOM_MIB"
--deterministic --num-speculative-tokens 0 --deterministic --num-speculative-tokens 0
--kv-cache-dtype fp8 --max-model-len "$MAX_MODEL_LEN" --kv-cache-dtype fp8 --max-model-len "$MAX_MODEL_LEN"
--ngram-placement ram --ngram-placement "$NGRAM_PLACEMENT"
--override-chat-template /models/tpl.jinja) --override-chat-template /models/tpl.jinja)
MOUNTS=(-v "$MODELS:/models:ro") MOUNTS=(-v "$MODELS:/models:ro")
TAPENV=() TAPENV=()
@@ -289,6 +308,17 @@ case "$MODE" in
MOUNTS+=(-v "$CALIB_DIR:/data/calib") MOUNTS+=(-v "$CALIB_DIR:/data/calib")
TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib
-e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-12000000}") -e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-12000000}")
# The tap's own authorization. A pusher must see this marker before it sends
# a prompt (calib-push.py --tap-marker), it names the container so a push
# aimed anywhere else refuses, and a boot that fails removes it again below.
# Measured 2026-10-29: with no marker, the chain pushed 4 rounds x 10M tokens
# into RESTORED PRODUCTION after the rig's boot died, and nothing recorded.
if [ "$DRY" = 1 ]; then
:
else
printf 'container=%s\nimage=%s\nmode=calib\nstarted=%s\npid=%s\n' \
"$C" "$IMAGE" "$(date -Is)" "$$" > "$CALIB_DIR/.rig-boot"
fi
;; ;;
esac esac
@@ -377,5 +407,12 @@ podman rm -f "$C" >/dev/null 2>&1 || true
"${CMD[@]}" 2>&1 | tee "$LOG" "${CMD[@]}" 2>&1 | tee "$LOG"
rc=${PIPESTATUS[0]} rc=${PIPESTATUS[0]}
# A boot that failed must not leave the tap marked as rig-authorized: the marker
# is what calib-push.py trusts, and the point of it is that a dead rig can never
# be pushed into again.
if [ "$MODE" = calib ] && [ "$rc" -ne 0 ] && [ -f "$CALIB_DIR/.rig-boot" ]; then
mv -f "$CALIB_DIR/.rig-boot" "$CALIB_DIR/.rig-boot.failed-$(date +%Y%m%d-%H%M%S)" 2>/dev/null || true
echo "the rig boot failed (rc=$rc): the tap marker is retired, so no pusher can take this tap for a live rig"
fi
echo "run-qwen38-bf16 $MODE exited rc=$rc; $SERVICE restored; log: $LOG" echo "run-qwen38-bf16 $MODE exited rc=$rc; $SERVICE restored; log: $LOG"
exit "$rc" exit "$rc"
+1 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.18.20 # serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server, # (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2 # digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's # conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's
+3 -3
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Serve prism-ml/Ternary-Bonsai-2-9B-gguf (PQ2_0 ternary quant WITH the MTP head # Serve prism-ml/Ternary-Bonsai-2-9B-gguf (PQ2_0 ternary quant WITH the MTP head
# in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML # in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
# vendor build 12118 (0.7.0-dev, commit 024474448), CUDA sm_86). # vendor build 10777, CUDA sm_86).
# #
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same # 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over # evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
@@ -78,7 +78,7 @@ if [[ -f "$ENV_FILE" ]]; then
fi fi
# --- defaults (used when the env file does not set them) ------------------- # --- defaults (used when the env file does not set them) -------------------
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server}" : "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-b10777/llama-server}"
: "${MODEL_FILE:=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf}" : "${MODEL_FILE:=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf}"
: "${ALIAS:=frank-zealot}" : "${ALIAS:=frank-zealot}"
: "${BIND_ADDR:=0.0.0.0}" : "${BIND_ADDR:=0.0.0.0}"
@@ -119,7 +119,7 @@ fi
# above are what pay for it. The draft cache dtype IS a further knob # above are what pay for it. The draft cache dtype IS a further knob
# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this # (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
# build's --help) and stays at its default here. Card gate: quantized V alongside # build's --help) and stays at its default here. Card gate: quantized V alongside
# MTP needs fork build >= prism-b10776; LLAMA_BIN is 12118 (b10777 kept as rollback) # MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
# prebuilt; the type-142 loader lock keeps it pinned regardless. # prebuilt; the type-142 loader lock keeps it pinned regardless.
# SPEC_TYPE=none is the rollback and strips the whole set (flags move together). # SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
: "${SPEC_TYPE:=draft-mtp}" : "${SPEC_TYPE:=draft-mtp}"
@@ -10,7 +10,7 @@ timestamp: 2026-09-03
## Server Overview ## Server Overview
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.18.20 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771). Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
## Hardware Specifications ## Hardware Specifications
### CPU ### CPU
@@ -105,7 +105,7 @@ cable between claw and server is required for 10G.
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM. - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget. - The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.18.20** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`. - The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
- The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either. - The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18. - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage. - 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
+33 -5
View File
@@ -562,6 +562,29 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation) fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions. never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.
### The quantization pipeline
- **A calibration tap is only a tap when the rig that opened it is running.** The runner writes
`.rig-boot` into the tap dir (container, image, pid) and retires it when its boot exits non-zero;
`calib-push.py` refuses without that marker, refuses when it names another container, and refuses
when the named container is not running; `qwen38-calib-window.sh` stops on a failed phase instead
of falling through its health wait. All three exist because on 2026-10-29 the unguarded chain's
rig boot died of an OOM, the health loop expired without failing, and the pusher fed four rounds
of 10M tokens into **restored production** — 86 minutes of live prefill that recorded nothing.
- **The n-gram table's pin is what starves a rig boot, and the tier is a knob.** On the BF16 payload
the table is 95.37 GiB, `--ngram-placement ram` pins it, and the calibration boot died at
`startup stopped: out of memory` with 230.22 GiB pinned of 246.07 GiB and 8.06 GiB available —
while the reference recording and every score boot succeeded at the same pool and slot count.
Calibration only needs the routed weights prefilled, so it reads the table from the container's
file tier (`--ngram-placement disk`); kldref and score keep production's placement, because their
numbers are only comparable at an identical shape.
- **Acceptance is three metrics and every slice, not a mean.** `scripts/kld-gate-qwen38.sh` grades
mean KL, the p99.9 tail, top-1 agreement and every `by_source` slice (code, prose, tools):
a mean can hide a few positions taking the whole hit, and an overall win can be bought by
regressing one distribution. The FP8-native anchor is an interval, not a requirement — without
the 172.8 GiB fetch the gate runs in two-report mode (candidate vs the served payload) and says
so on its first line.
## Contract (the wire) ## Contract (the wire)
- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions` - **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
@@ -638,14 +661,19 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
`scripts/qwen38-froggeric-v22.5.jinja`. `scripts/qwen38-froggeric-v22.5.jinja`.
- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh` - The campaign (qwen38-w5nl64, live): `scripts/run-qwen38-bf16.sh` (kldref / calib / score — the
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh` ONE measuring shape), `scripts/qwen38-calib-window.sh` (the gated calibration window),
`scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py` `scripts/convert-qwen38-w5nl64.sh` with
`scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe` (the arm driver and its template),
`scripts/kld-gate-qwen38.sh` (acceptance), `scripts/calib-push.py`,
`scripts/make-calib-corpus.py`, `scripts/rechunk-kld-corpus.py`, `scripts/make-calib-corpus.py`, `scripts/rechunk-kld-corpus.py`,
`scripts/qwen38_common.py` (the tokenizer, JSONL and refusal plumbing those three `scripts/qwen38_common.py` (the tokenizer, JSONL and refusal plumbing those three
corpus tools share — one `n_tokens`, so every tool counts the same way), corpus tools share — one `n_tokens`, so every tool counts the same way),
`scripts/derive-protected-experts.py` `scripts/derive-protected-experts.py`, `scripts/apply-protected-cast.sh`.
`scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`. - Retired swift15-era campaign scripts, superseded by the above and kept only as reference:
`scripts/convert-swift15-w4dn8.sh`, `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
`scripts/run-swift15-bf16.sh`, `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`,
`scripts/verify-swift15-ple.py`.
- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so - Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py` it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
`scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`. `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
+14 -21
View File
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.18.20, native v2 abliterated Qwen3.8-Flash-Next (canonical)" title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it." description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
tags: tags:
- llm - llm
@@ -10,10 +10,10 @@ tags:
- gfx1151 - gfx1151
- halogen - halogen
- endpoint - endpoint
timestamp: 2026-10-30 timestamp: 2026-10-21
--- ---
# frank-dragoon — halogen-flash-server 0.18.20, native v2 abliterated (Dankclaw iGPU) # frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18 under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
@@ -33,8 +33,8 @@ flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, ver
| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON | | `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked | | `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0` Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a`
(0.18.20, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's (0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
unauthenticated token port 8730 stays on container loopback. Engine stdout → unauthenticated token port 8730 stays on container loopback. Engine stdout →
`/home/dank/logs/frank_dragoon.log`, **not** the journal. `/home/dank/logs/frank_dragoon.log`, **not** the journal.
@@ -74,7 +74,6 @@ rollback plan — recoverable only until the trash is purged.
| Window | Payload | Runtime | Shape | Replaced because | Rollback | | Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 2026-10-30 → | same v2 abliterated `.hgn` trio (payload untouched) | flash-server **0.18.20** `e35133b2…` | unchanged (2 slots, 262,144 ceiling, pool 401,408, MTP always-on) | vendor moved two minor versions past the 0.16.2 quickstart pin; the payload lock **held** — the same boot served chat, a `tool_calls` turn, vision ("Red", 132 prompt tokens) and `/v1/embeddings`, with the NPU lane's 19-file set verified against the new catalog | env line back to `0c61bf84…` + redeploy |
| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` | | 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage | | 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` | | 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
@@ -191,11 +190,9 @@ created it — earlier topologies in .
## Contract (wire) ## Contract (wire)
- Live: `/v1/models`, `/health`, `/metrics`, `/props` (model_alias, total_slots, chat_template), - Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`
POST `/v1/chat/completions`, POST `/v1/embeddings` when the NPU lane is armed; `404`: `/ping`, `/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
`/completion`; `/tokenize` answers **405 to a GET** (the route exists and wants POST). **No wall-clock plus `usage.prompt_tokens`.
`Server-Timing` header and no `/completion` timing object** — measure wall-clock plus
`usage.prompt_tokens`.
- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies - Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476 vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
@@ -235,15 +232,11 @@ created it — earlier topologies in .
XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at
([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the ([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the
same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server
is exactly what runs today. The pinned 0.18.20 image carries an 18-model catalog is exactly what runs today. The pinned 0.16.2 image already carries the five-program catalog
(`/opt/halogen/npu/models.txt`: `decider-0.8b`, `decider-4b`, `qwen3-embedding-0.6b`, `qwen3-embedding-4b`, (`/opt/halogen/npu/models.txt`: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`,
`qwen3-reranker-0.6b`, `qwen3-reranker-4b`, `qwen3guard-gen-0.6b`, `qwen3guard-stream-0.6b`, `qwen3.5-2b`) and `docs/NPU.md` at tag `v0.16.2` documents it, so **`/v1/embeddings` is one env var and three mounts
`embeddinggemma-2`, `wemm-embedding-4b`, `qwen3.5-2b`, `flux2-klein-4b`, `qwen3-asr-0.6b`, `qwen3-asr-1.7b`, away — no runtime bump needed.** Wider catalog: `qwen3-embedding-4b` and `decider-4b` from 0.17.4, `embeddinggemma-2`
`d3-lite`, `d3-nano`, `qwen3-tts-0.6b-customvoice`, `qwen3-tts-1.7b-customvoice`), so **`/v1/embeddings` is one (text ≤ 8,192 tokens, pictures and audio in one vector space) and `wemm-embedding-4b` from 0.18.3.
env var and three mounts away — no runtime bump needed.** The catalog is **not** version-stable: 0.18.20
re-cut `qwen3-embedding-0.6b`'s device programs (14 `u*.elf` + an 8,495,512 B `consts.hnpb`, where 0.16.2
had 10 and 5,086,496 B) and the entrypoint verifies size+sha256 per file before it starts, so **an engine
bump re-fetches the staged set** — `scripts/download-frank-dragoon-npu.sh` carries the catalog block to do it.
Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts** Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts**
(not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string (not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string
@@ -289,7 +282,7 @@ lane is **armed and serving** since 2026-10-19 (`HALOGEN_NPU_MODELS=qwen3-embedd
unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by
size and sha256 against the catalog and the payload mount carries the tree in. size and sha256 against the catalog and the payload mount carries the tree in.
Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `19 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0, u1, u10, u11, u12, u13, u2, u3, u4, u5, u6, u7, u8, u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector). Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `15 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0…u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector).
Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default
set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus, set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus,
+3 -11
View File
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 12118), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed" title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed"
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws." description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
tags: tags:
- llm - llm
@@ -31,7 +31,7 @@ is the bug.
| Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) | | Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) |
| Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws | | Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws |
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file | | Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file |
| Runtime | `/home/dank/GitPackages/llama-prism-0244744/bin/llama-server` — PrismML **fork**, `0.7.0-dev (build 12118, commit 024474448)`; the only loader for this payload | | Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload |
| Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` | | Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
| Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) | | Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` | | Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
@@ -136,7 +136,6 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
| Window | Payload | Runtime | Shape | Replaced because | Rollback | | Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 2026-10-30 → | same 9B MTP file, Q8_0 mmproj and refusal adapter (payload untouched) | PrismML fork **12118** (`0.7.0-dev`, `024474448`) | unchanged (3 × 200,704; f16 K + q8_0 V; MTP depth 2) | fork rebuilt from `origin/prism`; the payload gate held — 202 `pq2_0` tensors and 202 Hadamard-folded weights loaded, MTP drafting engaged, chat/tools/vision answered | `LLAMA_BIN` back to `llama-prism-b10777/llama-server` (kept on disk) + redeploy |
| 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads | | 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads |
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move | | 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move | | 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
@@ -202,14 +201,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
- `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check. - `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check.
- `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change. - `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change.
- **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason. - **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason.
- **A fork rebuild needs an install RPATH and a bin/lib-aware `LLAMA_BIN`.** `cmake --install` strips the - **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter.
build-tree rpath, so the installed `llama-server` dies on `libllama-server-impl.so` unless the configure
carries `-DCMAKE_INSTALL_RPATH='$ORIGIN;$ORIGIN/../lib'`; the installed layout is `bin/` + `lib/`, not the
flat directory b10777 shipped, so the env's `LLAMA_BIN` has to name `…/bin/llama-server`. Build 12118 was
made from `origin/prism` with the 10777 configure (Release, `GGML_CUDA=ON`, `sm_86`,
`CMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13`, Ninja) — gcc 15 is the box default and nvcc 12.4 rejects it.
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter. Build 12118 clears all
three.
## Contract ## Contract
+1 -1
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bats #!/usr/bin/env bats
# Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11) # Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11)
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the # of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.18.20, # Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE # digest-pinned), driven by the root unit's serve script — 2 slots over ONE
# 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0) # 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0)
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune # and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
+4 -15
View File
@@ -126,22 +126,11 @@ run_stage() {
@test "the shipped env points at the fork runtime, the MTP pair and the refusal adapter" { @test "the shipped env points at the fork runtime, the MTP pair and the refusal adapter" {
# The payload path IS the endpoint's identity, and the runtime is part of # The payload path IS the endpoint's identity, and the runtime is part of
# it here: the file's custom tensor type 142 + prism.hadamard.* metadata # it here: the file's custom tensor type 142 + prism.hadamard.* metadata
# make the PrismML build the only loader, and that build must be >= b10776 # make the PrismML build the only loader, and this build must be >= b10776
# because MTP beside q8_0 V is illegal below it. A retarget that moves the # because MTP beside q8_0 V is illegal below it. A retarget that moves the
# model but leaves a stock llama.cpp behind ships an endpoint that cannot # model but leaves the old build behind ships an endpoint that cannot load
# load (type 142) or cannot run the documented MTP arm (the build gate). # (type 142) or cannot run the documented MTP arm (the build gate).
# grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server$' "$ENV_FILE"
# Assert the fork FAMILY, not the directory one rebuild happens to install:
# a literal path turns every runtime move into a test edit, and the suite
# runs on a box that has no /home/dank at all, so existence cannot be
# checked here either.
local bin
bin="$(sed -n 's/^LLAMA_BIN=//p' "$ENV_FILE")"
if [[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/llama-server$ ]] &&
[[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/bin/llama-server$ ]]; then
echo "LLAMA_BIN does not name a PrismML fork build: '$bin'" >&2
return 1
fi
grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0\.gguf$' "$ENV_FILE" grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0\.gguf$' "$ENV_FILE"
grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0\.gguf$' "$ENV_FILE" grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0\.gguf$' "$ENV_FILE"
grep -qE '^LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA\.gguf$' "$ENV_FILE" grep -qE '^LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA\.gguf$' "$ENV_FILE"
+174
View File
@@ -200,6 +200,9 @@ DERIVE="$REPO_ROOT/scripts/derive-protected-experts.py"
APPLY="$REPO_ROOT/scripts/apply-protected-cast.sh" APPLY="$REPO_ROOT/scripts/apply-protected-cast.sh"
CONVERT="$REPO_ROOT/scripts/convert-qwen38-w5nl64.sh" CONVERT="$REPO_ROOT/scripts/convert-qwen38-w5nl64.sh"
GATE="$REPO_ROOT/scripts/kld-gate-qwen38.sh" GATE="$REPO_ROOT/scripts/kld-gate-qwen38.sh"
RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
PUSHER="$REPO_ROOT/scripts/calib-push.py"
WINDOW="$REPO_ROOT/scripts/qwen38-calib-window.sh"
TEMPLATE="$REPO_ROOT/scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe" TEMPLATE="$REPO_ROOT/scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe"
# make_gram <dir> <layer> <expert> <energy> <rows> — a RADGRAM2 down-Gram whose # make_gram <dir> <layer> <expert> <energy> <rows> — a RADGRAM2 down-Gram whose
@@ -525,3 +528,174 @@ gate_case() { # gate_case <kl fp8> <kl iq4r> <kl cand> <top1 cand> [extra flags.
[[ "$output" == *"has no slice 'code'"* ]] [[ "$output" == *"has no slice 'code'"* ]]
[[ "$output" == *"it has: ops"* ]] [[ "$output" == *"it has: ops"* ]]
} }
# --- the n-gram tier: the pin that starved the calibration boot ---------------
@test "the calibration boot reads the n-gram table from the file tier" {
# 2026-10-29: 95.37 GiB pinned beside a 65.59 GiB/rank pool left 8.06 GiB of
# RAM and the boot died "startup stopped: out of memory". calib only needs the
# routed weights prefilled, so it reads the table off the container's file tier.
run bash "$RUNNER" calib --dry-run
[ "$status" -eq 0 ]
grep -A1 -x -- '--ngram-placement' <<<"$output" | grep -qx 'disk'
}
@test "the recording and every score pass keep production's n-gram placement" {
# Those numbers are only comparable when the shape is identical; the tier is
# part of the shape.
run bash "$RUNNER" kldref --dry-run
[ "$status" -eq 0 ]
grep -A1 -x -- '--ngram-placement' <<<"$output" | grep -qx 'ram'
}
@test "an unknown n-gram placement is refused by name before the boot" {
run env FRANK_QWEN38_NGRAM_PLACEMENT=ssd bash "$RUNNER" calib --dry-run
[ "$status" -eq 2 ]
[[ "$output" == *"auto|vram|ram|disk"* ]]
}
# --- the pusher: a tap without a live rig is not a tap ------------------------
# stub_podman <dir> <running|absent> — a podman that answers the one question the
# pusher asks: is the rig container running?
stub_podman() {
mkdir -p "$1/bin"
if [ "$2" = running ]; then
printf '#!/usr/bin/env bash\necho true\n' > "$1/bin/podman"
elif [ "$2" = stopped ]; then
printf '#!/usr/bin/env bash\necho false\n' > "$1/bin/podman"
else
printf '#!/usr/bin/env bash\nexit 125\n' > "$1/bin/podman"
fi
chmod +x "$1/bin/podman"
}
pusher_fixture() { # pusher_fixture <marker-container or empty>
D="$BATS_TEST_TMPDIR/push"
mkdir -p "$D/tap" "$D/bin"
printf '{"prompt":"hello"}\n' > "$D/corpus.jsonl"
if [ -n "$1" ]; then
printf 'container=%s\nimage=radiance@sha256:test\nmode=calib\npid=1\n' "$1" \
> "$D/tap/.rig-boot"
fi
echo "$D"
}
@test "the pusher refuses a tap that carries no rig marker: production is not a rig" {
# With no marker, the old driver fed four rounds of 10M tokens into restored
# production after the rig's boot had died, and recorded nothing.
D="$(pusher_fixture '')"
run python3 "$PUSHER" --corpus "$D/corpus.jsonl" --tap-dir "$D/tap" --port 1
[ "$status" -eq 2 ]
[[ "$output" == *"no rig marker"* ]]
}
@test "the pusher refuses a marker that names another container" {
D="$(pusher_fixture arbench)"
run python3 "$PUSHER" --corpus "$D/corpus.jsonl" --tap-dir "$D/tap" \
--container qwen38-bf16 --port 1
[ "$status" -eq 2 ]
[[ "$output" == *"names container 'arbench'"* ]]
[[ "$output" == *"targets 'qwen38-bf16'"* ]]
}
@test "the pusher refuses when the rig container is not running" {
D="$(pusher_fixture qwen38-bf16)"
stub_podman "$D" stopped
run env PATH="$D/bin:$PATH" python3 "$PUSHER" --corpus "$D/corpus.jsonl" \
--tap-dir "$D/tap" --container qwen38-bf16 --port 1
[ "$status" -eq 2 ]
[[ "$output" == *"is not running"* ]]
}
# --- the acceptance gate without an FP8 anchor -------------------------------
# kld_report_slices <path> <model> <kl all> <kl code> <top1 all> <top1 code>
kld_report_slices() {
python3 - "$@" <<'PY'
import json, sys
path, model, kl_all, kl_code, t_all, t_code = sys.argv[1:7]
def g(kl, t, n=25000):
return {"kld": {"mean": float(kl), "p99_9": float(kl) * 4},
"top1_agreement": float(t), "positions": n}
json.dump({"model": model, "all": g(kl_all, t_all),
"by_source": {"code": g(kl_code, t_code), "ops": g(kl_all, t_all)}},
open(path, "w"))
PY
}
gate2() { # gate2 <kl iq4r> <kl cand> <top1 iq4r> <top1 cand> [extra]
D="$BATS_TEST_TMPDIR/kl2"
mkdir -p "$D"
kld_report_slices "$D/iq4r.json" iq4r-moe "$1" "$1" "$3" "$3"
kld_report_slices "$D/cand.json" w5nl64 "$2" "$2" "$4" "$4"
run bash "$GATE" "$D/iq4r.json" "$D/cand.json" "${@:5}"
}
@test "a candidate that beats the served payload on mean, tail and top-1 passes without an anchor" {
gate2 0.0190 0.0090 0.962 0.971
[ "$status" -eq 0 ]
[[ "$output" == *"candidate vs served (no FP8 anchor)"* ]]
[[ "$output" == *"KLD GATE PASS"* ]]
}
@test "a better mean with a worse tail fails: the tail is where a mean hides its damage" {
D="$BATS_TEST_TMPDIR/kl3"
mkdir -p "$D"
kld_report_slices "$D/iq4r.json" iq4r-moe 0.0190 0.0190 0.962 0.962
python3 - "$D/cand.json" <<'PY'
import json, sys
# mean better than the served payload, p99.9 worse: the shape a mean-only gate waves through
g = {"kld": {"mean": 0.0090, "p99_9": 0.4000}, "top1_agreement": 0.971, "positions": 25000}
json.dump({"model": "w5nl64", "all": g, "by_source": {"code": dict(g), "ops": dict(g)}},
open(sys.argv[1], "w"))
PY
run bash "$GATE" "$D/iq4r.json" "$D/cand.json"
[ "$status" -eq 1 ]
[[ "$output" == *"[FAIL] 2."* ]]
[[ "$output" == *"KL p99.9"* ]]
}
@test "a candidate that wins overall but regresses one source slice fails" {
D="$BATS_TEST_TMPDIR/kl4"
mkdir -p "$D"
kld_report_slices "$D/iq4r.json" iq4r-moe 0.0190 0.0190 0.962 0.962
# 'all' is better, 'code' is worse: exactly the trade a mean-only gate cannot see
kld_report_slices "$D/cand.json" w5nl64 0.0090 0.0300 0.971 0.950
run bash "$GATE" "$D/iq4r.json" "$D/cand.json"
[ "$status" -eq 1 ]
[[ "$output" == *"[PASS] 1."* ]]
[[ "$output" == *"[FAIL] 4."* ]]
[[ "$output" == *"regressed: code"* ]]
}
# --- the calibration window: gated, and it says so ---------------------------
window_fixture() {
D="$BATS_TEST_TMPDIR/win"
mkdir -p "$D/campaign/kld-ref" "$D/campaign/calib" "$D/rad" "$D/corpora"
echo '{}' > "$D/campaign/kld-ref/kld.json"
: > "$D/rad/qwen38-bf16.rad"
: > "$D/corpora/calib.jsonl"
echo "$D"
}
@test "the calibration window refuses before touching a card without a reference recording" {
D="$BATS_TEST_TMPDIR/win-less"
mkdir -p "$D/campaign/calib"
run env FRANK_QWEN38_CAMPAIGN="$D/campaign" bash "$WINDOW"
[ "$status" -eq 2 ]
[[ "$output" == *"no reference recording"* ]]
}
@test "the calibration window's dry run prints the gated plan, including what it would bank" {
D="$(window_fixture)"
run env FRANK_QWEN38_CAMPAIGN="$D/campaign" \
FRANK_QWEN38_RAD="$D/rad/qwen38-bf16.rad" \
Q38_WINDOW_CALIB_CORPUS="$D/corpora/calib.jsonl" bash "$WINDOW" --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"run-qwen38-bf16.sh calib"* ]]
[[ "$output" == *"else kill the rig"* ]]
[[ "$output" == *"calib-push.py"* ]]
[[ "$output" == *"AGENTTRASH"* ]]
}