Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
19fdffe8bd | ||
|
|
f7aec7f154 | ||
|
|
e1d4d59773 | ||
|
|
acbb539b97 | ||
|
|
011ceeb344 | ||
|
|
33396d2daa | ||
|
|
58184008dd | ||
|
|
da5eb968f9 | ||
|
|
b3052e9243 | ||
|
|
3e5a37a21f | ||
|
|
1e4fa14772 |
No files matched your search
@@ -26,8 +26,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
| Target | Service | Runtime | Payload | Port |
|
||||
|--------|---------|---------|---------|------|
|
||||
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
|
||||
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
|
||||
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
|
||||
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.18.20, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
|
||||
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 12118, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
|
||||
|
||||
Payload identity, shape, eras, laws and rollback live in ONE page each:
|
||||
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
|
||||
@@ -293,8 +293,8 @@ means off or NIC dead; a merely wedged kernel usually still answers ARP.
|
||||
2. Bench against the service itself. No scratch ports on an LLM host — one
|
||||
process owns the card. Use the runtime's own harness (`llama-bench`; a
|
||||
live llama.cpp server also reports honest `timings` per request, including
|
||||
`draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build
|
||||
10777 that object only reaches you on **`/v1/chat/completions`**: a greedy
|
||||
`draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork that object
|
||||
only reaches you on **`/v1/chat/completions`** (measured on build 10777): a greedy
|
||||
`ignore_eos` run on the raw `/completion` path there answers **500 "The
|
||||
model produced output that does not match the expected Content-only
|
||||
format"** (measured 2026-10-18). halogen has no `/completion` path at all,
|
||||
@@ -439,16 +439,16 @@ disagrees with it is the bug.
|
||||
### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
|
||||
|
||||
- Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
|
||||
`scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY
|
||||
loader (the payload carries the fork's custom PQ2_0 tensor type plus
|
||||
`scripts/frank-zealot.service`; fork build `llama-prism-0244744` (build 12118)
|
||||
is the ONLY loader — `llama-prism-b10777` stays on disk as its rollback (the payload carries the fork's custom PQ2_0 tensor type plus
|
||||
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
|
||||
loader ignoring that metadata dequantizes wrong); payload
|
||||
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
|
||||
tower can crash the server with concurrent image requests) + the rank-1
|
||||
refusal-direction adapter applied at load by `--lora-scaled …:1`. The
|
||||
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
|
||||
— and build 10777 is also the gate that lets it load: older forks refuse it
|
||||
with `unsupported prism.hadamard.version: 2`.
|
||||
— and the fork BUILD is also the gate that lets it load: anything older than
|
||||
b10777 refuses it with `unsupported prism.hadamard.version: 2`.
|
||||
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
|
||||
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
|
||||
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
|
||||
@@ -741,6 +741,13 @@ linked runbooks.
|
||||
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
|
||||
says identity mode costs nothing, so the XDNA2 NPU returned for free.
|
||||
[[dankclaw-iommu-translation-off]]
|
||||
- **A deploy handler that only `start`s a quadlet no-ops an image bump.**
|
||||
`systemctl --user start` on an already-active unit does nothing, so installing
|
||||
a new digest under a running service keeps the OLD container serving while the
|
||||
deploy, its health gate and the Playwright suite all pass — the authentik
|
||||
handler did exactly that on 2026-10-30 (2026.8.3 → 2026.8.5 moved only the
|
||||
outpost, the one handler that restarted). A pin change is applied by
|
||||
`restart`, never `start`.
|
||||
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
|
||||
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
|
||||
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
|
||||
|
||||
@@ -23,7 +23,7 @@ Network=dankclaw_default
|
||||
# Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
|
||||
# the tag's nested manifest list, so pin the single-platform digest).
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
|
||||
Image=ghcr.io/goauthentik/ldap:2026.8.5@sha256:4c75bd27918418d2d6fbbfe4581bbd5a199fe38942d24559c4483edd64a1ea87
|
||||
Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
|
||||
Environment=AUTHENTIK_INSECURE=false
|
||||
Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}
|
||||
|
||||
@@ -20,7 +20,7 @@ Wants=authentik-server.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-outpost
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
|
||||
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
|
||||
Exec=worker
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
|
||||
@@ -14,7 +14,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-postgres
|
||||
Image=docker.io/library/postgres:16-alpine@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea
|
||||
Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
Environment=POSTGRES_DB=authentik
|
||||
|
||||
@@ -19,7 +19,7 @@ Wants=authentik-postgres.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-server
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
|
||||
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
|
||||
Exec=server
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
|
||||
@@ -16,7 +16,7 @@ Wants=authentik-server.service
|
||||
|
||||
[Container]
|
||||
ContainerName=authentik-worker
|
||||
Image=ghcr.io/goauthentik/server:2026.8.3@sha256:ab9b4e8cc4ab3f8d1198d2db6aeea66bafea1963b3f2843589e0d163f97d9849
|
||||
Image=ghcr.io/goauthentik/server:2026.8.5@sha256:602227a86455d6772e3d16b7d852713c4f864f40df5df894abd1285bb53979a5
|
||||
Exec=worker
|
||||
Network=dankclaw_default
|
||||
EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
|
||||
|
||||
@@ -15,7 +15,7 @@ After=network.target
|
||||
[Container]
|
||||
ContainerName=gitea-postgres
|
||||
Network=gitea.network
|
||||
Image=docker.io/library/postgres:16-alpine@sha256:721873c34ceb9f8d8fc265984940dc982404c105f19ad51be9fdc5970a6080ea
|
||||
Image=docker.io/library/postgres:16-alpine@sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab
|
||||
EnvironmentFile=/home/dank/Containers/gitea/env
|
||||
Environment=POSTGRES_USER=gitea
|
||||
Environment=POSTGRES_PASSWORD=${GITEA_DB_PASSWORD}
|
||||
|
||||
@@ -25,7 +25,7 @@ Description=Chrome DevTools Protocol daemon (headless-shell) for hermes
|
||||
[Container]
|
||||
ContainerName=chrome-devtools-protocol
|
||||
Network=host
|
||||
Image=docker.io/chromedp/headless-shell@sha256:2d349b544a1ea6b5b5fd7c0fe99215ff662339c57407ee2e8c0a11af93516b04
|
||||
Image=docker.io/chromedp/headless-shell@sha256:7bee201478b2f6266aae01243740ee85c16281e32f42d3aed9fdf04f4eace7a1
|
||||
Entrypoint=/headless-shell/headless-shell
|
||||
Exec=--no-sandbox --disable-gpu --use-gl=angle --use-angle=swiftshader --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222
|
||||
ShmSize=1073741824
|
||||
|
||||
@@ -23,7 +23,7 @@ After=gluetun.service network.target
|
||||
[Container]
|
||||
ContainerName=marreta_browser
|
||||
Network=gluetun.container
|
||||
Image=docker.io/lightpanda/browser:nightly@sha256:979c36ed869f1016b8dcf22939a0633c262f2c30e892a03a9f720eba2ff1931e
|
||||
Image=docker.io/lightpanda/browser:nightly@sha256:e72730e364d1fbf352c1149f29ffa4b278f3b1a55ff0dec3262298ebe624ec7a
|
||||
|
||||
[Service]
|
||||
TimeoutStartSec=300
|
||||
|
||||
@@ -33,7 +33,7 @@ Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=alertmanager
|
||||
Image=docker.io/prom/alertmanager:latest@sha256:e9733bafb1bdef9b00e25a21f8f99dc26a22224bf16641ad754d1649f4c3357a
|
||||
Image=docker.io/prom/alertmanager:latest@sha256:39d8bb3966c694a124428a3a88fbcffd40bd069129ecd47c059f8a3f17c55f3b
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
|
||||
|
||||
@@ -34,7 +34,7 @@ Wants=network-online.target prometheus.service
|
||||
|
||||
[Container]
|
||||
ContainerName=grafana
|
||||
Image=docker.io/grafana/grafana:latest@sha256:dbbc38ea21103cefb50396340cd88afec779c7e070b0a7bf5a75b4ce46651d3f
|
||||
Image=docker.io/grafana/grafana:latest@sha256:488984b716c0862639f91a698adbe2862677a5a07bad42da5f03f3030883b205
|
||||
Network=dankclaw_default
|
||||
User=0:0
|
||||
PublishPort=127.0.0.1:3002:3000
|
||||
|
||||
@@ -27,7 +27,7 @@ After=network.target
|
||||
[Container]
|
||||
ContainerName=open-webui
|
||||
Network=open-webui.network
|
||||
Image=ghcr.io/open-webui/open-webui:main@sha256:4860ba005ae8f19f08d1e12b0868855b7b339ea0dad9927f196820fc0de04575
|
||||
Image=ghcr.io/open-webui/open-webui:main@sha256:0b399a8ba34558e181d316750455f2a95376a38cd60a61b9c942e138898944bd
|
||||
PublishPort=3005:8080
|
||||
Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
|
||||
Environment=ANONYMIZED_TELEMETRY=False
|
||||
|
||||
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
|
||||
[Container]
|
||||
ContainerName=gluetun
|
||||
Network=searxng.network
|
||||
Image=docker.io/qmcgaw/gluetun:latest@sha256:c2ee4191762fca6b6a3c3359a731198a95d9c9af4db32d466e223bf3a84e779b
|
||||
Image=docker.io/qmcgaw/gluetun:latest@sha256:4af0c804003481e100180e1f8d4d5e9c48163fedc0f88d061881b615aef9d183
|
||||
AddCapability=NET_ADMIN
|
||||
AddDevice=/dev/net/tun:/dev/net/tun
|
||||
Environment=VPN_SERVICE_PROVIDER=${VPN_SERVICE_PROVIDER}
|
||||
|
||||
@@ -16,7 +16,7 @@ After=searxng-network.service network.target
|
||||
ContainerName=searxng-valkey
|
||||
Network=searxng.network
|
||||
NetworkAlias=valkey
|
||||
Image=docker.io/valkey/valkey:9-alpine@sha256:48332870af354a799964c0012ae1194a0bf2bf894eb508f945810596dc2d8d11
|
||||
Image=docker.io/valkey/valkey:9-alpine@sha256:bf09d58e96d6b4524a15396b65b242bbe18beebbcc15a04d8ee75ddac62fae0b
|
||||
Volume=/opt/searxng/valkey:/data
|
||||
|
||||
[Service]
|
||||
|
||||
@@ -16,7 +16,7 @@ After=gluetun.service searxng-valkey.service network.target
|
||||
[Container]
|
||||
ContainerName=searxng
|
||||
Network=gluetun.container
|
||||
Image=ghcr.io/searxng/searxng:latest@sha256:5ed0323b5f5f8701264ef65e239a2bd83cb57098f2b971fa30bf79c06eeae271
|
||||
Image=ghcr.io/searxng/searxng:latest@sha256:f4aa660c54d062dedb268be84e821ea24e39f48c7a496af8dd02cc9eedb93847
|
||||
Environment=SEARXNG_SECRET=${SEARXNG_SECRET}
|
||||
Environment=SEARXNG_BASE_URL=${SEARXNG_BASE_URL}
|
||||
Volume=/opt/searxng/config:/etc/searxng
|
||||
|
||||
@@ -29,7 +29,7 @@ Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=seerr
|
||||
Image=ghcr.io/seerr-team/seerr:v3.4.1@sha256:f4768de5f616248d723e05891f3345a1402123775d03bf0890dbfedc0831bda1
|
||||
Image=ghcr.io/seerr-team/seerr:v3.5.1@sha256:d8fab675b55ad293ded98fd71efcfd46054cbad9746534ae1cc4794a88c0b3be
|
||||
Network=dankclaw_default
|
||||
PublishPort=127.0.0.1:5055:5055
|
||||
Volume=/home/dank/Containers/seerr/data:/app/config
|
||||
|
||||
@@ -43,7 +43,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=calibre-server
|
||||
Image=lscr.io/linuxserver/calibre:9.15.0@sha256:59e4b0e036e62f228f8bcfae1ade42f47047719885fe454ddbad38e4597b987b
|
||||
Image=lscr.io/linuxserver/calibre:9.17.0@sha256:b198fc03a6abc6be7e251a9958e50e74634b00e447c7b25e92c191c8077ba378
|
||||
Network=pasta
|
||||
EnvironmentFile=/home/dank/Containers/calibre-server/env
|
||||
Environment=PUID=1000
|
||||
|
||||
@@ -16,7 +16,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=calibre-web
|
||||
Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402@sha256:802729966faed5e32d671cd4f92ba6196e9e754a6c0f740ce1ee9acd86b10e55
|
||||
Image=lscr.io/linuxserver/calibre-web:0.6.27-ls407@sha256:470b4ed9a7d6ea1475fef17dbcf957cb60880910296d226bd0dca652c05f68ca
|
||||
Network=pasta
|
||||
Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
|
||||
Environment=LANG=en_US.UTF-8
|
||||
|
||||
@@ -16,7 +16,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=jellyfin
|
||||
Image=ghcr.io/jellyfin/jellyfin:latest@sha256:c3efa69a22f2691fc88d6bb035e87a8560e395a6047f062fb381c61c3185e8f5
|
||||
Image=ghcr.io/jellyfin/jellyfin:latest@sha256:fea863d0cd9ec445985d9c75fc6ed64e38a1d2766cf0c250a33d553f4fc1b771
|
||||
Network=dankserver_appnet
|
||||
Environment=DEBIAN_FRONTEND=noninteractive
|
||||
Environment=DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
|
||||
|
||||
@@ -4,7 +4,7 @@ After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
Image=lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329@sha256:8f655bad7dcbc5398a78f70a5af12600b4ed3424f901566ed1eaa6710e9e2334
|
||||
Image=lscr.io/linuxserver/plex:1.43.5.11079-b3b13e506-ls330@sha256:01a53d9c7adcbb3701fbb0a4230c0a37c85453759b6b05a67356c3c7a09de08d
|
||||
ContainerName=plex
|
||||
Network=dankserver_appnet:alias=plex
|
||||
Environment=PUID=0
|
||||
|
||||
@@ -14,7 +14,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm-mariadb
|
||||
Image=docker.io/library/mariadb:11@sha256:f16d7a344413e519dff518f27bdb221388f84b0d274c97c4fb5f2cb0dc1b34fa
|
||||
Image=docker.io/library/mariadb:11@sha256:c4124d1afa875e47f2892edf26b482fa54f796e2f342d3717a04ca091a22a9bd
|
||||
Network=dankserver_appnet:alias=mariadb
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Environment=MARIADB_DATABASE=romm
|
||||
|
||||
@@ -12,7 +12,7 @@ After=network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm-redis
|
||||
Image=docker.io/library/redis:7-alpine@sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499
|
||||
Image=docker.io/library/redis:7-alpine@sha256:8da19c7399dca900c6822a1cf30f9d2285585e2c4b457ff4095b0f4500054d3f
|
||||
Network=dankserver_appnet:alias=redis
|
||||
Volume=/home/dank/Containers/romm/redis:/data
|
||||
HealthCmd=redis-cli ping
|
||||
|
||||
@@ -18,7 +18,7 @@ After=romm-mariadb.service romm-redis.service network.target
|
||||
|
||||
[Container]
|
||||
ContainerName=romm
|
||||
Image=ghcr.io/rommapp/romm:latest@sha256:03295f95a6290e752641f9553ac4198b0e5b71b879d57f7b2371b98f608feb54
|
||||
Image=ghcr.io/rommapp/romm:latest@sha256:8677fb7b81b1d3db20b849e989a1dadc2a11159e51eb34a2acdcb01102274edf
|
||||
Network=dankserver_appnet
|
||||
EnvironmentFile=/home/dank/Containers/romm/env
|
||||
Environment=DB_HOST=mariadb
|
||||
|
||||
@@ -4,7 +4,7 @@ After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245@sha256:c66d3afa8f7a701a48dd1b1f57b22eca0483d895d670fe11ecff783f6a9bbd73
|
||||
Image=lscr.io/linuxserver/tautulli:v2.18.2-ls250@sha256:529c7eabcc40554c1dc48f33f1f3cc3b1438462d73e0ec2dac8ac0b66aeeb8d4
|
||||
ContainerName=tautulli
|
||||
Network=dankserver_appnet:alias=tautulli
|
||||
Environment=PUID=0
|
||||
|
||||
+79
-18
@@ -22,6 +22,9 @@ Mechanics notes:
|
||||
prompts do not bias the Hessians (the corpus is drawn round-robin over
|
||||
source classes, so loss is class-neutral as long as failures are rare;
|
||||
a systematic 4xx is loud in the log and must be fixed, not ignored).
|
||||
- a round that seals no tap bytes is a FAILURE, not a retry: the engine
|
||||
answering 200 while the tap directory is wrong, full or read-only is
|
||||
how a corpus gets pushed four times for nothing.
|
||||
|
||||
Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
|
||||
[--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4]
|
||||
@@ -29,11 +32,18 @@ Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
|
||||
if _HERE not in sys.path:
|
||||
sys.path.insert(0, _HERE)
|
||||
|
||||
import qwen38_common as qc # noqa: E402 - sibling module, same directory
|
||||
|
||||
DONE_MARK = "calibration wrote"
|
||||
|
||||
|
||||
@@ -49,9 +59,9 @@ def tap_done(container):
|
||||
return DONE_MARK in (out.stdout + out.stderr)
|
||||
|
||||
|
||||
def push_one(url, prompt):
|
||||
def push_one(url, prompt, model):
|
||||
body = json.dumps({
|
||||
"model": "frank-archon",
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"max_tokens": 1,
|
||||
"temperature": 0,
|
||||
@@ -68,38 +78,89 @@ def push_one(url, prompt):
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
def push_round(url, prompts, model, parallel):
|
||||
"""Push the whole corpus once; returns how many requests were accepted."""
|
||||
ok = 0
|
||||
with concurrent.futures.ThreadPoolExecutor(parallel) as ex:
|
||||
for good in ex.map(lambda p: push_one(url, p, model), prompts):
|
||||
ok += 1 if good else 0
|
||||
return ok
|
||||
|
||||
|
||||
def tap_bytes(path):
|
||||
"""Bytes the tap has sealed so far; 0 when the directory is absent.
|
||||
|
||||
A tap whose directory is wrong, full or read-only stops writing while the
|
||||
engine keeps answering 200 — the driver then re-pushes the whole corpus for
|
||||
every round it is allowed and the loss is invisible until the hours are gone.
|
||||
"""
|
||||
total = 0
|
||||
try:
|
||||
for root, _, files in os.walk(path):
|
||||
for f in files:
|
||||
try:
|
||||
total += os.path.getsize(os.path.join(root, f))
|
||||
except OSError:
|
||||
pass
|
||||
except OSError:
|
||||
return 0
|
||||
return total
|
||||
|
||||
|
||||
def parse_args():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--port", type=int, default=7770)
|
||||
ap.add_argument("--parallel", type=int, default=64)
|
||||
ap.add_argument("--container", default="arbench")
|
||||
ap.add_argument("--model", default="frank-archon")
|
||||
ap.add_argument("--tap-dir", default="/home/dank/campaign/qwen38-w5nl64/calib")
|
||||
ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl")
|
||||
ap.add_argument("--max-rounds", type=int, default=4)
|
||||
a = ap.parse_args()
|
||||
url = f"http://127.0.0.1:{a.port}/v1/completions"
|
||||
return ap.parse_args()
|
||||
|
||||
|
||||
def load_prompts(a) -> list[str]:
|
||||
if not os.path.isfile(a.corpus):
|
||||
qc.refuse(f"no corpus at {a.corpus}")
|
||||
prompts = []
|
||||
with open(a.corpus) as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if line:
|
||||
prompts.append(json.loads(line)["prompt"])
|
||||
print(f"corpus: {len(prompts)} prompts", flush=True)
|
||||
for lineno, rec in qc.iter_jsonl(a.corpus):
|
||||
if not isinstance(rec.get("prompt"), str):
|
||||
qc.refuse(f"{a.corpus}:{lineno} has no string 'prompt' field")
|
||||
prompts.append(rec["prompt"])
|
||||
if not prompts:
|
||||
qc.refuse(f"the corpus at {a.corpus} holds no prompts")
|
||||
return prompts
|
||||
|
||||
|
||||
def main():
|
||||
a = parse_args()
|
||||
url = f"http://127.0.0.1:{a.port}/v1/completions"
|
||||
prompts = load_prompts(a)
|
||||
chars = sum(len(p) for p in prompts)
|
||||
print(f"corpus: {len(prompts)} prompts, ~{chars // 4} tokens (chars/4)",
|
||||
flush=True)
|
||||
|
||||
for rnd in range(1, a.max_rounds + 1):
|
||||
if tap_done(a.container):
|
||||
break
|
||||
before = tap_bytes(a.tap_dir)
|
||||
t0 = time.time()
|
||||
ok = 0
|
||||
with concurrent.futures.ThreadPoolExecutor(a.parallel) as ex:
|
||||
for good in ex.map(lambda p: push_one(url, p), prompts):
|
||||
ok += 1 if good else 0
|
||||
dt = time.time() - t0
|
||||
ok = push_round(url, prompts, a.model, a.parallel)
|
||||
dt = max(time.time() - t0, 1e-9)
|
||||
print(f"round {rnd}: {ok}/{len(prompts)} ok in {dt:.0f}s "
|
||||
f"({sum(len(p) for p in prompts)/4/dt:.0f} chars/s est)",
|
||||
flush=True)
|
||||
f"({chars / 4 / dt:.0f} tokens/s est)", flush=True)
|
||||
if ok == 0:
|
||||
print("FAIL: not one prompt was accepted; fix the engine or the argv "
|
||||
"before spending another round", flush=True)
|
||||
sys.exit(1)
|
||||
if tap_done(a.container):
|
||||
break
|
||||
after = tap_bytes(a.tap_dir)
|
||||
if after <= before:
|
||||
print(f"FAIL: the tap at {a.tap_dir} sealed nothing this round "
|
||||
f"({before} B before, {after} B after) — the engine is serving "
|
||||
"without recording (wrong dir, full disk, read-only mount)", flush=True)
|
||||
sys.exit(1)
|
||||
else:
|
||||
print("max rounds exhausted, tap not done", flush=True)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -21,7 +21,7 @@
|
||||
# deployed knob has since dropped to 262144.
|
||||
set -euo pipefail
|
||||
|
||||
IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||
IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0
|
||||
CUR=/home/dank/models/halogen-flashnext-v2
|
||||
NEW=/home/dank/models/halogen-v2-abliterated
|
||||
CUR_CK=qwen38-flash-next-v2-swift15-abliterated.hgn
|
||||
|
||||
@@ -71,10 +71,12 @@ PLAN_ONLY=0
|
||||
KEEP_RECIPE=0
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--arm) ARMS+=("$2"); shift 2 ;;
|
||||
--arm) [ "$#" -ge 2 ] || { echo "--arm needs a value (cd3|cd6|ao1)" >&2; exit 2; }
|
||||
ARMS+=("$2"); shift 2 ;;
|
||||
--dry-run) DRY=1; shift ;;
|
||||
--plan-only) PLAN_ONLY=1; shift ;;
|
||||
--target-count) TARGET_COUNT="$2"; shift 2 ;;
|
||||
--target-count) [ "$#" -ge 2 ] || { echo "--target-count needs a value" >&2; exit 2; }
|
||||
TARGET_COUNT="$2"; shift 2 ;;
|
||||
--keep-recipe) KEEP_RECIPE=1; shift ;;
|
||||
*) echo "unknown argument: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -124,6 +126,13 @@ fi
|
||||
for a in "${ARMS[@]}"; do
|
||||
[ -e "$OUTDIR/qwen38-w5nl64-$a.rad" ] && {
|
||||
echo "$OUTDIR/qwen38-w5nl64-$a.rad already exists; move it to trash first" >&2; exit 2; }
|
||||
# A killed run leaves its container behind (--rm only fires when podman itself
|
||||
# exits), and the next arm then dies on "name already in use" AFTER the plan
|
||||
# phase; refuse here, where the fix is one `podman rm -f`.
|
||||
podman container exists "qwen38-convert-$a" && {
|
||||
echo "container qwen38-convert-$a still exists (a killed run?);" \
|
||||
"'podman rm -f qwen38-convert-$a' and re-run" >&2
|
||||
exit 2; }
|
||||
done
|
||||
|
||||
# --- the protected plane, spliced into a per-run copy of the template --------------------------
|
||||
@@ -203,6 +212,7 @@ for arm in "${ARMS[@]}"; do
|
||||
rc=0; run_convert "$arm" || rc=$?
|
||||
[ "$rc" -eq 0 ] || { echo "arm $arm: rad-convert rc=$rc; log: $STATE/convert-$arm-$TS.log" >&2; overall=1; continue; }
|
||||
out="$OUTDIR/qwen38-w5nl64-$arm.rad"
|
||||
[ -f "$out" ] || { echo "arm $arm: rad-convert exited 0 but wrote no $out; log: $STATE/convert-$arm-$TS.log" >&2; overall=1; continue; }
|
||||
got_mib=$(( $(stat -c %s "$out") / 1048576 ))
|
||||
# The receipt: the artifact is what the plan predicted, within the metadata the plan does not
|
||||
# count. An arm that came out materially smaller lost a plane somewhere.
|
||||
|
||||
@@ -1362,11 +1362,17 @@ deploy_authentik() {
|
||||
# generated". Boot persistence comes from [Install] WantedBy=default.target
|
||||
# inside each .container file, which the generator honors at user-manager
|
||||
# start (linger must stay on for the deploy user).
|
||||
systemctl --user start authentik-postgres.service
|
||||
# RESTART, not start: `start` on an already-active unit is a no-op, so an
|
||||
# image bump would install the new digest and keep serving the old
|
||||
# container while every gate stayed green (measured 2026-10-30: server and
|
||||
# worker stayed on the previous release beside an outpost that moved,
|
||||
# because only the outpost handler restarts). The generator has already
|
||||
# re-rendered the unit from the .container, so a restart is what applies it.
|
||||
systemctl --user restart authentik-postgres.service
|
||||
wait_container_healthy authentik-postgres 120 || return 1
|
||||
systemctl --user start authentik-server.service
|
||||
systemctl --user restart authentik-server.service
|
||||
wait_container_healthy authentik-server 300 || return 1
|
||||
systemctl --user start authentik-worker.service
|
||||
systemctl --user restart authentik-worker.service
|
||||
}
|
||||
|
||||
deploy_authentik_outpost() {
|
||||
@@ -1482,7 +1488,7 @@ deploy_anubis() {
|
||||
# artifact would leave a restart-looping unit and a dark endpoint. The
|
||||
# port gate is the same as every cutover since 2026-09-16: nothing
|
||||
# starts until 7770 is released.
|
||||
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
|
||||
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0}"
|
||||
# The payload dir is its own twin because the NPU lane's containment gate below
|
||||
# reads it; every artifact path under it stays a full literal as usual.
|
||||
LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}"
|
||||
|
||||
@@ -8,8 +8,10 @@
|
||||
# (/opt/halogen/npu/models.txt inside the image) and refuses to load a model
|
||||
# whose files do not match.
|
||||
#
|
||||
# The manifest below IS that catalog block (dumped from the pinned 0.16.2
|
||||
# image on 2026-10-19), so the files land in the layout the entrypoint
|
||||
# The manifest below IS that catalog block (dumped from the pinned 0.18.20
|
||||
# image on 2026-10-30; the 0.16.2 block this replaced named different device
|
||||
# programs — 10 `u*.elf` and a 5,086,496 B consts.hnpb — so a version bump
|
||||
# re-fetches the set or the entrypoint refuses the boot), so the files land in the layout the entrypoint
|
||||
# expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and
|
||||
# <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep
|
||||
# /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD
|
||||
@@ -57,21 +59,25 @@ manifest() {
|
||||
case "$1" in
|
||||
qwen3-embedding-0.6b)
|
||||
NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b"
|
||||
NPU_REV="4960fd9d4423dd15373ad21c83db48dbdb3b2798"
|
||||
NPU_REV="8fd44c9f18861618a509bea75c51ff8a37e3bd5d"
|
||||
FILES=(
|
||||
"devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91"
|
||||
"devices/consts.hnpb 5086496 ae430e41545a0b1012938cda373b0c450364d3be631111cca3a8c43679f93336"
|
||||
"devices/devices.hnpm 24047 4e9bca9d95d084fb89ce5b99f579a00666c8ca1efae2613c53e3a4878cc9f174"
|
||||
"devices/u0.elf 4931968 b38f165886cb037b7d3951d9cb878473eaba58a432acd45416ff85fd88aa4e46"
|
||||
"devices/u1.elf 5456000 b1811feee851b8af833b37721da92099d89772110815bbbe2c3fa1c276f0ff42"
|
||||
"devices/u2.elf 6505552 406ea9a95b470628a7382e376f4a72cdf4d7aa4c59b4ddf0a4bdd139d63d6ff2"
|
||||
"devices/u3.elf 6817216 bc205d51b523d770151a0c61968f7c70bf624412827580bb6fe0aaf4650dc2ee"
|
||||
"devices/u4.elf 7444032 a5a91338a2603acdcd5c11df532033fda2d3f5479ea44ae87f98fafbdff9a62d"
|
||||
"devices/u5.elf 7432416 6c2bf41835569e0eba25ded7c05c7264ae69c6700bdeb4791057d884f2a4c126"
|
||||
"devices/u6.elf 4941232 919d0038ca088eafddf195f3466ff38db27d3635be1cffa5d4d4b5140902a68e"
|
||||
"devices/u7.elf 5488304 d904cf8afae9428dd167960549e18cce5441745f18c62f8c50975dffc077d3c2"
|
||||
"devices/u8.elf 4941232 db3aef69b17825854ee0dd2df475125a3f5c872f95c686793d6af1ee83596fa7"
|
||||
"devices/u9.elf 5488304 d182727e551084db8fb9600441d8aa373c7f09a3d267ae0a5e921ef92326db1c"
|
||||
"devices/consts.hnpb 8495512 5af7c1f2f24d45a9054872bba882461cf7f6ffda832e785b700643d6467ad88b"
|
||||
"devices/devices.hnpm 35465 85d870f6b900de1829659955f0dc701d57984580f5e5ec251add5d6969f03e3d"
|
||||
"devices/u0.elf 2975008 f74df12eb8c79e4e90cbedcc1dd8f142219772d400e3a142fde8a95378b105eb"
|
||||
"devices/u1.elf 3266848 ac5a3f89f934951b315b62ab53e1d0a74dbe6396134192641c405d08da332754"
|
||||
"devices/u2.elf 4015328 5e4f0d3342f262816289866d7e8c966f56a9f938c2e12e7341718cc7312aa480"
|
||||
"devices/u3.elf 4326976 94cf5d269fa286d91c26bf4654cedf101c047d23c4495b74c9c129cdc33b210f"
|
||||
"devices/u4.elf 4953792 cbd2796050d4c675f5cb3da117111087d510173e26f262f1b36aab66dd49dc91"
|
||||
"devices/u5.elf 4940112 7bf6a4470546bb888da02af395c07d92a24fb6d43aef27eb2151b3a9eda19e4d"
|
||||
"devices/u6.elf 6174544 10b9d50b635993e1f9c23775c0b94901d922c1e433cd10910e9f97174235f5ac"
|
||||
"devices/u7.elf 2983776 e0b06063b7379d05c85b3f435b819d96fda1223d8025d6d15958854b5711ace9"
|
||||
"devices/u8.elf 3298656 3bad204f782a68b97e27c1172530b13d7fbb0b495fa95f9504530a1048560644"
|
||||
"devices/u9.elf 2983776 875e313280358ce116e2a51a29a2c42794b5a9fadb3d48db702c90d18d838c2c"
|
||||
"devices/u10.elf 3298656 ef2e0f399f149a5476955bcf233c6fa2e896fcef1f86ecb0d8113c952c41a8b2"
|
||||
"devices/u11.elf 3298656 68a2519eee2d6a61ccc4ca2007e3186d44c9eb4e81162d742fe2f1756ec5a528"
|
||||
"devices/u12.elf 3298656 2bdc6abfa58cac1d5795a3ef52fe8a9ab4e7c2a4792822d615499a913401f728"
|
||||
"devices/u13.elf 3298656 4a3ce14f7ba15987b9bcbb4a010083e8aef68cd87c20cbe43403a98263179fe2"
|
||||
"qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1"
|
||||
"tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a"
|
||||
) ;;
|
||||
@@ -99,7 +105,18 @@ for MODEL in ${MODELS//,/ }; do
|
||||
# 62 GiB fetch hit this on 2026-10-18).
|
||||
url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \
|
||||
|| { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; }
|
||||
curl -fL --retry 5 -C - -o "$path" "$url"
|
||||
# Resume ONLY into a genuinely partial file. A staged file of a
|
||||
# DIFFERENT size is an artifact of another revision (a version
|
||||
# bump re-cuts the device programs), and `-C -` against it
|
||||
# appends the tail to the old bytes: the result is the right
|
||||
# BYTE COUNT with the wrong content, which reads as a corrupt
|
||||
# program, not as a fetch fault (measured 2026-10-30 on
|
||||
# consts.hnpb). Fetch those from zero.
|
||||
resume=()
|
||||
if [ -f "$path" ] && [ "$(stat -c %s "$path")" -lt "$want" ]; then
|
||||
resume=(-C -)
|
||||
fi
|
||||
curl -fL --retry 5 "${resume[@]+"${resume[@]}"}" -o "$path" "$url"
|
||||
fi
|
||||
fi
|
||||
[ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
|
||||
|
||||
@@ -10,8 +10,11 @@
|
||||
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
|
||||
# engine moves with the payload, never ahead of it.
|
||||
# Image digest-pinned (a tag pull silently re-points; the digest is what was
|
||||
# qualified — 0.16.2 is the version the model card names as its quickstart pin).
|
||||
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||
# qualified). 0.18.20 since 2026-10-30 (was 0.16.2, the model card's quickstart
|
||||
# pin) — the vendor has moved two minor versions and the payload lock is what
|
||||
# the deploy's health gate plus the boot ledger decide; rollback is the
|
||||
# previous digest, 0c61bf84…, on this same line.
|
||||
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0
|
||||
# Container identity kept from the 27B era: the unit's ExecStop names it.
|
||||
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
||||
# Predecessor identity the serve script clears if it is still holding the GPUs.
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
|
||||
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
|
||||
# frank-zealot — llama.cpp (PrismML fork build 12118 = 0.7.0-dev, commit
|
||||
# 024474448, CUDA) on the RTX 3090, :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
|
||||
# the refusal-direction adapter applied at load.
|
||||
#
|
||||
# ONE OWNER: shape, payload identity, eras, laws and every measured number
|
||||
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
|
||||
# knobs and only the reason a reader editing THIS line cannot recover
|
||||
# elsewhere. Where the two disagree, this file wins on values.
|
||||
LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
|
||||
LLAMA_BIN=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server
|
||||
# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
|
||||
# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
|
||||
# metadata dequantizes wrong, silently.
|
||||
@@ -64,7 +64,8 @@ CACHE_TYPE_V=q8_0
|
||||
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
|
||||
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
|
||||
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
|
||||
# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
|
||||
# beside MTP needs fork build >= b10776; LLAMA_BIN is 12118 (b10777 is kept on
|
||||
# disk as the rollback path).
|
||||
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
|
||||
# every --spec-draft-*); half a set has segfaulted at load.
|
||||
SPEC_TYPE=draft-mtp
|
||||
|
||||
+175
-112
@@ -34,14 +34,25 @@ import json
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
import sys
|
||||
|
||||
from jinja2 import Environment, FileSystemLoader
|
||||
from tokenizers import Tokenizer
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
|
||||
if _HERE not in sys.path:
|
||||
sys.path.insert(0, _HERE)
|
||||
|
||||
import qwen38_common as qc # noqa: E402 - sibling module, same directory
|
||||
|
||||
CODE_EXT = (".sh", ".py", ".jinja", ".recipe", ".env", ".example",
|
||||
".yaml", ".yml", ".bats", ".service", ".container", ".toml")
|
||||
JSONL_EXTS = (".jsonl", ".json")
|
||||
DOC_DIR = "serverAppConfigs-docs"
|
||||
FRAGMENT_TOKENS = 2400 # what one fragment may contribute
|
||||
CLASS_MIX = {"code": .45, "prose": .30, "tools": .25}
|
||||
HF_CAP = {"code": 40000, "prose": 60000, "tools": 0}
|
||||
POOL_CYCLES = 40 # how many times a class pool may repeat before giving up
|
||||
KLD_GUARD = 5000 # rendered conversations tried before declaring the pool short
|
||||
|
||||
FLEET_CMDS = [
|
||||
"systemctl status frank-archon --no-pager",
|
||||
@@ -142,7 +153,7 @@ def load_parquet_texts(path: str, kind: str, rng: random.Random,
|
||||
return out[:cap]
|
||||
|
||||
|
||||
def main() -> None:
|
||||
def parse_args() -> argparse.Namespace:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--repo", required=True)
|
||||
ap.add_argument("--tokenizer-json", required=True)
|
||||
@@ -156,22 +167,26 @@ def main() -> None:
|
||||
ap.add_argument("--hf-code-parquet", default="")
|
||||
ap.add_argument("--hf-prose-parquet", default="")
|
||||
ap.add_argument("--seed", type=int, default=20261006)
|
||||
a = ap.parse_args()
|
||||
return ap.parse_args()
|
||||
|
||||
rng = random.Random(a.seed)
|
||||
tokenizer = Tokenizer.from_file(a.tokenizer_json)
|
||||
env = Environment(loader=FileSystemLoader(os.path.dirname(a.template)),
|
||||
|
||||
def load_template(template: str) -> tuple[Environment, str]:
|
||||
"""The chat template as the engine will render it, plus its short name."""
|
||||
env = Environment(loader=FileSystemLoader(os.path.dirname(template)),
|
||||
trim_blocks=True, lstrip_blocks=True)
|
||||
env.filters["tojson"] = lambda v: json.dumps(v, ensure_ascii=False)
|
||||
env.filters["raise_exception"] = lambda m: (_ for _ in ()).throw(
|
||||
ValueError(m))
|
||||
tname = os.path.basename(a.template)
|
||||
return env, os.path.basename(template)
|
||||
|
||||
|
||||
def classify(repo: str) -> dict[str, list[str]]:
|
||||
"""Walk the checkout into the three source classes the mix draws from."""
|
||||
code, prose, jsonl = [], [], []
|
||||
for root, dirs, files in os.walk(a.repo):
|
||||
for root, dirs, files in os.walk(repo):
|
||||
dirs[:] = [d for d in dirs if not d.startswith(".")]
|
||||
for f in sorted(files):
|
||||
p = os.path.relpath(os.path.join(root, f), a.repo)
|
||||
p = os.path.relpath(os.path.join(root, f), repo)
|
||||
if DOC_DIR in p and f.endswith(".md"):
|
||||
prose.append(p)
|
||||
elif p.startswith(DOC_DIR):
|
||||
@@ -180,137 +195,185 @@ def main() -> None:
|
||||
code.append(p)
|
||||
elif f.endswith(JSONL_EXTS) and f != "package-lock.json":
|
||||
jsonl.append(p)
|
||||
# file-level split: bucket 0 -> KLD pool, 1..4 -> calibration pool
|
||||
pools: dict[str, tuple[list, list]] = {}
|
||||
for cls, lst in (("code", code), ("prose", prose), ("tools", jsonl)):
|
||||
pools[cls] = ([p for p in lst if fnv(p) % 5 == 0],
|
||||
[p for p in lst if fnv(p) % 5 != 0])
|
||||
return {"code": code, "prose": prose, "tools": jsonl}
|
||||
|
||||
def text_of(p: str) -> str:
|
||||
path = os.path.join(a.repo, p)
|
||||
raw = open(path, encoding="utf-8", errors="replace").read()
|
||||
if p.endswith(JSONL_EXTS):
|
||||
lines = [json.loads(l)["content"] for l in raw.splitlines()
|
||||
if l.strip() and "content" in l]
|
||||
raw = "\n".join(lines)
|
||||
toks = tokenizer.encode(raw, add_special_tokens=False).ids
|
||||
return tokenizer.decode(toks[:2400], skip_special_tokens=True)
|
||||
|
||||
def split_pools(classes: dict[str, list[str]]) -> dict[str, tuple[list, list]]:
|
||||
"""File-level split: bucket 0 -> KLD pool, everything else -> calibration."""
|
||||
return {cls: ([p for p in lst if fnv(p) % 5 == 0],
|
||||
[p for p in lst if fnv(p) % 5 != 0])
|
||||
for cls, lst in classes.items()}
|
||||
|
||||
|
||||
def jsonl_content(raw: str) -> str:
|
||||
"""The content fields of a content-keyed JSONL, or "" for anything else.
|
||||
|
||||
A .json in this repo is not necessarily a content-keyed JSONL: the substring
|
||||
test passes on any line that merely mentions "content", and a list-shaped
|
||||
document raises TypeError — both used to crash the whole build. Skip what
|
||||
does not fit instead.
|
||||
"""
|
||||
lines = []
|
||||
for line in raw.splitlines():
|
||||
if not line.strip() or "content" not in line:
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if isinstance(rec, dict) and isinstance(rec.get("content"), str):
|
||||
lines.append(rec["content"])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def fragment_text(repo: str, rel: str, tok) -> str:
|
||||
"""One file as a fragment: its first FRAGMENT_TOKENS tokens, decoded."""
|
||||
raw = open(os.path.join(repo, rel), encoding="utf-8",
|
||||
errors="replace").read()
|
||||
if rel.endswith(JSONL_EXTS):
|
||||
raw = jsonl_content(raw)
|
||||
ids = tok.encode(raw, add_special_tokens=False).ids
|
||||
return tok.decode(ids[:FRAGMENT_TOKENS], skip_special_tokens=True)
|
||||
|
||||
|
||||
def build_frags(a: argparse.Namespace, tok,
|
||||
rng: random.Random) -> dict[str, dict[str, list[str]]]:
|
||||
"""Both fragment pools per class, seeded so the same seed rebuilds the corpus."""
|
||||
pools = split_pools(classify(a.repo))
|
||||
cmds = harvest_cmds(a.repo, rng)
|
||||
# fragment pools per class, corpus-split first
|
||||
frags: dict[str, dict[str, list[str]]] = {}
|
||||
for cls in pools:
|
||||
kld_files, cal_files = pools[cls]
|
||||
fr = {}
|
||||
src = {"kld": kld_files, "cal": cal_files}
|
||||
for cls, (kld_files, cal_files) in pools.items():
|
||||
if cls == "tools":
|
||||
fr["kld"] = [tool_call(rng, cmds) for _ in range(600)]
|
||||
fr["cal"] = [tool_call(rng, cmds) for _ in range(40000)]
|
||||
frags[cls] = {"kld": [tool_call(rng, cmds) for _ in range(600)],
|
||||
"cal": [tool_call(rng, cmds) for _ in range(40000)]}
|
||||
else:
|
||||
fr["kld"] = [text_of(p) for p in src["kld"]]
|
||||
fr["cal"] = [text_of(p) for p in src["cal"]]
|
||||
frags[cls] = fr
|
||||
frags[cls] = {"kld": [fragment_text(a.repo, p, tok) for p in kld_files],
|
||||
"cal": [fragment_text(a.repo, p, tok) for p in cal_files]}
|
||||
# per-class fragment caps: prose rows are short (~66 tok) so need many;
|
||||
# codeparrot files are long (~2.6k tok) so a small cap suffices
|
||||
hfcap = {"code": 40000, "prose": 60000, "tools": 0}
|
||||
for cls, spec in (("code", a.hf_code_parquet),
|
||||
("prose", a.hf_prose_parquet)):
|
||||
for path in [p for p in spec.split(",") if p]:
|
||||
frags[cls]["cal"].extend(
|
||||
load_parquet_texts(path, cls, rng, cap=hfcap[cls]))
|
||||
load_parquet_texts(path, cls, rng, cap=HF_CAP[cls]))
|
||||
for cls in frags:
|
||||
rng.shuffle(frags[cls]["cal"])
|
||||
return frags
|
||||
|
||||
def toks(s: str) -> int:
|
||||
return len(tokenizer.encode(s, add_special_tokens=False).ids)
|
||||
|
||||
def emit_calib() -> tuple[int, list[dict]]:
|
||||
targets = {"code": .45, "prose": .30, "tools": .25}
|
||||
rows: list[dict] = []
|
||||
for cls, frac in targets.items():
|
||||
want = int(a.calib_tokens * frac)
|
||||
pool = frags[cls]["cal"]
|
||||
def pack_class(cls: str, pool: list[str], want: int, tok,
|
||||
chunk_tokens: int) -> list[dict]:
|
||||
"""Greedily pack `pool` into chunk_tokens-sized calibration prompts."""
|
||||
if not pool:
|
||||
return []
|
||||
rows: list[dict] = []
|
||||
# Count every fragment once: the loop cycles the pool up to POOL_CYCLES
|
||||
# times, and re-tokenizing a 2.4k-token fragment on each pass dominated the
|
||||
# build's wall time.
|
||||
counts = [qc.n_tokens(tok, f) for f in pool]
|
||||
got = 0
|
||||
i = 0
|
||||
buf: list[str] = []
|
||||
nbuf = 0
|
||||
while got < want:
|
||||
j = i % len(pool)
|
||||
frag = pool[j]
|
||||
i += 1
|
||||
n = counts[j]
|
||||
if nbuf + n > chunk_tokens and buf:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
buf, nbuf = [], 0
|
||||
if got >= want:
|
||||
break
|
||||
buf.append(frag)
|
||||
nbuf += n
|
||||
if i > len(pool) * POOL_CYCLES:
|
||||
break # pool exhausted with repeats; stop
|
||||
if buf and got < want:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
print(f"calib {cls}: {got} tokens, {i} fragments")
|
||||
return rows
|
||||
|
||||
|
||||
def emit_calib(a: argparse.Namespace, frags, tok,
|
||||
rng: random.Random) -> tuple[int, list[dict]]:
|
||||
"""calib.jsonl rows: the class mix, shuffled, prompts only."""
|
||||
rows: list[dict] = []
|
||||
for cls, frac in CLASS_MIX.items():
|
||||
rows.extend(pack_class(cls, frags[cls]["cal"], int(a.calib_tokens * frac),
|
||||
tok, a.chunk_tokens))
|
||||
rng.shuffle(rows)
|
||||
return len(rows), [{"prompt": r["prompt"]} for r in rows]
|
||||
|
||||
|
||||
def emit_kld(a: argparse.Namespace, frags, tok, rng: random.Random,
|
||||
env: Environment, tname: str) -> tuple[int, list[dict], int]:
|
||||
"""kld.jsonl rows: multi-turn conversations, each with a scored tail."""
|
||||
template = env.get_template(tname)
|
||||
rows: list[dict] = []
|
||||
total = scored_total = 0
|
||||
classes = ["code", "prose", "tools"]
|
||||
cursor = {c: 0 for c in classes}
|
||||
guard = 0
|
||||
while total < a.kld_tokens and guard < KLD_GUARD:
|
||||
guard += 1
|
||||
msgs, srcs = [], []
|
||||
for t in range(rng.choice([2, 3, 4])):
|
||||
cls = classes[(len(rows) + t) % 3]
|
||||
pool = frags[cls]["kld"]
|
||||
if not pool:
|
||||
continue
|
||||
got, i, buf, nbuf = 0, 0, [], 0
|
||||
while got < want:
|
||||
frag = pool[i % len(pool)]
|
||||
i += 1
|
||||
n = toks(frag)
|
||||
if nbuf + n > a.chunk_tokens and buf:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
buf, nbuf = [], 0
|
||||
if got >= want:
|
||||
break
|
||||
buf.append(frag)
|
||||
nbuf += n
|
||||
if i > len(pool) * 40:
|
||||
break # pool exhausted with repeats; stop
|
||||
if buf and got < want:
|
||||
rows.append({"prompt": "\n".join(buf), "cls": cls})
|
||||
got += nbuf
|
||||
print(f"calib {cls}: {got} tokens, {i} fragments")
|
||||
rng.shuffle(rows)
|
||||
return len(rows), [{k: v for k, v in r.items() if k == "prompt"}
|
||||
for r in rows]
|
||||
frag = pool[cursor[cls] % len(pool)]
|
||||
cursor[cls] += 1
|
||||
msgs.append({"role": "user",
|
||||
"content": "Summarize and explain this "
|
||||
f"engineering material:\n{frag}"})
|
||||
msgs.append({"role": "assistant",
|
||||
"content": frag[:4000]})
|
||||
srcs.append(cls)
|
||||
rendered = template.render(messages=msgs, add_generation_prompt=False)
|
||||
n = qc.n_tokens(tok, rendered)
|
||||
if n < a.kld_min_prompt:
|
||||
continue
|
||||
rows.append({"prompt": rendered,
|
||||
"score_from": n - a.score_tail,
|
||||
"source": "+".join(sorted(set(srcs)))})
|
||||
total += n
|
||||
scored_total += a.score_tail
|
||||
return len(rows), rows, scored_total
|
||||
|
||||
def emit_kld() -> tuple[int, list[dict]]:
|
||||
rows, total = [], 0
|
||||
classes = ["code", "prose", "tools"]
|
||||
i = {c: 0 for c in classes}
|
||||
guard = 0
|
||||
while total < a.kld_tokens and guard < 5000:
|
||||
guard += 1
|
||||
msgs, srcs = [], []
|
||||
for t in range(rng.choice([2, 3, 4])):
|
||||
cls = classes[(len(rows) + t) % 3]
|
||||
pool = frags[cls]["kld"]
|
||||
if not pool:
|
||||
continue
|
||||
frag = pool[i[cls] % len(pool)]
|
||||
i[cls] += 2
|
||||
msgs.append({"role": "user",
|
||||
"content": "Summarize and explain this "
|
||||
f"engineering material:\n{frag}"})
|
||||
msgs.append({"role": "assistant",
|
||||
"content": frag[:4000]})
|
||||
srcs.append(cls)
|
||||
rendered = env.get_template(tname).render(
|
||||
messages=msgs, add_generation_prompt=False)
|
||||
n = toks(rendered)
|
||||
if n < a.kld_min_prompt:
|
||||
continue
|
||||
rows.append({"prompt": rendered,
|
||||
"score_from": n - a.score_tail,
|
||||
"source": "+".join(sorted(set(srcs)))})
|
||||
total += n
|
||||
return len(rows), rows
|
||||
|
||||
os.makedirs(a.out_dir, exist_ok=True)
|
||||
nc, rows = emit_calib()
|
||||
with open(os.path.join(a.out_dir, "calib.jsonl"), "w") as f:
|
||||
for r in rows:
|
||||
f.write(json.dumps(r) + "\n")
|
||||
nk, krows = emit_kld()
|
||||
with open(os.path.join(a.out_dir, "kld.jsonl"), "w") as f:
|
||||
for r in krows:
|
||||
f.write(json.dumps(r) + "\n")
|
||||
scored = sum(len(tokenizer.encode(r["prompt"], add_special_tokens=False).ids)
|
||||
- r["score_from"] for r in krows)
|
||||
manifest = {
|
||||
def manifest(a: argparse.Namespace, tname: str, nc: int, nk: int,
|
||||
scored: int) -> dict:
|
||||
return {
|
||||
"seed": a.seed, "template": tname,
|
||||
"tokenizer": os.path.basename(a.tokenizer_json),
|
||||
"calib_prompts": nc, "kld_prompts": nk,
|
||||
"kld_scored_positions": scored,
|
||||
"kld_score_tail": a.score_tail,
|
||||
"disjoint_file_pools": "local KLD material only; HF code feeds calib",
|
||||
"calib_mix": {"code": .45, "prose": .30, "tools": .25},
|
||||
"calib_mix": dict(CLASS_MIX),
|
||||
"hf_code": a.hf_code_parquet, "hf_prose": a.hf_prose_parquet,
|
||||
}
|
||||
with open(os.path.join(a.out_dir, "manifest.json"), "w") as f:
|
||||
json.dump(manifest, f, indent=1)
|
||||
print(json.dumps(manifest, indent=1))
|
||||
|
||||
|
||||
def main() -> None:
|
||||
a = parse_args()
|
||||
rng = random.Random(a.seed)
|
||||
tok = qc.load_tokenizer(a.tokenizer_json)
|
||||
env, tname = load_template(a.template)
|
||||
|
||||
frags = build_frags(a, tok, rng)
|
||||
os.makedirs(a.out_dir, exist_ok=True)
|
||||
nc, calib_rows = emit_calib(a, frags, tok, rng)
|
||||
qc.write_jsonl(os.path.join(a.out_dir, "calib.jsonl"), calib_rows)
|
||||
nk, kld_rows, scored = emit_kld(a, frags, tok, rng, env, tname)
|
||||
qc.write_jsonl(os.path.join(a.out_dir, "kld.jsonl"), kld_rows)
|
||||
man = manifest(a, tname, nc, nk, scored)
|
||||
qc.write_json(os.path.join(a.out_dir, "manifest.json"), man)
|
||||
print(json.dumps(man, indent=1))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Executable
+94
@@ -0,0 +1,94 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared plumbing for the qwen38 campaign corpus tools.
|
||||
|
||||
The three tools move the same two shapes — a JSONL of prompt records and a
|
||||
one-artifact JSON manifest — and they all tokenize with the SAME tokenizer. That
|
||||
last part is not a detail: a KLD reference is only meaningful against the exact
|
||||
token sequence the corpus recorded, so `n_tokens` exists once, here, and every
|
||||
tool counts the same way. A window that decode/encode does not round-trip is a
|
||||
refusal, never a warning.
|
||||
|
||||
Loaded as a sibling module. Each tool puts its own directory on `sys.path`
|
||||
first, so `python3 scripts/<tool>.py` and the driver's container mount
|
||||
(`runpy.run_path('/work/<tool>.py')`, `-v scripts:/work:ro`) both resolve it —
|
||||
run the tools by path, not piped through stdin.
|
||||
"""
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
|
||||
def refuse(msg: str) -> None:
|
||||
"""Exit 2 with a REFUSED line — the code argparse uses for unusable input."""
|
||||
print(f"REFUSED: {msg}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
def load_tokenizer(path: str):
|
||||
try:
|
||||
from tokenizers import Tokenizer
|
||||
except ImportError as exc: # the corpus tools carry their own venv
|
||||
refuse(f"tokenizers is not importable by {sys.executable} — run the "
|
||||
f"corpus venv's python (on dankserver: "
|
||||
f"/home/dank/venvs/radcorpus/bin/python) or set "
|
||||
f"FRANK_QWEN38_TOKENIZER: {exc}")
|
||||
if not os.path.isfile(path):
|
||||
refuse(f"no tokenizer at {path}")
|
||||
return Tokenizer.from_file(path)
|
||||
|
||||
|
||||
def n_tokens(tok, text: str) -> int:
|
||||
"""The corpus's own count: no special tokens added, prompt side only."""
|
||||
return len(tok.encode(text, add_special_tokens=False).ids)
|
||||
|
||||
|
||||
def iter_jsonl(path: str):
|
||||
"""Yield `(lineno, record)` for every non-blank line; refuse on a bad line.
|
||||
|
||||
Field validation stays with the caller: the three corpora carry different
|
||||
fields (`prompt` alone, `prompt`+`cls`, `prompt`+`score_from`+`source`), and
|
||||
a record missing the one the tool needs is that tool's refusal to make.
|
||||
"""
|
||||
with open(path, encoding="utf-8", errors="replace") as fh:
|
||||
for lineno, line in enumerate(fh, 1):
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except json.JSONDecodeError as exc:
|
||||
refuse(f"{path}:{lineno} is not JSON: {exc}")
|
||||
yield lineno, rec
|
||||
|
||||
|
||||
def write_jsonl(path: str, rows) -> None:
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
for row in rows:
|
||||
fh.write(json.dumps(row, ensure_ascii=False) + "\n")
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def atomic_writer(path: str):
|
||||
"""Write `path` through a temporary sibling and rename on success.
|
||||
|
||||
An interrupted run then leaves no half corpus behind, and the temporary name
|
||||
keeps the one-artifact-one-name law the driver applies to a score report.
|
||||
"""
|
||||
d = os.path.dirname(os.path.abspath(path)) or "."
|
||||
fd, tmp = tempfile.mkstemp(dir=d, prefix="." + os.path.basename(path) + "-")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
yield fh
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
raise
|
||||
os.chmod(path, 0o644)
|
||||
|
||||
|
||||
def write_json(path: str, obj) -> None:
|
||||
with atomic_writer(path) as fh:
|
||||
json.dump(obj, fh, indent=1)
|
||||
fh.write("\n")
|
||||
@@ -38,12 +38,19 @@ one-artifact-one-name law run-qwen38-bf16.sh applies to a score report. The file
|
||||
is written through a temporary name and renamed, so an interrupted run leaves no
|
||||
half corpus behind. A JSON summary of what was written goes to stdout.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
|
||||
if _HERE not in sys.path:
|
||||
sys.path.insert(0, _HERE)
|
||||
|
||||
import qwen38_common as qc
|
||||
|
||||
MISSING = object() # "the parent record carried no source field"
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
@@ -69,95 +76,96 @@ def parse_args() -> argparse.Namespace:
|
||||
return ap.parse_args()
|
||||
|
||||
|
||||
def load_tokenizer(path: str):
|
||||
try:
|
||||
from tokenizers import Tokenizer
|
||||
except ImportError as exc: # the corpus scripts carry their own venv
|
||||
raise SystemExit(
|
||||
f"REFUSED: tokenizers is not importable by {sys.executable} — run the "
|
||||
f"corpus venv's python (on dankserver: /home/dank/venvs/radcorpus/bin/python) "
|
||||
f"or set FRANK_QWEN38_TOKENIZER: {exc}") from None
|
||||
if not os.path.isfile(path):
|
||||
raise SystemExit(f"REFUSED: no tokenizer at {path}")
|
||||
return Tokenizer.from_file(path)
|
||||
@dataclasses.dataclass(frozen=True)
|
||||
class Window:
|
||||
"""One scoring document cut out of a parent record."""
|
||||
prompt: str
|
||||
score_from: int
|
||||
n_tokens: int
|
||||
clamped: bool
|
||||
source: object = MISSING
|
||||
|
||||
def record(self) -> dict:
|
||||
rec = {"prompt": self.prompt, "score_from": self.score_from}
|
||||
if self.source is not MISSING:
|
||||
rec["source"] = self.source
|
||||
return rec
|
||||
|
||||
|
||||
def refuse(msg: str) -> None:
|
||||
print(f"REFUSED: {msg}", file=sys.stderr)
|
||||
sys.exit(2) # 2 is the code argparse uses for unusable arguments
|
||||
|
||||
|
||||
def main() -> None:
|
||||
a = parse_args()
|
||||
def validate(a: argparse.Namespace) -> None:
|
||||
"""Everything that makes this run unusable, named before a token is cut."""
|
||||
if a.max_doc_tokens <= 0:
|
||||
refuse(f"--max-doc-tokens must be positive (got {a.max_doc_tokens})")
|
||||
qc.refuse(f"--max-doc-tokens must be positive (got {a.max_doc_tokens})")
|
||||
if a.score_tail <= 0:
|
||||
refuse(f"--score-tail must be positive (got {a.score_tail})")
|
||||
qc.refuse(f"--score-tail must be positive (got {a.score_tail})")
|
||||
if a.max_doc_tokens <= a.score_tail:
|
||||
refuse(f"--max-doc-tokens {a.max_doc_tokens} cannot hold a {a.score_tail}-position "
|
||||
"score tail: every window would be all tail and no context")
|
||||
qc.refuse(f"--max-doc-tokens {a.max_doc_tokens} cannot hold a "
|
||||
f"{a.score_tail}-position score tail: every window would be all "
|
||||
"tail and no context")
|
||||
if not os.path.isfile(a.corpus):
|
||||
refuse(f"no corpus at {a.corpus}")
|
||||
qc.refuse(f"no corpus at {a.corpus}")
|
||||
if os.path.realpath(a.out) == os.path.realpath(a.corpus):
|
||||
refuse(f"--out {a.out} is the corpus itself; move it to trash first — the trash law "
|
||||
"means the old segmentation stays recoverable")
|
||||
qc.refuse(f"--out {a.out} is the corpus itself; move it to trash first — the "
|
||||
"trash law means the old segmentation stays recoverable")
|
||||
if os.path.exists(a.out) and os.path.getsize(a.out) > 0 and not a.force:
|
||||
refuse(f"{a.out} exists and is not empty; pass --force after moving it to trash")
|
||||
qc.refuse(f"{a.out} exists and is not empty; pass --force after moving it to trash")
|
||||
|
||||
tok = load_tokenizer(a.tokenizer_json)
|
||||
|
||||
def cut(rec: dict, tok, a: argparse.Namespace,
|
||||
where: str) -> tuple[list[Window], int]:
|
||||
"""Cut one record's token ids into windows, proving each one round-trips.
|
||||
|
||||
The round-trip encode is the whole reason this tool may claim the reference
|
||||
scores the corpus's own token sequence: decoding a window to text and
|
||||
encoding it back must return the identical ids, or the score is of a
|
||||
different document than the one on disk.
|
||||
"""
|
||||
ids = tok.encode(rec["prompt"], add_special_tokens=False).ids
|
||||
windows: list[Window] = []
|
||||
for start in range(0, max(len(ids), 1), a.max_doc_tokens):
|
||||
window = ids[start:start + a.max_doc_tokens]
|
||||
if not window:
|
||||
continue
|
||||
text = tok.decode(window)
|
||||
if tok.encode(text, add_special_tokens=False).ids != window:
|
||||
qc.refuse(f"{where} window at token {start} does not round-trip through "
|
||||
"text — the reference would score a different token sequence "
|
||||
"than the corpus holds")
|
||||
windows.append(Window(prompt=text,
|
||||
score_from=max(len(window) - a.score_tail, 0),
|
||||
n_tokens=len(window),
|
||||
clamped=len(window) < a.score_tail,
|
||||
source=rec["source"] if "source" in rec else MISSING))
|
||||
return windows, len(ids)
|
||||
|
||||
|
||||
def rechunk(a: argparse.Namespace, tok) -> dict:
|
||||
"""Write the re-chunked corpus and return the summary it printed."""
|
||||
docs_in = docs_out = tokens_in = tokens_out = 0
|
||||
longest_child = 0
|
||||
clamped = scored = 0
|
||||
longest_child = clamped = scored = 0
|
||||
sources: dict[str, int] = {}
|
||||
fd, tmp = tempfile.mkstemp(dir=os.path.dirname(os.path.abspath(a.out)) or ".",
|
||||
prefix=".rechunk-")
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as out:
|
||||
with open(a.corpus, encoding="utf-8") as src:
|
||||
for lineno, line in enumerate(src, 1):
|
||||
if not line.strip():
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except json.JSONDecodeError as exc:
|
||||
refuse(f"{a.corpus}:{lineno} is not JSON: {exc}")
|
||||
prompt = rec.get("prompt")
|
||||
if not isinstance(prompt, str):
|
||||
refuse(f"{a.corpus}:{lineno} has no string 'prompt' field")
|
||||
ids = tok.encode(prompt, add_special_tokens=False).ids
|
||||
docs_in += 1
|
||||
tokens_in += len(ids)
|
||||
for start in range(0, max(len(ids), 1), a.max_doc_tokens):
|
||||
window = ids[start:start + a.max_doc_tokens]
|
||||
if not window:
|
||||
continue
|
||||
score_from = max(len(window) - a.score_tail, 0)
|
||||
if len(window) < a.score_tail:
|
||||
clamped += 1
|
||||
scored += len(window) - score_from
|
||||
child = {"prompt": tok.decode(window),
|
||||
"score_from": score_from}
|
||||
if "source" in rec:
|
||||
child["source"] = rec["source"]
|
||||
out.write(json.dumps(child, ensure_ascii=False) + "\n")
|
||||
docs_out += 1
|
||||
tokens_out += len(window)
|
||||
longest_child = max(longest_child, len(window))
|
||||
if "source" in rec:
|
||||
for cls in str(rec["source"]).split("+"):
|
||||
sources[cls] = sources.get(cls, 0) + 1
|
||||
if tokens_out != tokens_in:
|
||||
raise SystemExit(f"REFUSED: the windows hold {tokens_out} tokens and the corpus "
|
||||
f"has {tokens_in} — re-chunking lost or invented tokens")
|
||||
os.replace(tmp, a.out)
|
||||
except BaseException:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
raise
|
||||
os.chmod(a.out, 0o644)
|
||||
|
||||
print(json.dumps({
|
||||
with qc.atomic_writer(a.out) as out:
|
||||
for lineno, rec in qc.iter_jsonl(a.corpus):
|
||||
where = f"{a.corpus}:{lineno}"
|
||||
if not isinstance(rec.get("prompt"), str):
|
||||
qc.refuse(f"{where} has no string 'prompt' field")
|
||||
windows, n_parent = cut(rec, tok, a, where)
|
||||
docs_in += 1
|
||||
tokens_in += n_parent
|
||||
for w in windows:
|
||||
out.write(json.dumps(w.record(), ensure_ascii=False) + "\n")
|
||||
docs_out += 1
|
||||
tokens_out += w.n_tokens
|
||||
longest_child = max(longest_child, w.n_tokens)
|
||||
clamped += 1 if w.clamped else 0
|
||||
scored += w.n_tokens - w.score_from
|
||||
if "source" in rec:
|
||||
for cls in str(rec["source"]).split("+"):
|
||||
sources[cls] = sources.get(cls, 0) + 1
|
||||
if tokens_out != tokens_in:
|
||||
raise SystemExit(f"REFUSED: the windows hold {tokens_out} tokens and the corpus "
|
||||
f"has {tokens_in} — re-chunking lost or invented tokens")
|
||||
return {
|
||||
"corpus": a.corpus,
|
||||
"out": a.out,
|
||||
"tokenizer": os.path.basename(a.tokenizer_json),
|
||||
@@ -171,7 +179,13 @@ def main() -> None:
|
||||
"scored_positions": scored,
|
||||
"clamped_score_from": clamped,
|
||||
"source_documents": sources,
|
||||
}, indent=1))
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
a = parse_args()
|
||||
validate(a)
|
||||
print(json.dumps(rechunk(a, qc.load_tokenizer(a.tokenizer_json)), indent=1))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -120,7 +120,8 @@ while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
kldref|calib|score) MODE="$1" ;;
|
||||
--dry-run) DRY=1 ;;
|
||||
--image) IMAGE="$2"; shift ;;
|
||||
--image) [ "$#" -ge 2 ] || { echo "--image needs a reference" >&2; exit 2; }
|
||||
IMAGE="$2"; shift ;;
|
||||
*)
|
||||
# score's two positional arguments, and nothing else may be bare.
|
||||
if [ "$MODE" = score ] && [ "${#POS[@]}" -lt 2 ]; then
|
||||
@@ -250,7 +251,8 @@ case "$MODE" in
|
||||
# time the engine reads a recording and writes a report instead of the
|
||||
# reverse. The reference dir mounts READ-ONLY — a scoring pass must not be
|
||||
# able to touch the zero point — and the report gets its own writable bind.
|
||||
if [ "$OUT_DIR" = "$KLD_REF_DIR" ] || same_dir "$OUT_DIR" "$KLD_REF_DIR"; then
|
||||
if [ "$OUT_DIR" = "$KLD_REF_DIR" ] || same_dir "$OUT_DIR" "$KLD_REF_DIR" \
|
||||
|| [ "$(realpath -m "$OUT_DIR")" = "$(realpath -m "$KLD_REF_DIR")" ]; then
|
||||
echo "REFUSED: score output $SCORE_OUT is inside the reference dir $KLD_REF_DIR, which this mode mounts read-only" >&2
|
||||
exit 2
|
||||
fi
|
||||
@@ -276,9 +278,17 @@ case "$MODE" in
|
||||
# through MAX_SEQS, exactly as kldref and score do.
|
||||
ARGS+=(--model "$MODEL_IN" --port 7770
|
||||
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
||||
# The tap is only real if this directory exists and the engine can write it:
|
||||
# without the check a typo'd CALIB_DIR auto-creates a root-owned empty dir in
|
||||
# whatever path was given, and the run burns hours serving prompts into a tap
|
||||
# nobody reads. RADIANCE_CALIB_TOKENS is the whole corpus' floor, not a round's.
|
||||
[ "$DRY" = 1 ] || [ -w "$CALIB_DIR" ] || {
|
||||
echo "REFUSED: calib tap dir $CALIB_DIR is not writable (mkdir it, chown it, then re-run)" >&2
|
||||
exit 2
|
||||
}
|
||||
MOUNTS+=(-v "$CALIB_DIR:/data/calib")
|
||||
TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib
|
||||
-e RADIANCE_CALIB_TOKENS=12000000)
|
||||
-e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-12000000}")
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -352,7 +362,14 @@ cp -f "$TEMPLATE" "$TPL" || {
|
||||
[ "$MODE" = kldref ] && mkdir -p "$KLD_REF_DIR"
|
||||
[ "$MODE" = calib ] && mkdir -p "$CALIB_DIR"
|
||||
|
||||
restore() { sudo systemctl start "$SERVICE" >/dev/null 2>&1 || true; }
|
||||
# Stop the campaign container BEFORE handing the cards back: a killed run leaves
|
||||
# its conmon alive (--rm only fires when podman itself exits), and production
|
||||
# then boots beside a container still holding the GPUs — measured 2026-10-29,
|
||||
# where the service reported `active` while its engine starved on the rig's VRAM.
|
||||
restore() {
|
||||
podman rm -f "$C" >/dev/null 2>&1 || true
|
||||
sudo systemctl start "$SERVICE" >/dev/null 2>&1 || true
|
||||
}
|
||||
trap restore EXIT
|
||||
echo "run-qwen38-bf16 $MODE: stopping $SERVICE ($(date -Is)); log: $LOG"
|
||||
sudo systemctl stop "$SERVICE" >/dev/null 2>&1 || true
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2
|
||||
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.18.20
|
||||
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
|
||||
# digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2
|
||||
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
# Serve prism-ml/Ternary-Bonsai-2-9B-gguf (PQ2_0 ternary quant WITH the MTP head
|
||||
# in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
|
||||
# vendor build 10777, CUDA sm_86).
|
||||
# vendor build 12118 (0.7.0-dev, commit 024474448), CUDA sm_86).
|
||||
#
|
||||
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
|
||||
# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
|
||||
@@ -78,7 +78,7 @@ if [[ -f "$ENV_FILE" ]]; then
|
||||
fi
|
||||
|
||||
# --- defaults (used when the env file does not set them) -------------------
|
||||
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-b10777/llama-server}"
|
||||
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server}"
|
||||
: "${MODEL_FILE:=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf}"
|
||||
: "${ALIAS:=frank-zealot}"
|
||||
: "${BIND_ADDR:=0.0.0.0}"
|
||||
@@ -119,7 +119,7 @@ fi
|
||||
# above are what pay for it. The draft cache dtype IS a further knob
|
||||
# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
|
||||
# build's --help) and stays at its default here. Card gate: quantized V alongside
|
||||
# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
|
||||
# MTP needs fork build >= prism-b10776; LLAMA_BIN is 12118 (b10777 kept as rollback)
|
||||
# prebuilt; the type-142 loader lock keeps it pinned regardless.
|
||||
# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
|
||||
: "${SPEC_TYPE:=draft-mtp}"
|
||||
|
||||
@@ -337,8 +337,12 @@ def main():
|
||||
if any(meta[shard_ids[i]][4][1] != hd for i in range(n_shards)):
|
||||
log("ERROR: PLE shards disagree on HD")
|
||||
return 2
|
||||
if hd % 128 != 0 and hd != 160:
|
||||
log(f"WARNING: PLE HD={hd} is neither 160 nor a multiple of 128")
|
||||
if hd != 160 and hd % 128 != 0:
|
||||
# The pack below writes lanes of 128 codes; any other HD would produce a
|
||||
# table that decodes to the wrong rows. The old warning let it continue.
|
||||
log(f"ERROR: PLE HD={hd} is neither 160 nor a multiple of 128, and the "
|
||||
"128-lane pack cannot express it")
|
||||
return 2
|
||||
|
||||
# ---- idempotence gate -------------------------------------------------
|
||||
stacked_names = [f"{p}.mlp.experts.gate_up_proj" for p in prefixes]
|
||||
@@ -521,12 +525,16 @@ def verify(stage, prefixes, n_e, shapes, meta, shard_ids, rows, hd,
|
||||
}
|
||||
for name, (dt, shp) in want.items():
|
||||
e = hdr[name]
|
||||
assert e["dtype"] == dt and list(e["shape"]) == shp, name
|
||||
assert e["data_offsets"][1] - e["data_offsets"][0] == \
|
||||
int(np.prod(shp)) * (1 if dt == F8 else 2), name
|
||||
if e["dtype"] != dt or list(e["shape"]) != shp:
|
||||
raise RuntimeError(f"{name}: header is {e['dtype']}{e['shape']}, expected {dt}{shp}")
|
||||
if e["data_offsets"][1] - e["data_offsets"][0] != int(np.prod(shp)) * (1 if dt == F8 else 2):
|
||||
raise RuntimeError(f"{name}: data span is {e['data_offsets']}, shape {shp} needs "
|
||||
f"{int(np.prod(shp)) * (1 if dt == F8 else 2)} B")
|
||||
_, phdr = parse_header(out_path)
|
||||
e = phdr[next(iter(phdr))]
|
||||
assert e["dtype"] == BF16 and list(e["shape"]) == [sum(rows), hd]
|
||||
if e["dtype"] != BF16 or list(e["shape"]) != [sum(rows), hd]:
|
||||
raise RuntimeError(f"PLE table header is {e['dtype']}{e['shape']}, "
|
||||
f"expected {BF16}[{sum(rows)}, {hd}]")
|
||||
|
||||
# sampled PLE rows: recompute and compare bytes exactly
|
||||
lut = e4m3_lut()
|
||||
@@ -546,7 +554,8 @@ def verify(stage, prefixes, n_e, shapes, meta, shard_ids, rows, hd,
|
||||
with open(out_path, "rb") as ofh:
|
||||
ofh.seek(out_off + got_row * hd * 2)
|
||||
got = ofh.read(hd * 2)
|
||||
assert got == want, f"PLE row {got_row} mismatch"
|
||||
if got != want:
|
||||
raise RuntimeError(f"PLE row {got_row} does not match the decoded source")
|
||||
mm.close()
|
||||
log("verify: new-file headers, sizes and sampled PLE rows all exact; "
|
||||
"expert slabs were byte-checked in-stack")
|
||||
|
||||
@@ -10,7 +10,7 @@ timestamp: 2026-09-03
|
||||
|
||||
## Server Overview
|
||||
|
||||
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
|
||||
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.18.20 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
|
||||
## Hardware Specifications
|
||||
|
||||
### CPU
|
||||
@@ -105,7 +105,7 @@ cable between claw and server is required for 10G.
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
|
||||
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
|
||||
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.18.20** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
|
||||
- The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
|
||||
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
|
||||
- 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
|
||||
|
||||
@@ -641,7 +641,10 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
|
||||
- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
|
||||
`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
|
||||
`scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
|
||||
`scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`
|
||||
`scripts/make-calib-corpus.py`, `scripts/rechunk-kld-corpus.py`,
|
||||
`scripts/qwen38_common.py` (the tokenizer, JSONL and refusal plumbing those three
|
||||
corpus tools share — one `n_tokens`, so every tool counts the same way),
|
||||
`scripts/derive-protected-experts.py`
|
||||
`scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
|
||||
- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
|
||||
it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
||||
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.18.20, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
|
||||
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
|
||||
tags:
|
||||
- llm
|
||||
@@ -10,10 +10,10 @@ tags:
|
||||
- gfx1151
|
||||
- halogen
|
||||
- endpoint
|
||||
timestamp: 2026-10-21
|
||||
timestamp: 2026-10-30
|
||||
---
|
||||
|
||||
# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
|
||||
# frank-dragoon — halogen-flash-server 0.18.20, native v2 abliterated (Dankclaw iGPU)
|
||||
|
||||
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
|
||||
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
|
||||
@@ -33,8 +33,8 @@ flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, ver
|
||||
| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
|
||||
| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
|
||||
|
||||
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a`
|
||||
(0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
|
||||
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0`
|
||||
(0.18.20, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
|
||||
unauthenticated token port 8730 stays on container loopback. Engine stdout →
|
||||
`/home/dank/logs/frank_dragoon.log`, **not** the journal.
|
||||
|
||||
@@ -74,6 +74,7 @@ rollback plan — recoverable only until the trash is purged.
|
||||
|
||||
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
||||
|---|---|---|---|---|---|
|
||||
| 2026-10-30 → | same v2 abliterated `.hgn` trio (payload untouched) | flash-server **0.18.20** `e35133b2…` | unchanged (2 slots, 262,144 ceiling, pool 401,408, MTP always-on) | vendor moved two minor versions past the 0.16.2 quickstart pin; the payload lock **held** — the same boot served chat, a `tool_calls` turn, vision ("Red", 132 prompt tokens) and `/v1/embeddings`, with the NPU lane's 19-file set verified against the new catalog | env line back to `0c61bf84…` + redeploy |
|
||||
| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
|
||||
| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
|
||||
| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
|
||||
@@ -190,9 +191,11 @@ created it — earlier topologies in .
|
||||
|
||||
## Contract (wire)
|
||||
|
||||
- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`
|
||||
`/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
|
||||
wall-clock plus `usage.prompt_tokens`.
|
||||
- Live: `/v1/models`, `/health`, `/metrics`, `/props` (model_alias, total_slots, chat_template),
|
||||
POST `/v1/chat/completions`, POST `/v1/embeddings` when the NPU lane is armed; `404`: `/ping`,
|
||||
`/completion`; `/tokenize` answers **405 to a GET** (the route exists and wants POST). **No
|
||||
`Server-Timing` header and no `/completion` timing object** — measure wall-clock plus
|
||||
`usage.prompt_tokens`.
|
||||
- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
|
||||
mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
|
||||
vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
|
||||
@@ -232,11 +235,15 @@ created it — earlier topologies in .
|
||||
XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at
|
||||
([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the
|
||||
same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server
|
||||
is exactly what runs today. The pinned 0.16.2 image already carries the five-program catalog
|
||||
(`/opt/halogen/npu/models.txt`: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`,
|
||||
`qwen3.5-2b`) and `docs/NPU.md` at tag `v0.16.2` documents it, so **`/v1/embeddings` is one env var and three mounts
|
||||
away — no runtime bump needed.** Wider catalog: `qwen3-embedding-4b` and `decider-4b` from 0.17.4, `embeddinggemma-2`
|
||||
(text ≤ 8,192 tokens, pictures and audio in one vector space) and `wemm-embedding-4b` from 0.18.3.
|
||||
is exactly what runs today. The pinned 0.18.20 image carries an 18-model catalog
|
||||
(`/opt/halogen/npu/models.txt`: `decider-0.8b`, `decider-4b`, `qwen3-embedding-0.6b`, `qwen3-embedding-4b`,
|
||||
`qwen3-reranker-0.6b`, `qwen3-reranker-4b`, `qwen3guard-gen-0.6b`, `qwen3guard-stream-0.6b`,
|
||||
`embeddinggemma-2`, `wemm-embedding-4b`, `qwen3.5-2b`, `flux2-klein-4b`, `qwen3-asr-0.6b`, `qwen3-asr-1.7b`,
|
||||
`d3-lite`, `d3-nano`, `qwen3-tts-0.6b-customvoice`, `qwen3-tts-1.7b-customvoice`), so **`/v1/embeddings` is one
|
||||
env var and three mounts away — no runtime bump needed.** The catalog is **not** version-stable: 0.18.20
|
||||
re-cut `qwen3-embedding-0.6b`'s device programs (14 `u*.elf` + an 8,495,512 B `consts.hnpb`, where 0.16.2
|
||||
had 10 and 5,086,496 B) and the entrypoint verifies size+sha256 per file before it starts, so **an engine
|
||||
bump re-fetches the staged set** — `scripts/download-frank-dragoon-npu.sh` carries the catalog block to do it.
|
||||
|
||||
Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts**
|
||||
(not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string
|
||||
@@ -282,7 +289,7 @@ lane is **armed and serving** since 2026-10-19 (`HALOGEN_NPU_MODELS=qwen3-embedd
|
||||
unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by
|
||||
size and sha256 against the catalog and the payload mount carries the tree in.
|
||||
|
||||
Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `15 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0…u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector).
|
||||
Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `19 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0, u1, u10, u11, u12, u13, u2, u3, u4, u5, u6, u7, u8, u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector).
|
||||
|
||||
Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default
|
||||
set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed"
|
||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 12118), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed"
|
||||
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
|
||||
tags:
|
||||
- llm
|
||||
@@ -31,7 +31,7 @@ is the bug.
|
||||
| Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) |
|
||||
| Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws |
|
||||
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file |
|
||||
| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload |
|
||||
| Runtime | `/home/dank/GitPackages/llama-prism-0244744/bin/llama-server` — PrismML **fork**, `0.7.0-dev (build 12118, commit 024474448)`; the only loader for this payload |
|
||||
| Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
|
||||
| Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
|
||||
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
|
||||
@@ -136,6 +136,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
|
||||
|
||||
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
||||
|---|---|---|---|---|---|
|
||||
| 2026-10-30 → | same 9B MTP file, Q8_0 mmproj and refusal adapter (payload untouched) | PrismML fork **12118** (`0.7.0-dev`, `024474448`) | unchanged (3 × 200,704; f16 K + q8_0 V; MTP depth 2) | fork rebuilt from `origin/prism`; the payload gate held — 202 `pq2_0` tensors and 202 Hadamard-folded weights loaded, MTP drafting engaged, chat/tools/vision answered | `LLAMA_BIN` back to `llama-prism-b10777/llama-server` (kept on disk) + redeploy |
|
||||
| 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads |
|
||||
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
|
||||
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
|
||||
@@ -201,7 +202,14 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
|
||||
- `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check.
|
||||
- `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change.
|
||||
- **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason.
|
||||
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter.
|
||||
- **A fork rebuild needs an install RPATH and a bin/lib-aware `LLAMA_BIN`.** `cmake --install` strips the
|
||||
build-tree rpath, so the installed `llama-server` dies on `libllama-server-impl.so` unless the configure
|
||||
carries `-DCMAKE_INSTALL_RPATH='$ORIGIN;$ORIGIN/../lib'`; the installed layout is `bin/` + `lib/`, not the
|
||||
flat directory b10777 shipped, so the env's `LLAMA_BIN` has to name `…/bin/llama-server`. Build 12118 was
|
||||
made from `origin/prism` with the 10777 configure (Release, `GGML_CUDA=ON`, `sm_86`,
|
||||
`CMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13`, Ninja) — gcc 15 is the box default and nvcc 12.4 rejects it.
|
||||
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter. Build 12118 clears all
|
||||
three.
|
||||
|
||||
## Contract
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11)
|
||||
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
|
||||
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
|
||||
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.18.20,
|
||||
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE
|
||||
# 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0)
|
||||
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
|
||||
|
||||
@@ -126,11 +126,22 @@ run_stage() {
|
||||
@test "the shipped env points at the fork runtime, the MTP pair and the refusal adapter" {
|
||||
# The payload path IS the endpoint's identity, and the runtime is part of
|
||||
# it here: the file's custom tensor type 142 + prism.hadamard.* metadata
|
||||
# make the PrismML build the only loader, and this build must be >= b10776
|
||||
# make the PrismML build the only loader, and that build must be >= b10776
|
||||
# because MTP beside q8_0 V is illegal below it. A retarget that moves the
|
||||
# model but leaves the old build behind ships an endpoint that cannot load
|
||||
# (type 142) or cannot run the documented MTP arm (the build gate).
|
||||
grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server$' "$ENV_FILE"
|
||||
# model but leaves a stock llama.cpp behind ships an endpoint that cannot
|
||||
# load (type 142) or cannot run the documented MTP arm (the build gate).
|
||||
#
|
||||
# Assert the fork FAMILY, not the directory one rebuild happens to install:
|
||||
# a literal path turns every runtime move into a test edit, and the suite
|
||||
# runs on a box that has no /home/dank at all, so existence cannot be
|
||||
# checked here either.
|
||||
local bin
|
||||
bin="$(sed -n 's/^LLAMA_BIN=//p' "$ENV_FILE")"
|
||||
if [[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/llama-server$ ]] &&
|
||||
[[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/bin/llama-server$ ]]; then
|
||||
echo "LLAMA_BIN does not name a PrismML fork build: '$bin'" >&2
|
||||
return 1
|
||||
fi
|
||||
grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0\.gguf$' "$ENV_FILE"
|
||||
grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0\.gguf$' "$ENV_FILE"
|
||||
grep -qE '^LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA\.gguf$' "$ENV_FILE"
|
||||
|
||||
Reference in new issue
Block a user