Compare commits

...
Author SHA1 Message Date
Frank 19fdffe8bd docs(agents): a start-only quadlet handler no-ops an image bump
The law today's authentik bump paid for: start on an active unit is a no-op, so
the new digest installs and the old container keeps serving behind three green
gates. Restart applies a pin.
2026-10-30 15:34:10 -07:00
Frank f7aec7f154 fix(authentik): restart the units so an image bump actually applies
The handler called `systemctl --user start` on all three units. Start is a
no-op for an already-active unit, so installing a new digest under an active
authentik left the old container serving: today's 2026.8.3 -> 2026.8.5 bump
moved only the OUTPOST (whose handler restarts) and left server + worker on
2026.8.3 while the deploy, the health gate and the e2e suite were all green.
The generator re-renders from the .container, so a restart is the step that
applies a pin.
2026-10-30 15:34:02 -07:00
Frank e1d4d59773 docs(dragoon,zealot): record both runtime pins as they now run
frank-dragoon: engine 0.18.20 (image e35133b2…), the 18-model NPU catalog, the
re-cut qwen3-embedding-0.6b device programs, and the wire surface the new engine
actually answers (GET /tokenize is 405, /props is 200, /completion and /ping
stay 404). Eras gains the 2026-10-30 row.

frank-zealot: fork build 12118 (0.7.0-dev, 024474448) with the payload-load
evidence (202 pq2_0 tensors, 202 Hadamard-folded weights), plus the law a
rebuild pays for: cmake --install strips the rpath and installs bin/+lib/, so
the configure needs CMAKE_INSTALL_RPATH and LLAMA_BIN names …/bin/llama-server.
2026-10-30 15:27:21 -07:00
Frank acbb539b97 feat(frank-zealot): PrismML fork b10777 -> build 12118 (0.7.0-dev, 024474448)
Rebuilt the fork from origin/prism at 024474448 with the same configure the
10777 cutover used (Release, GGML_CUDA=ON, sm_86, gcc-13 host compiler, Ninja),
installed to /home/dank/GitPackages/llama-prism-0244744. The payload gate holds:
libllama carries 42 prism.hadamard strings (b10777 had 29) and libggml-base
still registers PQ2_0, so type 142 and the hadamard metadata still load.

LLAMA_BIN inherits a bin/lib install layout now, so the install needs an RPATH
($ORIGIN;$ORIGIN/../lib) — cmake's install step strips the build rpath and the
binary otherwise dies on libllama-server-impl.so.

The unit test that pinned the 10777 path now asserts the invariant its own
comment described (LLAMA_BIN names a llama-prism-* build) instead of a
directory one rebuild happens to install; the suite runs on a box with no
/home/dank, so existence is not checkable there.

Rollback: repoint LLAMA_BIN at llama-prism-b10777/llama-server (still on disk).
2026-10-30 15:25:24 -07:00
Frank 011ceeb344 fix(npu fetch): never resume into a file from another revision
A version bump re-cuts the device programs, so a staged file of a different
size is stale rather than partial. `curl -C -` against it appended the new
tail to the old bytes and produced the right BYTE COUNT with the wrong
content — the sha256 gate caught it (consts.hnpb, 2026-10-30), and the
failure reads as a corrupt program rather than a fetch fault. Resume only
when the staged file is smaller than the manifest expects.
2026-10-30 15:13:42 -07:00
Frank 33396d2daa feat(frank-dragoon): halogen-flash-server 0.16.2 -> 0.18.20, NPU lane re-pinned
The engine pin moves to the newest vendor release (0.18.20, image built
2026-10-30). The payload lock is the gate this deploy has to pass: the v2
.hgn loads only in the flash-server line, so /health plus the boot ledger
decide, and the rollback is the 0c61bf84 digest on the same env line.

0.18.20's NPU catalog is NOT the 0.16.2 one: it names 14 device programs
(u0..u13) and an 8,495,512 B consts.hnpb where 0.16.2 had 10 and 5,086,496 B,
and the entrypoint verifies size+sha256 for every staged file before it
starts. HALOGEN_NPU_MODELS names qwen3-embedding-0.6b, so the staging set is
re-pinned to revision 8fd44c9f with the new block — bumping the engine alone
would refuse the boot on the NPU lane.
2026-10-30 15:13:19 -07:00
13 changed files with 123 additions and 63 deletions

No files matched your search

+15 -8
View File
@@ -26,8 +26,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Payload | Port | | Target | Service | Runtime | Payload | Port |
|--------|---------|---------|---------|------| |--------|---------|---------|---------|------|
| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 | | Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 | | Dankclaw | `frank-dragoon` | halogen-flash-server 0.18.20, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 | | Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 12118, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj + rank-1 refusal adapter | 7771 |
Payload identity, shape, eras, laws and rollback live in ONE page each: Payload identity, shape, eras, laws and rollback live in ONE page each:
[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env` [[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
@@ -293,8 +293,8 @@ means off or NIC dead; a merely wedged kernel usually still answers ARP.
2. Bench against the service itself. No scratch ports on an LLM host — one 2. Bench against the service itself. No scratch ports on an LLM host — one
process owns the card. Use the runtime's own harness (`llama-bench`; a process owns the card. Use the runtime's own harness (`llama-bench`; a
live llama.cpp server also reports honest `timings` per request, including live llama.cpp server also reports honest `timings` per request, including
`draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build `draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork that object
10777 that object only reaches you on **`/v1/chat/completions`**: a greedy only reaches you on **`/v1/chat/completions`** (measured on build 10777): a greedy
`ignore_eos` run on the raw `/completion` path there answers **500 "The `ignore_eos` run on the raw `/completion` path there answers **500 "The
model produced output that does not match the expected Content-only model produced output that does not match the expected Content-only
format"** (measured 2026-10-18). halogen has no `/completion` path at all, format"** (measured 2026-10-18). halogen has no `/completion` path at all,
@@ -439,16 +439,16 @@ disagrees with it is the bug.
### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771 ### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
- Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`, - Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
`scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY `scripts/frank-zealot.service`; fork build `llama-prism-0244744` (build 12118)
loader (the payload carries the fork's custom PQ2_0 tensor type plus is the ONLY loader — `llama-prism-b10777` stays on disk as its rollback (the payload carries the fork's custom PQ2_0 tensor type plus
`prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a `prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
loader ignoring that metadata dequantizes wrong); payload loader ignoring that metadata dequantizes wrong); payload
`Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16 `Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
tower can crash the server with concurrent image requests) + the rank-1 tower can crash the server with concurrent image requests) + the rank-1
refusal-direction adapter applied at load by `--lora-scaled …:1`. The refusal-direction adapter applied at load by `--lora-scaled …:1`. The
adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild adapter's rollback is `LORA_FILE=` (empty) in the env — one line, no rebuild
— and build 10777 is also the gate that lets it load: older forks refuse it — and the fork BUILD is also the gate that lets it load: anything older than
with `unsupported prism.hadamard.version: 2`. b10777 refuses it with `unsupported prism.hadamard.version: 2`.
- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at - Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088 200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024, B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
@@ -741,6 +741,13 @@ linked runbooks.
`/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic `/etc/default/grub`. It replaced `amd_iommu=off`, and the local A/B on byte-identical traffic
says identity mode costs nothing, so the XDNA2 NPU returned for free. says identity mode costs nothing, so the XDNA2 NPU returned for free.
[[dankclaw-iommu-translation-off]] [[dankclaw-iommu-translation-off]]
- **A deploy handler that only `start`s a quadlet no-ops an image bump.**
`systemctl --user start` on an already-active unit does nothing, so installing
a new digest under a running service keeps the OLD container serving while the
deploy, its health gate and the Playwright suite all pass — the authentik
handler did exactly that on 2026-10-30 (2026.8.3 → 2026.8.5 moved only the
outpost, the one handler that restarted). A pin change is applied by
`restart`, never `start`.
- **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT - **Rootless Quadlet containers run with `--replace --rm`**: `podman logs` covers only the CURRENT
instance; history across recreations lives in `journalctl --user -u <name>.service`. Read instance; history across recreations lives in `journalctl --user -u <name>.service`. Read
rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing. rootless storage WITHOUT sudo — it is per-user storage, so `sudo podman logs` finds nothing.
+1 -1
View File
@@ -21,7 +21,7 @@
# deployed knob has since dropped to 262144. # deployed knob has since dropped to 262144.
set -euo pipefail set -euo pipefail
IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a IMG=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0
CUR=/home/dank/models/halogen-flashnext-v2 CUR=/home/dank/models/halogen-flashnext-v2
NEW=/home/dank/models/halogen-v2-abliterated NEW=/home/dank/models/halogen-v2-abliterated
CUR_CK=qwen38-flash-next-v2-swift15-abliterated.hgn CUR_CK=qwen38-flash-next-v2-swift15-abliterated.hgn
+10 -4
View File
@@ -1362,11 +1362,17 @@ deploy_authentik() {
# generated". Boot persistence comes from [Install] WantedBy=default.target # generated". Boot persistence comes from [Install] WantedBy=default.target
# inside each .container file, which the generator honors at user-manager # inside each .container file, which the generator honors at user-manager
# start (linger must stay on for the deploy user). # start (linger must stay on for the deploy user).
systemctl --user start authentik-postgres.service # RESTART, not start: `start` on an already-active unit is a no-op, so an
# image bump would install the new digest and keep serving the old
# container while every gate stayed green (measured 2026-10-30: server and
# worker stayed on the previous release beside an outpost that moved,
# because only the outpost handler restarts). The generator has already
# re-rendered the unit from the .container, so a restart is what applies it.
systemctl --user restart authentik-postgres.service
wait_container_healthy authentik-postgres 120 || return 1 wait_container_healthy authentik-postgres 120 || return 1
systemctl --user start authentik-server.service systemctl --user restart authentik-server.service
wait_container_healthy authentik-server 300 || return 1 wait_container_healthy authentik-server 300 || return 1
systemctl --user start authentik-worker.service systemctl --user restart authentik-worker.service
} }
deploy_authentik_outpost() { deploy_authentik_outpost() {
@@ -1482,7 +1488,7 @@ deploy_anubis() {
# artifact would leave a restart-looping unit and a dark endpoint. The # artifact would leave a restart-looping unit and a dark endpoint. The
# port gate is the same as every cutover since 2026-09-16: nothing # port gate is the same as every cutover since 2026-09-16: nothing
# starts until 7770 is released. # starts until 7770 is released.
LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}" LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0}"
# The payload dir is its own twin because the NPU lane's containment gate below # The payload dir is its own twin because the NPU lane's containment gate below
# reads it; every artifact path under it stays a full literal as usual. # reads it; every artifact path under it stays a full literal as usual.
LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}" LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}"
+33 -16
View File
@@ -8,8 +8,10 @@
# (/opt/halogen/npu/models.txt inside the image) and refuses to load a model # (/opt/halogen/npu/models.txt inside the image) and refuses to load a model
# whose files do not match. # whose files do not match.
# #
# The manifest below IS that catalog block (dumped from the pinned 0.16.2 # The manifest below IS that catalog block (dumped from the pinned 0.18.20
# image on 2026-10-19), so the files land in the layout the entrypoint # image on 2026-10-30; the 0.16.2 block this replaced named different device
# programs — 10 `u*.elf` and a 5,086,496 B consts.hnpb — so a version bump
# re-fetches the set or the entrypoint refuses the boot), so the files land in the layout the entrypoint
# expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and # expects — <dir>/<model-id>/devices/*, <dir>/<model-id>/<model-id>.hnpw and
# <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep # <dir>/<model-id>/tokenizer/tokenizer.json — and the endpoint can keep
# /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD # /models read-only and open no outbound connection at boot (HALOGEN_DOWNLOAD
@@ -57,21 +59,25 @@ manifest() {
case "$1" in case "$1" in
qwen3-embedding-0.6b) qwen3-embedding-0.6b)
NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b" NPU_REPO="peonist-ai/halogen-npu-qwen3-embedding-0.6b"
NPU_REV="4960fd9d4423dd15373ad21c83db48dbdb3b2798" NPU_REV="8fd44c9f18861618a509bea75c51ff8a37e3bd5d"
FILES=( FILES=(
"devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91" "devices/NOTICE.md 3248 0f267b3d2523ddf14997ee58660a492a9eaf10ccf6b08bf19dfdc85b9a478b91"
"devices/consts.hnpb 5086496 ae430e41545a0b1012938cda373b0c450364d3be631111cca3a8c43679f93336" "devices/consts.hnpb 8495512 5af7c1f2f24d45a9054872bba882461cf7f6ffda832e785b700643d6467ad88b"
"devices/devices.hnpm 24047 4e9bca9d95d084fb89ce5b99f579a00666c8ca1efae2613c53e3a4878cc9f174" "devices/devices.hnpm 35465 85d870f6b900de1829659955f0dc701d57984580f5e5ec251add5d6969f03e3d"
"devices/u0.elf 4931968 b38f165886cb037b7d3951d9cb878473eaba58a432acd45416ff85fd88aa4e46" "devices/u0.elf 2975008 f74df12eb8c79e4e90cbedcc1dd8f142219772d400e3a142fde8a95378b105eb"
"devices/u1.elf 5456000 b1811feee851b8af833b37721da92099d89772110815bbbe2c3fa1c276f0ff42" "devices/u1.elf 3266848 ac5a3f89f934951b315b62ab53e1d0a74dbe6396134192641c405d08da332754"
"devices/u2.elf 6505552 406ea9a95b470628a7382e376f4a72cdf4d7aa4c59b4ddf0a4bdd139d63d6ff2" "devices/u2.elf 4015328 5e4f0d3342f262816289866d7e8c966f56a9f938c2e12e7341718cc7312aa480"
"devices/u3.elf 6817216 bc205d51b523d770151a0c61968f7c70bf624412827580bb6fe0aaf4650dc2ee" "devices/u3.elf 4326976 94cf5d269fa286d91c26bf4654cedf101c047d23c4495b74c9c129cdc33b210f"
"devices/u4.elf 7444032 a5a91338a2603acdcd5c11df532033fda2d3f5479ea44ae87f98fafbdff9a62d" "devices/u4.elf 4953792 cbd2796050d4c675f5cb3da117111087d510173e26f262f1b36aab66dd49dc91"
"devices/u5.elf 7432416 6c2bf41835569e0eba25ded7c05c7264ae69c6700bdeb4791057d884f2a4c126" "devices/u5.elf 4940112 7bf6a4470546bb888da02af395c07d92a24fb6d43aef27eb2151b3a9eda19e4d"
"devices/u6.elf 4941232 919d0038ca088eafddf195f3466ff38db27d3635be1cffa5d4d4b5140902a68e" "devices/u6.elf 6174544 10b9d50b635993e1f9c23775c0b94901d922c1e433cd10910e9f97174235f5ac"
"devices/u7.elf 5488304 d904cf8afae9428dd167960549e18cce5441745f18c62f8c50975dffc077d3c2" "devices/u7.elf 2983776 e0b06063b7379d05c85b3f435b819d96fda1223d8025d6d15958854b5711ace9"
"devices/u8.elf 4941232 db3aef69b17825854ee0dd2df475125a3f5c872f95c686793d6af1ee83596fa7" "devices/u8.elf 3298656 3bad204f782a68b97e27c1172530b13d7fbb0b495fa95f9504530a1048560644"
"devices/u9.elf 5488304 d182727e551084db8fb9600441d8aa373c7f09a3d267ae0a5e921ef92326db1c" "devices/u9.elf 2983776 875e313280358ce116e2a51a29a2c42794b5a9fadb3d48db702c90d18d838c2c"
"devices/u10.elf 3298656 ef2e0f399f149a5476955bcf233c6fa2e896fcef1f86ecb0d8113c952c41a8b2"
"devices/u11.elf 3298656 68a2519eee2d6a61ccc4ca2007e3186d44c9eb4e81162d742fe2f1756ec5a528"
"devices/u12.elf 3298656 2bdc6abfa58cac1d5795a3ef52fe8a9ab4e7c2a4792822d615499a913401f728"
"devices/u13.elf 3298656 4a3ce14f7ba15987b9bcbb4a010083e8aef68cd87c20cbe43403a98263179fe2"
"qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1" "qwen3-embedding-0.6b.hnpw 813032248 b9f9901429df7b46ec17153aa3899f738adbea8e4356080da0c6e6586fb750d1"
"tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a" "tokenizer/tokenizer.json 11423705 def76fb086971c7867b829c23a26261e38d9d74e02139253b38aeb9df8b4b50a"
) ;; ) ;;
@@ -99,7 +105,18 @@ for MODEL in ${MODELS//,/ }; do
# 62 GiB fetch hit this on 2026-10-18). # 62 GiB fetch hit this on 2026-10-18).
url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \ url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$HF/$NPU_REPO/resolve/$NPU_REV/$f") \
|| { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; } || { echo "cannot resolve $NPU_REPO/$f" >&2; exit 1; }
curl -fL --retry 5 -C - -o "$path" "$url" # Resume ONLY into a genuinely partial file. A staged file of a
# DIFFERENT size is an artifact of another revision (a version
# bump re-cuts the device programs), and `-C -` against it
# appends the tail to the old bytes: the result is the right
# BYTE COUNT with the wrong content, which reads as a corrupt
# program, not as a fetch fault (measured 2026-10-30 on
# consts.hnpb). Fetch those from zero.
resume=()
if [ -f "$path" ] && [ "$(stat -c %s "$path")" -lt "$want" ]; then
resume=(-C -)
fi
curl -fL --retry 5 "${resume[@]+"${resume[@]}"}" -o "$path" "$url"
fi fi
fi fi
[ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; } [ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
+5 -2
View File
@@ -10,8 +10,11 @@
# The .hgn format is the lock: it loads ONLY in the flash-server line, so the # The .hgn format is the lock: it loads ONLY in the flash-server line, so the
# engine moves with the payload, never ahead of it. # engine moves with the payload, never ahead of it.
# Image digest-pinned (a tag pull silently re-points; the digest is what was # Image digest-pinned (a tag pull silently re-points; the digest is what was
# qualified — 0.16.2 is the version the model card names as its quickstart pin). # qualified). 0.18.20 since 2026-10-30 (was 0.16.2, the model card's quickstart
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a # pin) — the vendor has moved two minor versions and the payload lock is what
# the deploy's health gate plus the boot ledger decide; rollback is the
# previous digest, 0c61bf84…, on this same line.
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0
# Container identity kept from the 27B era: the unit's ExecStop names it. # Container identity kept from the 27B era: the unit's ExecStop names it.
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
# Predecessor identity the serve script clears if it is still holding the GPUs. # Predecessor identity the serve script clears if it is still holding the GPUs.
+5 -4
View File
@@ -1,12 +1,12 @@
# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090, # frank-zealot — llama.cpp (PrismML fork build 12118 = 0.7.0-dev, commit
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus # 024474448, CUDA) on the RTX 3090, :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
# the refusal-direction adapter applied at load. # the refusal-direction adapter applied at load.
# #
# ONE OWNER: shape, payload identity, eras, laws and every measured number # ONE OWNER: shape, payload identity, eras, laws and every measured number
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the # live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
# knobs and only the reason a reader editing THIS line cannot recover # knobs and only the reason a reader editing THIS line cannot recover
# elsewhere. Where the two disagree, this file wins on values. # elsewhere. Where the two disagree, this file wins on values.
LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server LLAMA_BIN=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server
# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.* # The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard # metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
# metadata dequantizes wrong, silently. # metadata dequantizes wrong, silently.
@@ -64,7 +64,8 @@ CACHE_TYPE_V=q8_0
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of # weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for # draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V # by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777. # beside MTP needs fork build >= b10776; LLAMA_BIN is 12118 (b10777 is kept on
# disk as the rollback path).
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and # SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
# every --spec-draft-*); half a set has segfaulted at load. # every --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=draft-mtp SPEC_TYPE=draft-mtp
+1 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.16.2 # serve-frank-dragoon.sh — frank-dragoon endpoint: halogen-flash-server 0.18.20
# (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server, # (closed-source engine, image ghcr.io/peonist-ai/halogen-flash-server,
# digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2 # digest-pinned in frank-dragoon.env) serving Quat3rnion's NATIVE v2
# conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's # conversion (2026-10-11) of the abliterated Qwen3.8-Flash-Next on Dankclaw's
+3 -3
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Serve prism-ml/Ternary-Bonsai-2-9B-gguf (PQ2_0 ternary quant WITH the MTP head # Serve prism-ml/Ternary-Bonsai-2-9B-gguf (PQ2_0 ternary quant WITH the MTP head
# in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML # in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
# vendor build 10777, CUDA sm_86). # vendor build 12118 (0.7.0-dev, commit 024474448), CUDA sm_86).
# #
# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same # 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over # evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
@@ -78,7 +78,7 @@ if [[ -f "$ENV_FILE" ]]; then
fi fi
# --- defaults (used when the env file does not set them) ------------------- # --- defaults (used when the env file does not set them) -------------------
: "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-b10777/llama-server}" : "${LLAMA_BIN:=/home/dank/GitPackages/llama-prism-0244744/bin/llama-server}"
: "${MODEL_FILE:=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf}" : "${MODEL_FILE:=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf}"
: "${ALIAS:=frank-zealot}" : "${ALIAS:=frank-zealot}"
: "${BIND_ADDR:=0.0.0.0}" : "${BIND_ADDR:=0.0.0.0}"
@@ -119,7 +119,7 @@ fi
# above are what pay for it. The draft cache dtype IS a further knob # above are what pay for it. The draft cache dtype IS a further knob
# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this # (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
# build's --help) and stays at its default here. Card gate: quantized V alongside # build's --help) and stays at its default here. Card gate: quantized V alongside
# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777 # MTP needs fork build >= prism-b10776; LLAMA_BIN is 12118 (b10777 kept as rollback)
# prebuilt; the type-142 loader lock keeps it pinned regardless. # prebuilt; the type-142 loader lock keeps it pinned regardless.
# SPEC_TYPE=none is the rollback and strips the whole set (flags move together). # SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
: "${SPEC_TYPE:=draft-mtp}" : "${SPEC_TYPE:=draft-mtp}"
@@ -10,7 +10,7 @@ timestamp: 2026-09-03
## Server Overview ## Server Overview
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771). Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.18.20 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
## Hardware Specifications ## Hardware Specifications
### CPU ### CPU
@@ -105,7 +105,7 @@ cable between claw and server is required for 10G.
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM. - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget. - The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`. - The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.18.20** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
- The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either. - The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18. - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage. - 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
+21 -14
View File
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)" title: "frank-dragoon on Dankclaw — halogen-flash-server 0.18.20, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it." description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
tags: tags:
- llm - llm
@@ -10,10 +10,10 @@ tags:
- gfx1151 - gfx1151
- halogen - halogen
- endpoint - endpoint
timestamp: 2026-10-21 timestamp: 2026-10-30
--- ---
# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU) # frank-dragoon — halogen-flash-server 0.18.20, native v2 abliterated (Dankclaw iGPU)
Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18 under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
@@ -33,8 +33,8 @@ flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, ver
| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON | | `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked | | `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a` Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:e35133b206da3094cc4d70c78800b26235bd9124b9aeb0e029d4a7c153b9cff0`
(0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's (0.18.20, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
unauthenticated token port 8730 stays on container loopback. Engine stdout → unauthenticated token port 8730 stays on container loopback. Engine stdout →
`/home/dank/logs/frank_dragoon.log`, **not** the journal. `/home/dank/logs/frank_dragoon.log`, **not** the journal.
@@ -74,6 +74,7 @@ rollback plan — recoverable only until the trash is purged.
| Window | Payload | Runtime | Shape | Replaced because | Rollback | | Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 2026-10-30 → | same v2 abliterated `.hgn` trio (payload untouched) | flash-server **0.18.20** `e35133b2…` | unchanged (2 slots, 262,144 ceiling, pool 401,408, MTP always-on) | vendor moved two minor versions past the 0.16.2 quickstart pin; the payload lock **held** — the same boot served chat, a `tool_calls` turn, vision ("Red", 132 prompt tokens) and `/v1/embeddings`, with the NPU lane's 19-file set verified against the new catalog | env line back to `0c61bf84…` + redeploy |
| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` | | 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | — | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage | | 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` | | 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | — | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
@@ -190,9 +191,11 @@ created it — earlier topologies in .
## Contract (wire) ## Contract (wire)
- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props` - Live: `/v1/models`, `/health`, `/metrics`, `/props` (model_alias, total_slots, chat_template),
`/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure POST `/v1/chat/completions`, POST `/v1/embeddings` when the NPU lane is armed; `404`: `/ping`,
wall-clock plus `usage.prompt_tokens`. `/completion`; `/tokenize` answers **405 to a GET** (the route exists and wants POST). **No
`Server-Timing` header and no `/completion` timing object** — measure wall-clock plus
`usage.prompt_tokens`.
- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies - Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476 vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
@@ -232,11 +235,15 @@ created it — earlier topologies in .
XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at
([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the ([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the
same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server same container behind the same port**, and a request picks one by its `model`; leave the variable out and the server
is exactly what runs today. The pinned 0.16.2 image already carries the five-program catalog is exactly what runs today. The pinned 0.18.20 image carries an 18-model catalog
(`/opt/halogen/npu/models.txt`: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`, (`/opt/halogen/npu/models.txt`: `decider-0.8b`, `decider-4b`, `qwen3-embedding-0.6b`, `qwen3-embedding-4b`,
`qwen3.5-2b`) and `docs/NPU.md` at tag `v0.16.2` documents it, so **`/v1/embeddings` is one env var and three mounts `qwen3-reranker-0.6b`, `qwen3-reranker-4b`, `qwen3guard-gen-0.6b`, `qwen3guard-stream-0.6b`,
away — no runtime bump needed.** Wider catalog: `qwen3-embedding-4b` and `decider-4b` from 0.17.4, `embeddinggemma-2` `embeddinggemma-2`, `wemm-embedding-4b`, `qwen3.5-2b`, `flux2-klein-4b`, `qwen3-asr-0.6b`, `qwen3-asr-1.7b`,
(text ≤ 8,192 tokens, pictures and audio in one vector space) and `wemm-embedding-4b` from 0.18.3. `d3-lite`, `d3-nano`, `qwen3-tts-0.6b-customvoice`, `qwen3-tts-1.7b-customvoice`), so **`/v1/embeddings` is one
env var and three mounts away — no runtime bump needed.** The catalog is **not** version-stable: 0.18.20
re-cut `qwen3-embedding-0.6b`'s device programs (14 `u*.elf` + an 8,495,512 B `consts.hnpb`, where 0.16.2
had 10 and 5,086,496 B) and the entrypoint verifies size+sha256 per file before it starts, so **an engine
bump re-fetches the staged set** — `scripts/download-frank-dragoon-npu.sh` carries the catalog block to do it.
Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts** Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding-0.6b"`, `input` a list of **texts**
(not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string (not token ids), 1024 unit-length floats back, the Qwen instruct prefix goes inside the string
@@ -282,7 +289,7 @@ lane is **armed and serving** since 2026-10-19 (`HALOGEN_NPU_MODELS=qwen3-embedd
unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by
size and sha256 against the catalog and the payload mount carries the tree in. size and sha256 against the catalog and the payload mount carries the tree in.
Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `15 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0…u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector). Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `19 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0, u1, u10, u11, u12, u13, u2, u3, u4, u5, u6, u7, u8, u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector).
Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default
set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus, set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus,
+11 -3
View File
@@ -1,6 +1,6 @@
--- ---
type: reference type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed" title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 12118), Ternary-Bonsai-2-9B PQ2_0-MTP + refusal adapter, 3 × 200704, f16 K + q8_0 V, MTP armed"
description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws." description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
tags: tags:
- llm - llm
@@ -31,7 +31,7 @@ is the bug.
| Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) | | Projector | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, sha256 `e0cb6f6988cea323…` — **Q8_0 is mandatory**, not a preference (see Contract) |
| Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws | | Adapter | `…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` — **3,155,424 B**, sha256 `861b9c099fb5cd81…` — a rank-1 refusal-direction LoRA (`adapter.type lora`, 128 tensors, every transformer layer, embedding untouched), applied at load by `--lora-scaled <file>:1`. Its metadata names `Ternary-Bonsai-2-9B-PTQ1_0.gguf` (`3561df7a…`) as the pack it was fitted against — **not this packing, and not on this host**; see Laws |
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file | | Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (payload + projector) · `scripts/download-frank-zealot-uncensored-lora.sh [DEST]` (the adapter) — both `--verify-only`-able, `-f`, byte count and sha256 per file |
| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork**, `0.2.0-dev (build 10777, commit e3de71522)`; the only loader for this payload | | Runtime | `/home/dank/GitPackages/llama-prism-0244744/bin/llama-server` — PrismML **fork**, `0.7.0-dev (build 12118, commit 024474448)`; the only loader for this payload |
| Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` | | Trio | `scripts/frank-zealot.service` (root, relinked, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`, `TimeoutStopSec=180`) · `scripts/frank-zealot.env` · `scripts/serve-frank-zealot.sh` |
| Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) | | Address / alias | `BIND_ADDR=0.0.0.0`, `LISTEN_PORT=7771`, `ALIAS=frank-zealot` (a flag, not a fact) |
| Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` | | Flags | `--ctx-size 602112 --parallel 3` ⇒ 200,704 per slot · `--cache-type-k f16 --cache-type-v q8_0` (asymmetric, **14,406.00 MiB**) · `--flash-attn on` · `--batch-size 6144 --ubatch-size 1024` · `--n-gpu-layers 99` · `--jinja --reasoning auto --reasoning-format deepseek` · `--metrics` · `--mmproj` · **`--lora-scaled Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf:1`** · `--log-verbosity 4` · `--presence-penalty 1.0` · **`--spec-type draft-mtp`** with `--spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0` |
@@ -136,6 +136,7 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
| Window | Payload | Runtime | Shape | Replaced because | Rollback | | Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 2026-10-30 → | same 9B MTP file, Q8_0 mmproj and refusal adapter (payload untouched) | PrismML fork **12118** (`0.7.0-dev`, `024474448`) | unchanged (3 × 200,704; f16 K + q8_0 V; MTP depth 2) | fork rebuilt from `origin/prism`; the payload gate held — 202 `pq2_0` tensors and 202 Hadamard-folded weights loaded, MTP drafting engaged, chat/tools/vision answered | `LLAMA_BIN` back to `llama-prism-b10777/llama-server` (kept on disk) + redeploy |
| 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads | | 2026-10-19 → | the same 9B MTP file and projector, **plus** `Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf` (3,155,424 B `861b9c09…`) applied by `--lora-scaled …:1` | PrismML fork 10777 (its verified build) | — (shape unchanged; the adapter is behaviour) | Frank ordered the refusal direction removed on this endpoint | one env line — `LORA_FILE=` (or `LORA_SCALE=0`) — and redeploy; the plain pack is the same file it already loads |
| 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move | | 2026-10-18 21:06 → 2026-10-19 | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352 | one line each — `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` — and redeploy; no payload move |
| 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move | | 2026-10-18 13:19 → same evening (the boot `68afc6d` replaced) | the same 9B MTP file and projector | PrismML fork 10777 | — | Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable | one env line — `CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0` — and redeploy; no payload move |
@@ -201,7 +202,14 @@ with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY c
- `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check. - `serve-frank-zealot.sh` is a direct `ExecStart`, so a missing shebang is a `203/EXEC` boot loop at `Mem peak: 1.9M` with the GPU untouched: `head -1` is a deploy check.
- `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change. - `No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; KV spilled into unified memory halves decode silently, so re-read `nvidia-smi` after any shape change.
- **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason. - **An adapter's provenance is metadata, not a gate.** Its GGUF carries `prism.abliterate.base_gguf = Ternary-Bonsai-2-9B-PTQ1_0.gguf` and that pack's sha256; the endpoint applies the file to the PQ2_0+MTP packing anyway, because the loader matches **tensor names, not hashes**. So an adapter's effect is only ever proven behaviourally (a refusal probe against the live endpoint), never by the card, and an adapter whose rank landed on `token_embd` would be a crash rather than a degradation — the publisher leaves the embedding untouched for exactly that reason.
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter. - **A fork rebuild needs an install RPATH and a bin/lib-aware `LLAMA_BIN`.** `cmake --install` strips the
build-tree rpath, so the installed `llama-server` dies on `libllama-server-impl.so` unless the configure
carries `-DCMAKE_INSTALL_RPATH='$ORIGIN;$ORIGIN/../lib'`; the installed layout is `bin/` + `lib/`, not the
flat directory b10777 shipped, so the env's `LLAMA_BIN` has to name `…/bin/llama-server`. Build 12118 was
made from `origin/prism` with the 10777 configure (Release, `GGML_CUDA=ON`, `sm_86`,
`CMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13`, Ninja) — gcc 15 is the box default and nvcc 12.4 rejects it.
- **The fork pin now has a third reason.** The payload's custom type 142, the b10776 floor that pairs MTP with a quantized V, and now this: older PrismML builds (e.g. `prism-b10709`) refuse the adapter outright with `unsupported prism.hadamard.version: 2`, so a runtime that can load the pack can still be unable to carry the adapter. Build 12118 clears all
three.
## Contract ## Contract
+1 -1
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bats #!/usr/bin/env bats
# Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11) # Seam: frank-dragoon serves Quat3rnion's NATIVE v2 conversion (2026-10-11)
# of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the # of the abliterated Qwen3.8-Flash-Next (a .hgn flash-server payload) on the
# Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2, # Strix Halo iGPU through the halogen-flash-server CONTAINER (0.18.20,
# digest-pinned), driven by the root unit's serve script — 2 slots over ONE # digest-pinned), driven by the root unit's serve script — 2 slots over ONE
# 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0) # 401,408-position KV pool at the native 262,144 ceiling, MTP head always on (HALOGEN_SPEC_ADAPT=0)
# and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune # and the vision tower mounted. 2026-10-17 cutover from the Swift1.5-finetune
+15 -4
View File
@@ -126,11 +126,22 @@ run_stage() {
@test "the shipped env points at the fork runtime, the MTP pair and the refusal adapter" { @test "the shipped env points at the fork runtime, the MTP pair and the refusal adapter" {
# The payload path IS the endpoint's identity, and the runtime is part of # The payload path IS the endpoint's identity, and the runtime is part of
# it here: the file's custom tensor type 142 + prism.hadamard.* metadata # it here: the file's custom tensor type 142 + prism.hadamard.* metadata
# make the PrismML build the only loader, and this build must be >= b10776 # make the PrismML build the only loader, and that build must be >= b10776
# because MTP beside q8_0 V is illegal below it. A retarget that moves the # because MTP beside q8_0 V is illegal below it. A retarget that moves the
# model but leaves the old build behind ships an endpoint that cannot load # model but leaves a stock llama.cpp behind ships an endpoint that cannot
# (type 142) or cannot run the documented MTP arm (the build gate). # load (type 142) or cannot run the documented MTP arm (the build gate).
grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server$' "$ENV_FILE" #
# Assert the fork FAMILY, not the directory one rebuild happens to install:
# a literal path turns every runtime move into a test edit, and the suite
# runs on a box that has no /home/dank at all, so existence cannot be
# checked here either.
local bin
bin="$(sed -n 's/^LLAMA_BIN=//p' "$ENV_FILE")"
if [[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/llama-server$ ]] &&
[[ ! "$bin" =~ ^/home/dank/GitPackages/llama-prism-[^/]+/bin/llama-server$ ]]; then
echo "LLAMA_BIN does not name a PrismML fork build: '$bin'" >&2
return 1
fi
grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0\.gguf$' "$ENV_FILE" grep -qE '^MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0\.gguf$' "$ENV_FILE"
grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0\.gguf$' "$ENV_FILE" grep -qE '^MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0\.gguf$' "$ENV_FILE"
grep -qE '^LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA\.gguf$' "$ENV_FILE" grep -qE '^LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA\.gguf$' "$ENV_FILE"