Frank
6dcf4a5913
Model swap on the radiance endpoint: MODEL_FILE moves from the in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (116,416 MiB) to the engine author's published qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, official Qwen3.8-Flash-Next bf16 -> 4-bit non-uniform codebook experts + int8 trunk, card KL 0.0789 / top-1 92.85 % vs bf16). NO argument changed: 25600/5120 vram pair, 44928 host pool, fp8 KV, 3 x 200000, MTP depth 2, wht6, MNBT 2048, froggeric v22.5 — the 2026-10-13 shape carries verbatim. New fetch script scripts/download-frank-archon-iq4r.sh (hfget + exact byte count 121,969,901,568 + LFS sha256, --verify-only default, ungated, no token on the command line). Unit/serve/deployer lore re-keyed; the seam pins the new payload id + sha256 and keeps the w4dn8 rollback provenance. Homepage card points at the publisher repo. Rollback: revert this commit (w4dn8 .rad stays on disk). The fp8-native rad was deleted from dankserver 2026-10-15 (Frank's one-time rm, volume for this payload); rebuild path documented.
72 lines
3.9 KiB
Desktop File
72 lines
3.9 KiB
Desktop File
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
|
# (the publisher's 4-bit-codebook container of the official
|
|
# Qwen/Qwen3.8-Flash-Next, qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire),
|
|
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 (the
|
|
# swap commit is the revert; its .rad stays on disk).
|
|
#
|
|
# Same port, same served alias, same container identity as the unit name;
|
|
# the harness models.yml id is unchanged. The serve script drives
|
|
# `podman run` directly: the digest-pinned stilldeadcode/radiance image
|
|
# (287 MiB, kernels prebuilt for gfx1201) has the engine as its
|
|
# entrypoint. The image lives in dank's ROOTLESS podman storage.
|
|
# WHY A ROOT UNIT (not a rootless quadlet): radiance pins ~132 GiB of HOST
|
|
# RAM (2 x 40 GiB expert pools + the 47.7 GiB PLE ngram table + the prefix
|
|
# host tier). Pinned pages are charged against RLIMIT_MEMLOCK; dank's login
|
|
# session on this host carries 8 MiB and a user quadlet is capped by the
|
|
# user manager's own hard limit — raising that means restarting
|
|
# user@1000.service and bouncing every user container on the host. A root
|
|
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
|
# engine still runs out of dank's rootless podman storage.
|
|
[Unit]
|
|
Description=frank-archon — radiance Qwen3.8-Flash-Next (publisher iq4r) on 2x R9700 TP=2
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
|
# endpoint that systemd refuses to retry is worse than a retrying one).
|
|
StartLimitIntervalSec=0
|
|
|
|
[Service]
|
|
User=dank
|
|
Group=dank
|
|
# The engine pins ~132 GiB of host RAM (expert pools + PLE table); an
|
|
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
|
|
# PID1 grants infinity here — parity with the fleet's other LLM units.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
|
|
# The `+` prefix runs this as root even though the service is User=dank — the
|
|
# ttm pages cap is a root-owned knob and the pinned-RAM ledger (~147 GiB) is
|
|
# unusable below it. After a reboot the knob is back at its default (half of
|
|
# RAM), so the raise must be part of the start, not a note in a runbook: the
|
|
# 2026-10-12 power failure reset it and the endpoint refused to boot 6,919
|
|
# times. The serve script re-verifies the cap after this hook.
|
|
ExecStartPre=+/home/dank/serverAppConfigs/scripts/ttm-preflight.sh --raise
|
|
# systemd stop: wait alone cannot stop the container child; stop it
|
|
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
|
# holds no queue to drain — in-flight decodes are lost, which a stop
|
|
# during a bench window accepts; radiance takes SIGTERM on the engine
|
|
# process and podman reaps the container).
|
|
ExecStop=/usr/bin/podman stop -t 120 frank-archon
|
|
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
|
|
# publisher's compose healthcheck allows a 15-minute start_period; the
|
|
# vLLM-era budget (1800 s) carried forward.
|
|
# Restart=always, NOT on-failure (the 2026-10-01 vLLM lesson carries over:
|
|
# a clean API-server exit after a worker fault reads as success to
|
|
# systemd). `always` restarts on any container death; an explicit
|
|
# `systemctl stop` still wins, so bench windows and deploys are unaffected.
|
|
Restart=always
|
|
RestartSec=10
|
|
# ExecStop allows the engine 120 s to drain (`podman stop -t 120`); without
|
|
# an explicit TimeoutStopSec systemd's 90 s default cuts the stop short,
|
|
# kills it, and records 'Failed with result timeout' (measured 2026-10-01
|
|
# 13:27:17). Must exceed the podman grace with margin.
|
|
TimeoutStopSec=180
|
|
# The engine pins tens of GiB of host RAM (expert rows + PLE table); it is
|
|
# the last process on this host that should be chosen by the kernel OOM
|
|
# killer.
|
|
OOMScoreAdjust=-500
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|