frank-primary: radiance native-FP8 TP=2 cutover (engine, payload build, rollback)

Replace the FreeToken PR587 endpoint (single-GPU: its TP=2 dies in the
pynccl.cu ROCm JIT) with the radiance engine on both R9700s, TP=2 via
libr4d's fused one-shot all-reduce. Official Qwen/Qwen3.8-Flash-Next-FP8
payload at native block-FP8 precision:

- scripts/build-frank-primary-rad.sh: staging (quantization_config drop =
  plugin SELECTION law) -> checkpoint reshape -> rad-convert with ONE
  --quant rule (PLE rtn fixed-scale); plan-only first.
- scripts/stack-qwen38-official-fp8.py: the two value-preserving reshapes
  (experts byte-stacked into the plugin's stacked layout; PLE's 128
  scaleless fp8 shards decoded to one bf16 shard with the checkpoint's own
  scale; the rtn rule re-encodes at that same scale - codes round-trip
  exactly and land the one-element scale plane the plugin declares).
- scripts/hfget.sh + verified download (131/131 shards sha256-match).
- froggeric v22.5 chat template vendored (--override-chat-template).
- serve script (pinned ledger: 2x40 GiB pools + 47.7 GiB PLE + prefix host),
  TTM preflight fixed for kernels with pages_limit only (no pages_max -
  an absent knob is skipped, not read as zero); pages_limit raised to
  52,428,800 pages = 200 GiB on the host pre-cutover.
- root unit, relinked; deployer gates image digest + .rad floor 158,000 MiB
  + health; prometheus re-adds the /metrics job.
- rollback: tag frank-primary-freetoken-pr587-20261005 (FreeToken trio +
  Dockerfile); payload + image stay on disk until the window closes.

deploy-remote.sh also carries the parallel session's frank-small payload
gates (swift15) - committed here per Frank's direction to commit the tree.
This commit is contained in:
Frank committed 2026-10-05 23:26:51 -07:00
1 parent 8617b79a18
commit 1919ad7f31
14 files changed
+2108 -504

No files matched your search

+28 -29
View File
@@ -1,26 +1,25 @@
# frank-primary.service — FreeToken (PR #587 ROCm) endpoint (orcarouter
# Qwen3.8-Flash-Next-Uncensored-FP8, 131-shard payload) on ONE R9700
# (gfx1201), dankserver :7770. 2026-10-05 cutover from the tcclaviger vLLM
# fork (git tag frank-primary-vllm-shapeG-20261003 is the pre-cutover
# state; the retired unit/env/serve trio is in that tag).
# frank-primary.service — radiance (StillDeadcode, Apache-2.0) endpoint
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
# frank-primary-freetoken-pr587-20261005 is the pre-cutover state; the
# retired unit/env/serve trio and Dockerfile are in that tag).
#
# Same port, same served alias, same container identity as the unit name;
# the harness models.yml id is unchanged. The serve script drives
# `podman run` directly (the image carries `ft serve`) and then waits on
# the container. The image is built ON THE HOST from
# scripts/docker/frank-primary-freetoken.Dockerfile (FreeToken PR head
# 77a7201 in the PR's own ROCm 7.14 base image).
#
# WHY A ROOT UNIT (not a rootless quadlet): FreeToken pins ~120 GiB of
# expert weights plus the ~47.7 GiB PLE ngram table in HOST RAM. Pinned
# pages are charged against RLIMIT_MEMLOCK; dank's login session on this
# host carries 8 MiB and a user quadlet is capped by the user manager's own
# hard limit — raising that means restarting user@1000.service and bouncing
# every user container on the host. A root unit grants
# LimitMEMLOCK=infinity to the dank process via PID1, and the image stays
# in dank's rootless podman storage.
# `podman run` directly: the digest-pinned stilldeadcode/radiance image
# (287 MiB, kernels prebuilt for gfx1201) has the engine as its
# entrypoint. The image lives in dank's ROOTLESS podman storage.
# WHY A ROOT UNIT (not a rootless quadlet): radiance pins ~132 GiB of HOST
# RAM (2 x 40 GiB expert pools + the 47.7 GiB PLE ngram table + the prefix
# host tier). Pinned pages are charged against RLIMIT_MEMLOCK; dank's login
# session on this host carries 8 MiB and a user quadlet is capped by the
# user manager's own hard limit — raising that means restarting
# user@1000.service and bouncing every user container on the host. A root
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
# engine still runs out of dank's rootless podman storage.
[Unit]
Description=frank-primary — FreeToken (PR587) Qwen3.8-Flash-Next-Uncensored-FP8 on 1x R9700
Description=frank-primary — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
After=network-online.target
Wants=network-online.target
# Never let the start-rate limiter give up after a crash loop (a failed LLM
@@ -30,21 +29,21 @@ StartLimitIntervalSec=0
[Service]
User=dank
Group=dank
# The expert LRU + PLE pin ~170 GiB of host RAM; an RLIMIT_MEMLOCK smaller
# than the pinned set fails the load mid-start. PID1 grants infinity here —
# parity with the fleet's other LLM units.
# The engine pins ~132 GiB of host RAM (expert pools + PLE table); an
# RLIMIT_MEMLOCK smaller than the pinned set fails the load mid-start.
# PID1 grants infinity here — parity with the fleet's other LLM units.
LimitMEMLOCK=infinity
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
# systemd stop: wait alone cannot stop the container child; stop it
# explicitly. ft serve drops connections on SIGTERM; no graceful drain
# contract is claimed (the OpenAI front end holds no queue to drain —
# in-flight decodes are lost, which a stop during a bench window accepts).
# explicitly. No graceful drain contract is claimed (the OpenAI front end
# holds no queue to drain — in-flight decodes are lost, which a stop
# during a bench window accepts; radiance takes SIGTERM on the engine
# process and podman reaps the container).
ExecStop=/usr/bin/podman stop -t 120 frank-primary
# Boot cost: 131-shard FP8 page-in + PLE pin + expert upload. vLLM-era
# budget (1800 s) carried forward; the FreeToken first boot measured here
# once it ran (see the runbook for the real number).
TimeoutStartSec=1800
# Boot cost: .rad load (weights + PLE upload) + expert host-pool pin. The
# publisher's compose healthcheck allows a 15-minute start_period; the
# vLLM-era budget (1800 s) carried forward.
# Restart=always, NOT on-failure (the 2026-10-01 vLLM lesson carries over:
# a clean API-server exit after a worker fault reads as success to
# systemd). `always` restarts on any container death; an explicit