Frank
cd0be97b39
The shape block and boot ledger now quote the live 1.8.1 boot: experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %) = 747,492 tokens, 3.7 full-length sessions, `--checkpoint-slots 40`, `queue_cap 32`, sampler 12 rows, `experts-from-host` 8.99 GiB a rank. The KV Capacity law is rewritten from the pinned source instead of inference: `core/mem/vram_budget.cpp:146-156` funds the cache as `elastic - expert_floor`, `core/mem/kv.cpp:266` sets its ceiling from `max_seqs x max_ctx` (demand, never funding), and `core/place/planner.cpp:100-104` makes the expert floor fall one-for-one with `--host-pool-mib` — which is why +4,224 MiB of pinned host pool bought the pool and cost 3.01 GiB of residency a card. The old "funds ~90 %, never 100 %" phrasing was inference; the byte-identical 45,568 row is the ceiling case, not a mystery. Slot-cost law carries the four-slot price (state + checkpoint store + rows, still under the 25-row exact-wire bound). The two-slot era is one row in Eras with its rollback commit. Date stamps in the trio comments corrected to 2026-10-20.
150 lines
9.8 KiB
Bash
150 lines
9.8 KiB
Bash
# frank-archon.env — radiance (StillDeadcode) on BOTH R9700s (TP=2),
|
|
# Dankserver :7770, publisher's 4-bit-codebook container of the official
|
|
# Qwen3.8-Flash-Next.
|
|
#
|
|
# ONE OWNER: shape, payload identity, eras, laws and every measured number live
|
|
# in serverAppConfigs-docs/LLM/frank-archon.md. This file carries the knobs plus
|
|
# only the reason a reader editing THIS line cannot recover from the page.
|
|
# Where the two disagree, this file wins on values; the boot ledger arbitrates
|
|
# the engine shape.
|
|
FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
|
|
# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
|
|
# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
|
|
# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
|
|
# now split by the engine's budget resolver — see below), and the prefix cache
|
|
# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
|
|
# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
|
|
# suspect is untangled (the page's Dead ends section names it).
|
|
# The .rad file mounts under /models; the tune cache mounts separately below.
|
|
# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
|
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
|
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
|
|
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
|
# state dir's tune/ subdir carries rad-tune's kernel-selection cache; the engine
|
|
# reads $RADIANCE_HOME/tune/<machine>.tune and that defaults to /opt/radiance
|
|
# (inside the image), so the cache only survives on a host mount.
|
|
# Container identity is the SERVICE name: one name across systemctl and
|
|
# `podman ps`; the runtime changes, the name does not. The legacy list retires
|
|
# every predecessor identity that could hold the GPUs or the port.
|
|
FRANK_ARCHON_CONTAINER=frank-archon
|
|
FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
|
|
# Served id. Radiance VALIDATES the requested model name, so every client must
|
|
# send exactly this value (unlike llama.cpp and halogen, which ignore it).
|
|
FRANK_ARCHON_SERVED_NAME=frank-archon
|
|
FRANK_ARCHON_PORT=7770
|
|
# Frank's chat-template call: froggeric's fixed Qwen3.8 template, vendored in
|
|
# the repo; it overrides the template baked into the .rad, and radiance derives
|
|
# the reasoning and tool-call reply format from it.
|
|
FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
|
|
|
|
# --- engine shape (the boot ledger arbitrates every number below) ---------
|
|
# TP=2 needs the 2026-09-20 board (x8/x8 on root ports): a collective cannot be
|
|
# dispatched on a card whose PCIe chain lacks atomics. Radiance's wire is
|
|
# libr4d's own, not RCCL.
|
|
FRANK_ARCHON_TP=2
|
|
# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice). It
|
|
# stays EXACT for single-sequence decode and any step of <= 25 rows — see
|
|
# MAX_SEQS x (1 + SPEC_TOKENS) below.
|
|
FRANK_ARCHON_TP_WIRE=wht6
|
|
# Decode slots. A slot costs KV and pinned RAM, never decode: the cache target is
|
|
# exactly MAX_SEQS x MAX_MODEL_LEN and the checkpoint store asks (1 + what the
|
|
# retention policy keeps) per slot. Ordered 2026-10-20: FOUR slots over a pool of at
|
|
# least 700,000 fp8 tokens, rounding UP if the resolver cannot land it exactly. The
|
|
# target is therefore 4 x 200,000 = 800,000 tokens and HOST_POOL_MIB decides whether
|
|
# the cache gets it (expert_vram_floor, core/place/planner.cpp:100-104: the card must
|
|
# keep expert-plane-minus-host-room). The 2-slot shape was the answer to a
|
|
# 365,204-token pool that backed 1.8 full-height sessions; at this pool the third and
|
|
# fourth conversations are backed by the cache instead of over-committing it.
|
|
FRANK_ARCHON_MAX_SEQS=4
|
|
# Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
|
|
# limit, and models.yml contextWindow MUST quote this number (over-length is a
|
|
# refusal, not a truncation). The pool is sized for seqs x maxlen, not one
|
|
# sequence.
|
|
FRANK_ARCHON_MAX_MODEL_LEN=200000
|
|
FRANK_ARCHON_PLACEMENT=expert_tiered
|
|
# Pinned host pool for expert rows that do not fit VRAM, PINNED ONCE PER RANK: the
|
|
# host must pin tp x this number, and the refusal that names it is
|
|
# core/engine_bringup.cpp:250-265. Under the budget resolver this is ALSO the KV
|
|
# lever -- the card keeps expert-plane-minus-this-room, so every MiB added moves one
|
|
# MiB of experts off the card and into the cache. Derived from the boot ledger (a
|
|
# 23.73 GiB floor at 44,928 room puts plane + 2 units at 67.60 GiB a rank), 49,152
|
|
# drops the floor to 19.6 GiB and leaves the cache ~6.9 GiB, which is enough to back
|
|
# the whole 4 x 200,000 target; the leftover returns to the experts. Cost is paid
|
|
# twice over: pinned RAM (tp x 48 GiB, against the TTM target below) and expert
|
|
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
|
|
# x8 link when the router picks it. The ledger's coverage line is the referee, not
|
|
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
|
|
FRANK_ARCHON_HOST_POOL_MIB=49152
|
|
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
|
# floor is built from it.
|
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
|
# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
|
|
# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
|
|
# puts the static (non-expert) weights off the top, then divides the rest between
|
|
# the expert plane and the KV cache — experts first when the cache has no idle
|
|
# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
|
|
# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
|
|
# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
|
|
# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
|
|
# it after every resize instead of asserting it here.
|
|
# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
|
|
# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
|
|
# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
|
|
FRANK_ARCHON_KV_DTYPE=fp8
|
|
# The checkpoint's own MTP head at depth 2: this fleet's ladder was measured at
|
|
# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
|
|
FRANK_ARCHON_SPEC_TOKENS=2
|
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
|
# Step sharing. A prefill chunk that shares a step with decoding sequences holds
|
|
# the device for the chunk's whole time, and every decode row in that step waits
|
|
# that long for its next token: measured at this shape, decode-only steps 15.5 ms
|
|
# against chunk steps 166 ms, which is the harness's ~146 t/s falling to ~11 t/s
|
|
# while another session prefills (law and census: [[frank-archon]]).
|
|
# `auto` is a PREFILL-SPEED dial, not a decoder dial: it reserves one row plus the
|
|
# draft window per decoding sequence (3 tokens) and an eighth of the budget per
|
|
# waiting request or waiting prefill, and the source states it will not halve for
|
|
# decode rows alone. Live proof: prefill_chunks_capped 5,018 across 62,386 chunk
|
|
# steps. A number is the only value that applies whenever the chunk shares a step,
|
|
# and it must not go under the prefill stager's smallest pass -- 1025 rows here,
|
|
# named on the boot line "prefill staging: ... a step of more than 1024 tokens" --
|
|
# because below it every routed layer reads its pooled experts synchronously, so
|
|
# 512 is a trap and the vendor measures the whole job slower there. Interleave
|
|
# inserts N decode-only steps after a chunk step taken beside decoders: it protects
|
|
# their cadence and holds a waiting request out of those steps. That is the decoder
|
|
# dial. ARMED at 1 on 2026-10-20: at N=1 a stream takes its token from the chunk step
|
|
# AND one from the held step, ~2 tokens per 182 ms instead of one per 166 ms; the
|
|
# price falls on the prefilling prompt (its chunks lose every held step) and on any
|
|
# request still waiting to enter. `--prefill-span` re-funds a prefill only when it is
|
|
# ALONE, so it cannot buy this back while decoders run. Meter:
|
|
# radiance:engine_steps_held_total must leave zero, and the §9 buckets must move
|
|
# before N rises.
|
|
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
|
|
FRANK_ARCHON_PREFILL_INTERLEAVE=1
|
|
# Prefix cache: pinned host memory only. The disk tier was removed outright
|
|
# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
|
|
# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
|
|
# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
|
|
# this change: a drop is extra prefill, and prefill is exactly the traffic that
|
|
# costs decode its cadence.
|
|
FRANK_ARCHON_PREFIX_CACHE_MIB=8192
|
|
# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
|
|
# read. `ram` REFUSES the start if memlock is too small — the unit's
|
|
# LimitMEMLOCK=infinity is the other half of this law.
|
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
|
# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
|
|
# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
|
|
# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
|
|
# itself before refusing by name, so recovery is a boot, not a visit.
|
|
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
|
|
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
|
|
# at the 49,152 pool with 4 slots that computes to 161,848 MiB.
|
|
FRANK_ARCHON_TTM_MIN_MIB=162000
|
|
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
|
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
|
# this unquoted on purpose, so keep multi-flag values as one string.
|
|
FRANK_ARCHON_EXTRA_ARGS=
|
|
# Deployer gate: a truncated fetch must be a fast refusal, not a long-timeout
|
|
# boot. Floor sits under the live payload and under every rollback payload
|
|
# (see the page's Eras table for what is on disk and what needs a re-download).
|
|
FRANK_ARCHON_RAD_MIN_MIB=110000
|