fix(frank-archon): raise the ttm pages cap in the serve path; reshape to 4 x 200,000 fp8 KV
The 2026-10-12 23:17 power failure reset /sys/module/ttm/parameters/pages_limit from the manual 2026-10-05 raise (200 GiB) back to the kernel default (half of RAM = 125,988 MiB), below the pinned-RAM ledger floor of 150,000 MiB. The serve script's preflight named the refusal and exited 1; Restart=always re-ran it 6,919 times over ~20 hours and frank-archon served nobody. scripts/ttm-preflight.sh (sourced by the serve script) now RAISES the knob itself before refusing: the unit is root, the knob is a 0644 root module parameter, and the cap is a runtime knob whose default is half of RAM — so a manual sysfs write is not a repair procedure, it is a countdown to the next reboot. FRANK_ARCHON_TTM_TARGET_MIB=204800 records the host ceiling; the gate re-reads the cap after writing and only then refuses, so a knob the kernel will not move is still a named failure. Shape change (Frank's call): FOUR decode slots x 200,000 tokens with fp8 (quantized) KV. The pool must back seqs x max-model-len, so fp8 is what funds the fourth slot: at bf16 the four slots cost 11,679 MiB of pool and would cut the resident expert slab to 17,441 MiB (-33 %); at fp8 they cost 6,601 MiB, so the pool stays 6,912 MiB (predicts 843,566 tokens), the VRAM pair stays at the measured 29,120 MiB/card ceiling, the weights slab and the 44 GiB host pool are untouched, and the fused TP all-reduce rides 4 x (1+2) = 12 rows, inside the 25-row bound where the wht6 wire is bit-exact.
This commit is contained in:
+56
-39
@@ -3,10 +3,13 @@
|
|||||||
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
|
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
|
||||||
# dankserver :7770.
|
# dankserver :7770.
|
||||||
#
|
#
|
||||||
# SHAPE (2026-10-11): 2 decode slots x 200,000 tokens with an UNQUANTIZED
|
# SHAPE (2026-10-13): FOUR decode slots x 200,000 tokens with an fp8
|
||||||
# (bf16) KV cache — the pool is sized for the pair, and the weights slab pays
|
# (quantized) KV cache. fp8 is what makes four slots affordable: the pool must
|
||||||
# for it (see the KV block). The fp8-native payload this file was written for
|
# back seqs x max-model-len, and at bf16 that costs 11,679 MiB (weights would
|
||||||
# is the rollback target and keeps its own trio under the fp8-native runbook.
|
# fall to 17,441 MiB, -33 %), at fp8 only 6,601 MiB — so the slab stays at its
|
||||||
|
# 2026-10-11 size. See the KV block for the arithmetic and the measured dtype
|
||||||
|
# law. The fp8-native payload this file was written for is the rollback target
|
||||||
|
# and keeps its own trio under the fp8-native runbook.
|
||||||
#
|
#
|
||||||
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
||||||
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
@@ -113,18 +116,21 @@ FRANK_ARCHON_TP=2
|
|||||||
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
||||||
# engine default.
|
# engine default.
|
||||||
FRANK_ARCHON_TP_WIRE=wht6
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
|
# Decode slots: FOUR, so the served window is a CAPACITY CONTRACT — four
|
||||||
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
||||||
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
||||||
# tcclaviger vLLM fork's torch.compile shape specialization on
|
# tcclaviger vLLM fork's torch.compile shape specialization on
|
||||||
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
||||||
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
||||||
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
||||||
# (30.9-33.1 t/s, inside the +-3% noise floor). Side benefit: the fused TP
|
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
|
||||||
# all-reduce is declared over seqs x (1 + spec) = 6 rows, under the 25-row
|
# not speed, so the price of the fourth slot is KV, not decode.
|
||||||
# bound at which the wht6 wire stays EXACT (single-sequence decode and any
|
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
|
||||||
# step of <= 25 rows is bit-exact; only busier steps are lossy).
|
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
|
||||||
FRANK_ARCHON_MAX_SEQS=2
|
# 4 x 3 = 12 here, still under the 25-row bound at which the wht6 wire stays
|
||||||
|
# EXACT (single-sequence decode and any step of <= 25 rows is bit-exact; only
|
||||||
|
# busier steps are lossy).
|
||||||
|
FRANK_ARCHON_MAX_SEQS=4
|
||||||
# Served window. Native is 262,144; 200,000 is the publisher's production
|
# Served window. Native is 262,144; 200,000 is the publisher's production
|
||||||
# value (compose) and what models.yml's contextWindow must quote — a longer
|
# value (compose) and what models.yml's contextWindow must quote — a longer
|
||||||
# prompt is refused by the engine, never truncated. The engine treats
|
# prompt is refused by the engine, never truncated. The engine treats
|
||||||
@@ -156,14 +162,16 @@ FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
|||||||
# so a bigger KV pool steals weight residency from the slab AND from the
|
# so a bigger KV pool steals weight residency from the slab AND from the
|
||||||
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
|
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
|
||||||
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
|
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
|
||||||
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. Expect -3..-8%
|
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
|
||||||
# vs the fp8 control (136 t/s median at 44.5k depth, acceptance 0.7246,
|
# exact-KV A/B measured the price of this exact slab size (22208): prefill
|
||||||
# ~190 t/s on a live 512-token probe); re-tune from a boot ledger + probe,
|
# -29 %, TTFT at the full window +25 %, decode inside the noise band on 2 of 3
|
||||||
# never from that estimate.
|
# passes, draft acceptance UP 0.661 -> 0.740. Since 2026-10-13 the pool is fp8
|
||||||
|
# (see the KV block), which is what funds the FOURTH slot at this slab size;
|
||||||
|
# re-tune from a boot ledger + probe, never from that estimate.
|
||||||
FRANK_ARCHON_VRAM_WEIGHTS_MIB=22208
|
FRANK_ARCHON_VRAM_WEIGHTS_MIB=22208
|
||||||
FRANK_ARCHON_VRAM_KV_MIB=6912
|
FRANK_ARCHON_VRAM_KV_MIB=6912
|
||||||
# KV cache UNQUANTIZED (bf16). This engine's flag takes ONLY bf16|fp8 — there
|
# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
|
||||||
# is ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
|
# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
|
||||||
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
|
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
|
||||||
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
|
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
|
||||||
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
|
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
|
||||||
@@ -177,30 +185,33 @@ FRANK_ARCHON_VRAM_KV_MIB=6912
|
|||||||
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
|
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
|
||||||
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
|
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
|
||||||
# checkpoint region 8 x 55.99 = 448 MiB. So:
|
# checkpoint region 8 x 55.99 = 448 MiB. So:
|
||||||
# pool_mib >= tokens x 13.8125 KiB + 110 x seqs + 448
|
# pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
|
||||||
# 6912 -> (6912 - 220 - 448) x 1024 / 13.8125 = 462,946 tokens; the engine
|
# 4 x 200,000 at fp8 needs 5,713 + 440 + 448 = 6,601 MiB; the pool is 6,912,
|
||||||
# reported 462,752 (0.04% page rounding) — 2 x 200,000 with 62k of slack,
|
# and it predicts (6912 - 440 - 448) x 1024 / 7.3125 = 843,566 tokens, i.e.
|
||||||
# and the ledger says it outright: "backs 2 sequences all at 200000 tokens
|
# FOUR full-height conversations with 5.4 % slack. The ledger line is the
|
||||||
# at once, which is the worst case".
|
# referee ("backs N sequences all at 200000 tokens at once").
|
||||||
# (the fp8 pool of 3543 gave 386,904 — it never held 2 x 200k, it held 1.93
|
# At bf16 the same four slots cost 10,791 + 888 = 11,679 MiB, which would
|
||||||
# sessions, and a third long conversation preempted to the host/disk tiers).
|
# cut the resident slab to 17,441 MiB (-33 % vs the fp8-era 26,071) — that
|
||||||
# Secondary cost: the prefix-cache entry grows with the attention plane
|
# is why fp8 is not a preference here, it is the only dtype that funds four
|
||||||
# (one entry 58.50 KiB at fp8 -> 110.50 KiB at bf16, measured on the two boot
|
# slots at this window. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed
|
||||||
# ledgers' `kv tier transfer` lines), so the host tier below doubles to keep
|
# 462,752 tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904
|
||||||
# the same number of parked conversations, out of RAM the host has
|
# (1.93 sessions).
|
||||||
# (104 GiB available with the service loaded).
|
# Secondary saving: the prefix-cache entry shrinks with the attention plane
|
||||||
FRANK_ARCHON_KV_DTYPE=bf16
|
# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
|
||||||
|
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
|
||||||
|
# many conversations as it did at bf16 — no re-size needed to park four slots.
|
||||||
|
FRANK_ARCHON_KV_DTYPE=fp8
|
||||||
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
||||||
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
||||||
# flag states it so the ledger records the intent).
|
# flag states it so the ledger records the intent).
|
||||||
FRANK_ARCHON_SPEC_TOKENS=2
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
|
||||||
# The host tier is sized in ENTRIES, and one entry doubles with the cache
|
# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
|
||||||
# dtype (58.50 KiB at fp8 -> 117 KiB at bf16, per the ledger's `kv tier
|
# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
|
||||||
# transfer` line): 4096 MiB at bf16 parks ~1.4 sessions of 200k where it
|
# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
|
||||||
# parked ~2.9 at fp8. Doubled to keep the capacity, at RAM the host has
|
# parked set fits with the tier as it stands — at RAM the host has (104 GiB
|
||||||
# (104 GiB available with the service loaded).
|
# available with the service loaded).
|
||||||
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
||||||
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
||||||
@@ -211,11 +222,17 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
|||||||
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
||||||
# takes the min over the knobs that exist). Below the plan the load dies
|
# takes the min over the knobs that exist). Below the plan the load dies
|
||||||
# mid-pin with an allocator error instead of this named refusal — raise the
|
# mid-pin with an allocator error instead of this named refusal.
|
||||||
# knob first (runtime sysfs write; kernel cmdline fallback + one reboot only
|
# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
|
||||||
# if the file is read-only), see the runbook. Cap raised 2026-10-05 to
|
# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
|
||||||
# 52,428,800 pages = 200 GiB, which still clears this floor.
|
# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
|
||||||
|
# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
|
||||||
|
# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
|
||||||
|
# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
|
||||||
|
# root unit is privileged; the knob is a 0644 root module parameter), so the
|
||||||
|
# recovery is a boot, not a visit to the machine.
|
||||||
FRANK_ARCHON_TTM_MIN_MIB=150000
|
FRANK_ARCHON_TTM_MIN_MIB=150000
|
||||||
|
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
||||||
# multi-flag values as one string; the serve script expands it unquoted on
|
# multi-flag values as one string; the serve script expands it unquoted on
|
||||||
# purpose.
|
# purpose.
|
||||||
|
|||||||
@@ -13,10 +13,11 @@
|
|||||||
# log states every budget before anything is allocated (weights vs KV
|
# log states every budget before anything is allocated (weights vs KV
|
||||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||||
# the P2P answer per pair) — read it after every resize.
|
# the P2P answer per pair) — read it after every resize.
|
||||||
# Deployed shape (2026-10-11): 2 decode slots x 200,000 tokens, bf16
|
# Deployed shape (2026-10-13): FOUR decode slots x 200,000 tokens, fp8
|
||||||
# (unquantized) KV in a 6,912 MiB pool, the weights slab cut to 22,208 MiB to
|
# (quantized) KV in a 6,912 MiB pool — the dtype that funds four slots
|
||||||
# pay for it, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
|
# without cutting the weights slab below its 22,208 MiB size, 44 GiB/rank
|
||||||
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
|
# pinned expert pool, MTP depth 2, prefix cache host 8 GiB + disk 64 GiB, PLE
|
||||||
|
# pinned in RAM.
|
||||||
#
|
#
|
||||||
# The root unit (scripts/frank-archon.service, User=dank,
|
# The root unit (scripts/frank-archon.service, User=dank,
|
||||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
||||||
@@ -61,6 +62,7 @@ set +a
|
|||||||
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
||||||
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
||||||
|
:"${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
||||||
|
|
||||||
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
||||||
@@ -117,34 +119,19 @@ if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
# TTM preflight: the pinned ledger (2 x FRANK_ARCHON_HOST_POOL_MIB + the
|
||||||
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
# 47.7 GiB PLE table + the prefix host tier, ~147 GiB) rides TTM's shared
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
# pages, capped by ttm's pages_limit (the current kernel exposes NO
|
||||||
# takes the min over the knobs that exist). Below FRANK_ARCHON_TTM_MIN_MIB
|
# pages_max; the cap function takes the min over the knobs that exist).
|
||||||
# the load dies mid-pin with an allocator error instead of this named
|
# Below FRANK_ARCHON_TTM_MIN_MIB the load dies mid-pin with an allocator
|
||||||
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
# error, so the gate refuses first — and RAISES the knob before refusing:
|
||||||
# fallback + one reboot only if the file is read-only), see the runbook.
|
# the cap is a runtime knob whose default is half of RAM, so a power failure
|
||||||
# RAISED 2026-10-05 pre-cutover to 52,428,800 pages = 200 GiB.
|
# resets it (measured 2026-10-12 23:17 on this host: 200 GiB -> 125,988 MiB,
|
||||||
ttm_cap_mib() {
|
# and Restart=always re-ran the refusal 6,919 times over ~20 h).
|
||||||
# min over the knobs THIS kernel exposes — the current kernel has
|
# FRANK_ARCHON_TTM_TARGET_MIB is the recorded host ceiling, not the floor.
|
||||||
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
# shellcheck source=/dev/null
|
||||||
# not a zero (a zero would refuse a healthy cap).
|
source "$SCRIPT_DIR/ttm-preflight.sh"
|
||||||
local v cur=""
|
ttm_ensure_cap || exit 1
|
||||||
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
|
|
||||||
[ -r "$p" ] || continue
|
|
||||||
v="$(cat "$p" 2>/dev/null)" || continue
|
|
||||||
[ -n "$v" ] || continue
|
|
||||||
v=$(( v * 4 / 1024 ))
|
|
||||||
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
|
||||||
done
|
|
||||||
echo "${cur:-0}"
|
|
||||||
}
|
|
||||||
TTM_CAP="$(ttm_cap_mib)"
|
|
||||||
if [ "$TTM_CAP" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
|
||||||
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
|
||||||
echo "frank-archon: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_ARCHON_TTM_MIN_MIB in 4 KiB pages" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# A crashed/killed unit can leave the container name registered (systemd
|
# A crashed/killed unit can leave the container name registered (systemd
|
||||||
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
||||||
|
|||||||
Executable
+76
@@ -0,0 +1,76 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# ttm-preflight.sh — the TTM pages-cap gate for engines that PIN host RAM.
|
||||||
|
#
|
||||||
|
# Sourced, never executed. An engine that pins tens of GiB of host RAM (the
|
||||||
|
# radiance expert pools + the PLE n-gram table) has its pinned pages charged
|
||||||
|
# against the kernel's ttm page cap: /sys/module/ttm/parameters/pages_limit
|
||||||
|
# (and pages_max on kernels that expose one). The cap function below takes the
|
||||||
|
# MIN over the knobs the running kernel actually exposes — an absent knob is
|
||||||
|
# skipped, never read as zero, because a zero would refuse a healthy cap.
|
||||||
|
#
|
||||||
|
# THE LAW THIS FILE EXISTS FOR: the cap is a RUNTIME knob. The kernel default is
|
||||||
|
# half of RAM, so every raise is lost on reboot. On 2026-10-12 23:17 a power
|
||||||
|
# failure on dankserver reset it 200 GiB -> 125,988 MiB, and frank-archon's
|
||||||
|
# preflight refused the start — 6,919 crash-restarts over ~20 hours, with
|
||||||
|
# `Restart=always` cheerfully re-running a refusal. A manual sysfs write is
|
||||||
|
# therefore not an acceptable repair procedure: the gate raises the knob itself
|
||||||
|
# (the units that need it run as root, and the knob is a 0644 root-owned module
|
||||||
|
# parameter), re-reads, and only then refuses.
|
||||||
|
#
|
||||||
|
# Contract (set by the sourcing script's env file):
|
||||||
|
# FRANK_ARCHON_TTM_MIN_MIB floor the pinned ledger needs; refuse below it
|
||||||
|
# FRANK_ARCHON_TTM_TARGET_MIB value to raise the cap to when it is below the
|
||||||
|
# floor (the recorded host ceiling — writing the
|
||||||
|
# floor itself would leave the ledger no slack)
|
||||||
|
# FRANK_ARCHON_TTM_SYS_DIR knob directory; defaults to the real sysfs
|
||||||
|
# path, a test points it at a temp dir.
|
||||||
|
# Pages are 4 KiB: pages = MiB x 256.
|
||||||
|
|
||||||
|
: "${FRANK_ARCHON_TTM_SYS_DIR:=/sys/module/ttm/parameters}"
|
||||||
|
|
||||||
|
# Prints the effective cap in MiB, 0 when no knob is readable.
|
||||||
|
ttm_cap_mib() {
|
||||||
|
local p v cur=""
|
||||||
|
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
|
||||||
|
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
|
||||||
|
[ -r "$p" ] || continue
|
||||||
|
v="$(cat "$p" 2>/dev/null)" || continue
|
||||||
|
[ -n "$v" ] || continue
|
||||||
|
v=$(( v * 4 / 1024 ))
|
||||||
|
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
||||||
|
done
|
||||||
|
echo "${cur:-0}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Writes $1 MiB (as 4 KiB pages) into every writable knob present. Failures are
|
||||||
|
# swallowed: the caller re-reads the cap, and a write the kernel refused shows
|
||||||
|
# up there (ttm refuses a limit below the pages it has already pinned).
|
||||||
|
ttm_raise_cap() {
|
||||||
|
local pages=$(( $1 * 256 )) p
|
||||||
|
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
|
||||||
|
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
|
||||||
|
if [ -w "$p" ]; then
|
||||||
|
echo "$pages" > "$p" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# Returns 0 when the cap clears FRANK_ARCHON_TTM_MIN_MIB, raising it first if
|
||||||
|
# needed; returns 1 with a named refusal when it cannot be made to.
|
||||||
|
ttm_ensure_cap() {
|
||||||
|
local cap
|
||||||
|
cap="$(ttm_cap_mib)"
|
||||||
|
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
|
echo "frank-archon: ttm pages cap is ${cap} MiB, below the pinned-ledger" \
|
||||||
|
"floor $FRANK_ARCHON_TTM_MIN_MIB — raising the knob to" \
|
||||||
|
"$FRANK_ARCHON_TTM_TARGET_MIB MiB ($(( FRANK_ARCHON_TTM_TARGET_MIB * 256 )) pages)" >&2
|
||||||
|
ttm_raise_cap "$FRANK_ARCHON_TTM_TARGET_MIB"
|
||||||
|
cap="$(ttm_cap_mib)"
|
||||||
|
fi
|
||||||
|
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
|
echo "ERROR: ttm pages cap is ${cap} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
|
echo "frank-archon: raise $FRANK_ARCHON_TTM_SYS_DIR/pages_limit (and pages_max if the kernel exposes one) to >= $(( FRANK_ARCHON_TTM_MIN_MIB * 256 )) in 4 KiB pages; a read-only knob needs the module option (options ttm pages_limit=N) and a reboot" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
echo "frank-archon: ttm pages cap ${cap} MiB clears the pinned-ledger floor $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
|
}
|
||||||
@@ -9,8 +9,9 @@
|
|||||||
# flags alone do not say: the served window clears the harness worst case,
|
# flags alone do not say: the served window clears the harness worst case,
|
||||||
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
||||||
# engine HARD limit, not a ratio), the per-card VRAM pair stays inside the
|
# engine HARD limit, not a ratio), the per-card VRAM pair stays inside the
|
||||||
# measured card ceiling, the pinned-host ledger has a ttm pages cap floor and
|
# the pinned-host ledger has a ttm pages cap floor, the serve script ENFORCES
|
||||||
# the serve script ENFORCES it, the served window is what models.yml must
|
# it and RAISES the knob itself when a reboot reset it (the cap's default is
|
||||||
|
# half of RAM), the served window is what models.yml must
|
||||||
# quote, the host-networking contract (no --publish), the image gate on a
|
# quote, the host-networking contract (no --publish), the image gate on a
|
||||||
# name@sha256 ref, the .rad payload gate with its size floor, the
|
# name@sha256 ref, the .rad payload gate with its size floor, the
|
||||||
# chat-template override mount, the self-healing law (the run returns before
|
# chat-template override mount, the self-healing law (the run returns before
|
||||||
@@ -24,6 +25,7 @@ load helpers
|
|||||||
|
|
||||||
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
||||||
SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
|
SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
|
||||||
|
TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh"
|
||||||
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
||||||
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||||
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
|
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
|
||||||
@@ -34,6 +36,7 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
||||||
[ -f "$UNIT" ]
|
[ -f "$UNIT" ]
|
||||||
[ -x "$SERVE" ]
|
[ -x "$SERVE" ]
|
||||||
|
[ -f "$TTM_LIB" ]
|
||||||
[ -f "$ENV_FILE" ]
|
[ -f "$ENV_FILE" ]
|
||||||
[ -f "$DOWNLOAD" ]
|
[ -f "$DOWNLOAD" ]
|
||||||
[ -x "$BUILD" ]
|
[ -x "$BUILD" ]
|
||||||
@@ -120,37 +123,40 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
|
||||||
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
|
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
|
||||||
[ "$ml" -ge 177356 ]
|
[ "$ml" -ge 177356 ]
|
||||||
# Concurrency: 2 slots x the served window is the capacity contract, and
|
# Concurrency: 4 slots x the served window is the capacity contract, and
|
||||||
# the fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
|
# the fused TP all-reduce is declared over seqs x (1 + spec) = 12 rows,
|
||||||
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
||||||
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
||||||
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
||||||
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
||||||
# tied inside the +-3% noise floor.
|
# tied inside the +-3% noise floor — slots buy concurrency, not speed.
|
||||||
grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_SEQS=4$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
|
||||||
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
||||||
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
||||||
# KV pool moves the pool with it. KV is UNQUANTIZED bf16 (the engine takes
|
# KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
|
||||||
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
||||||
# in host RAM per Frank's law.
|
# in host RAM per Frank's law.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_KV_DTYPE=bf16$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
||||||
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
|
||||||
# The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
|
# The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
|
||||||
# under the ttm pages cap, and the env states the floor the serve
|
# under the ttm pages cap, and the env states the floor the serve
|
||||||
# script enforces (>= 140,000 MiB — the current cap ~123 GiB fails it,
|
# script enforces (>= 140,000 MiB — the kernel's own default cap, half of
|
||||||
# which is the point).
|
# RAM, fails it, which is the point) plus the value it raises the knob to
|
||||||
|
# when a reboot reset it (the target must clear the floor with slack).
|
||||||
grep -q '^FRANK_ARCHON_TTM_MIN_MIB=' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TTM_MIN_MIB=' "$ENV_FILE"
|
||||||
ttmm=$(awk -F= '/^FRANK_ARCHON_TTM_MIN_MIB=/{print $2}' "$ENV_FILE")
|
ttmm=$(awk -F= '/^FRANK_ARCHON_TTM_MIN_MIB=/{print $2}' "$ENV_FILE")
|
||||||
|
ttmt=$(awk -F= '/^FRANK_ARCHON_TTM_TARGET_MIB=/{print $2}' "$ENV_FILE")
|
||||||
|
[ "$ttmt" -gt "$ttmm" ]
|
||||||
pool=$(awk -F= '/^FRANK_ARCHON_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
|
pool=$(awk -F= '/^FRANK_ARCHON_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
|
||||||
ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
|
ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
|
||||||
# 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
|
# 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
|
||||||
@@ -272,6 +278,57 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
grep -q -- '--device /dev/dri' "$SERVE"
|
grep -q -- '--device /dev/dri' "$SERVE"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "the ttm gate raises a reset pages cap and refuses when the write fails" {
|
||||||
|
# The cap is a RUNTIME knob: the kernel default is half of RAM, so every
|
||||||
|
# reboot resets it (measured: the 2026-10-12 23:17 power failure on
|
||||||
|
# dankserver took it 200 GiB -> 125,988 MiB, and the preflight refusal was
|
||||||
|
# re-run 6,919 times by Restart=always over ~20 h). Behaviour required of
|
||||||
|
# the gate, in both directions:
|
||||||
|
# knob below the floor + writable -> cap ends at the target, start passes
|
||||||
|
# knob below the floor + unwritable -> cap stays low, start REFUSES by name
|
||||||
|
local dir="$BATS_TEST_TMPDIR/ttm"
|
||||||
|
mkdir -p "$dir"
|
||||||
|
export FRANK_ARCHON_TTM_SYS_DIR="$dir"
|
||||||
|
export FRANK_ARCHON_TTM_MIN_MIB=150000 FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
|
|
||||||
|
echo 32253039 > "$dir/pages_limit" # 125,988 MiB: the kernel default here
|
||||||
|
run bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$(cat "$dir/pages_limit")" = "52428800" ] # 204,800 MiB in 4 KiB pages
|
||||||
|
|
||||||
|
echo 32253039 > "$dir/pages_limit"
|
||||||
|
chmod 444 "$dir/pages_limit" # the write refused: cap cannot move
|
||||||
|
run bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"ERROR: ttm pages cap is 125988 MiB"* ]]
|
||||||
|
[ "$(cat "$dir/pages_limit")" = "32253039" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the ttm cap counts only the knobs the kernel exposes" {
|
||||||
|
# An absent knob is not a zero: a zero-cap reading would refuse a healthy
|
||||||
|
# host. The effective cap is the MIN over the knobs that exist; pages_max,
|
||||||
|
# when a kernel exposes one, is the binding one.
|
||||||
|
local dir="$BATS_TEST_TMPDIR/ttm2"
|
||||||
|
mkdir -p "$dir"
|
||||||
|
export FRANK_ARCHON_TTM_SYS_DIR="$dir"
|
||||||
|
export FRANK_ARCHON_TTM_MIN_MIB=150000 FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
|
|
||||||
|
echo 51200000 > "$dir/pages_limit" # 200,000 MiB, no pages_max knob
|
||||||
|
run bash -c '. "$1"; echo "$(ttm_cap_mib)"' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$output" = "200000" ]
|
||||||
|
|
||||||
|
echo 30000000 > "$dir/pages_max" # 117,187 MiB: now the binding knob
|
||||||
|
run bash -c '. "$1"; echo "$(ttm_cap_mib)"' _ "$TTM_LIB"
|
||||||
|
[ "$output" = "117187" ]
|
||||||
|
|
||||||
|
chmod 444 "$dir/pages_limit" "$dir/pages_max" # both writes refused
|
||||||
|
run bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"ERROR: ttm pages cap is 117187 MiB"* ]]
|
||||||
|
[ "$(cat "$dir/pages_max")" = "30000000" ]
|
||||||
|
}
|
||||||
|
|
||||||
@test "the serve script clears stale containers and self-heals via podman wait" {
|
@test "the serve script clears stale containers and self-heals via podman wait" {
|
||||||
# A crashed unit can leave the container name registered forever after;
|
# A crashed unit can leave the container name registered forever after;
|
||||||
# the legacy identities hold the same GPUs and die here too. Then block
|
# the legacy identities hold the same GPUs and die here too. Then block
|
||||||
|
|||||||
Reference in New Issue
Block a user