Merge pull request 'fix(frank-archon): size the host pool so the KV cache clears 3 x 200,000' (#30) from fix/frank-archon-cache-ceiling into main
This commit was merged in pull request #30.
This commit is contained in:
@@ -63,7 +63,12 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
|
||||
# rest, so growing the pool moves experts off the card and shrinking it spills
|
||||
# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
|
||||
# refused a boot). The ledger's coverage line is the referee, not this file.
|
||||
FRANK_ARCHON_HOST_POOL_MIB=44928
|
||||
# Sized so the cache clears the 3 x MAX_MODEL_LEN contract: the 1.8.1 boot at
|
||||
# 44,928 resolved the cache to 556,348 tokens — 2.8 sessions — and three
|
||||
# concurrent full-height requests came back 400 in 0.0 s. Tokens scale with the
|
||||
# GiB the cache wins, so +640 MiB a rank buys ~44,000 tokens of ceiling with
|
||||
# ~2.5% off the expert slab.
|
||||
FRANK_ARCHON_HOST_POOL_MIB=45568
|
||||
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
||||
# floor is built from it.
|
||||
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||
@@ -107,11 +112,14 @@ FRANK_ARCHON_PREFIX_CACHE_MIB=8192
|
||||
# read. `ram` REFUSES the start if memlock is too small — the unit's
|
||||
# LimitMEMLOCK=infinity is the other half of this law.
|
||||
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||
# TTM preflight. The pinned ledger (~147 GiB) rides TTM's shared pages, and the
|
||||
# kernel RESETS pages_limit to half of RAM on every boot; below the plan the
|
||||
# load dies mid-pin with an anonymous allocator error. ttm-preflight.sh raises
|
||||
# the knob itself before refusing by name, so recovery is a boot, not a visit.
|
||||
FRANK_ARCHON_TTM_MIN_MIB=150000
|
||||
# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
|
||||
# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
|
||||
# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
|
||||
# itself before refusing by name, so recovery is a boot, not a visit.
|
||||
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
|
||||
# per the bats ledger test; at the 45,568 pool that computes to 150,200 MiB, so
|
||||
# the floor sits just above it.
|
||||
FRANK_ARCHON_TTM_MIN_MIB=152000
|
||||
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
||||
# this unquoted on purpose, so keep multi-flag values as one string.
|
||||
|
||||
@@ -155,7 +155,7 @@ nosudo_bin() {
|
||||
# so the pool's token count is a boot-ledger line — verified live by the
|
||||
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
|
||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=45568$' "$ENV_FILE"
|
||||
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
|
||||
|
||||
Reference in New Issue
Block a user