frank-primary: pin radiance v1.2.0, translate expert-vs-cache-ratio to vram knobs
Engine v1.2.0 (Codeberg e95d397, Docker Hub tag 1.2.0, index digest 665775ed...) removes --expert-vs-cache-ratio. The explicit --vram-weights-mib 26071 / --vram-kv-mib 3543 pair transcribes the live 1.0.13 boot ledger at ratio 0.82 (weights 25.46 GiB, KV 3.46 GiB per card); bracket arms scale KV linearly in (1-r): 0.90 -> 1968, 0.95 -> 984. Serve script, autoresearch harness, and bats gates follow; the bats shape asserts are realigned to the deployed seqs=3/spec=2 sweep result (95923d3,df6b5bb) they failed to track.
This commit is contained in:
1 parent
9aee00f2b3
commit
0258ae0897
4 files changed
+34
-14
No files matched your search
@@ -50,8 +50,16 @@
|
||||
# publisher's throughput peak for this model. Stated explicitly with
|
||||
# --num-speculative-tokens 3.
|
||||
# - The fused TP all-reduce is declared over max-num-seqs x
|
||||
# (1 + --num-speculative-tokens) rows; 2 x 4 = 8 rows fits either wire.
|
||||
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
# (1 + --num-speculative-tokens) rows; 3 x 3 = 9 rows fits either wire.
|
||||
# Engine pin: Codeberg tag v1.2.0 (commit e95d397), Docker Hub tag 1.2.0
|
||||
# (no `v` prefix on tags), index digest below, pushed 2026-10-06. 1.2.0
|
||||
# REMOVED --expert-vs-cache-ratio in favour of the explicit
|
||||
# --vram-weights-mib / --vram-kv-mib pair (core/config.cpp), and added
|
||||
# MXFP4 dense kernels, --p2p host twins, a stop-string fix, a DFlash2
|
||||
# fix, and a MoE weight-rotation change (92b8aaf). Prior pin
|
||||
# sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
|
||||
# = tag 1.0.13, the rollback engine.
|
||||
FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa
|
||||
# The .rad container FILE (mounted under /models) and the state dir for the
|
||||
# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
|
||||
# .rad files; the raw HF download and its staging dir live beside it and
|
||||
@@ -106,9 +114,15 @@ FRANK_PRIMARY_PLACEMENT=expert_tiered
|
||||
FRANK_PRIMARY_HOST_POOL_MIB=40960
|
||||
# VRAM left unclaimed per card (publisher default 96 MiB).
|
||||
FRANK_PRIMARY_GPU_HEADROOM_MIB=96
|
||||
# Weights share of claimed VRAM (publisher production value; higher = more
|
||||
# prefix-cache hit rate, less KV headroom).
|
||||
FRANK_PRIMARY_EXPERT_VS_CACHE=0.82
|
||||
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
||||
# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace
|
||||
# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82:
|
||||
# vram_weights 25.46 GiB, vram_kv 3.46 GiB per card -> 26071 / 3543.
|
||||
# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor
|
||||
# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The
|
||||
# boot ledger arbitrates; re-derive after any payload change.
|
||||
FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071
|
||||
FRANK_PRIMARY_VRAM_KV_MIB=3543
|
||||
# KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers,
|
||||
# 2 KV heads, 256 head_dim, K+V).
|
||||
FRANK_PRIMARY_KV_DTYPE=fp8
|
||||
|
||||
Reference in new issue
Block a user