# frank-primary.env — tcclaviger RDNA4 vLLM production config (dankserver :7770).
#
# 2026-10-02 shape D: fp8 KV, MAXLEN 200704 x SEQS 3, the KV pool cut to ~2
# full-length contexts. The window shrinks from the native 262144 to 200704 and
# the pool from 6.1e9 to 3.15e9; --max-num-seqs stays 3 because SEQS=2 CANNOT
# boot this model (measured 2x2 below). Same digest-pinned tcclaviger RDNA4
# fork. Rollback for the fp8+MTP4 era is git tag frank-primary-fp8-mtp-20261001.
#
# HIP reads bare env vars TEMP, HOST, GPU_MAX_HEAP_SIZE — every key here is
# prefixed FRANK_PRIMARY_ so none can leak.
#

# --- runtime ----------------------------------------------------------------
# Digest-pinned: the floating :latest tag moved once already. Local alias
# qwen-fp8-runtime:tuned-20260929 (image ID 3322dbbbc96d) points at the same
# bytes in rootless storage.
FRANK_PRIMARY_IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea

# --- payload ----------------------------------------------------------------
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8
FRANK_PRIMARY_MODEL_SHARDS=131

# --- identity ---------------------------------------------------------------
# The harness models.yml id stays frank-primary (provider 192.168.0.235:7770).
FRANK_PRIMARY_CONTAINER=frank-primary-qwen
FRANK_PRIMARY_SERVED_NAME=frank-primary
FRANK_PRIMARY_PUBLISH=7770:7770
# Containers removed before every start: the retired R9V identity and the ad
# hoc experiment identity (both hold the GPUs / name registrations).
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"

# --- qualified engine shape (2026-10-02, shape D) ---------------------------
# WHAT THE SLOT COUNT DOES AND DOES NOT DO. The endpoint advertises 3 slots
# (SEQS 3), but the POOL, not the slot count, decides how many full-length
# conversations are resident. MEASURED LIVE 2026-10-02 at this shape:
#   GPU KV cache size: 398,658 tokens,
#   Maximum concurrency for 200,704 tokens per request: 1.99x
# (3,146,129,408 B reserved — 7,892 B/token). The per-token price is
# SHAPE-DEPENDENT, not a constant: the 262144 shape priced at 7,701 B/token
# (6,100,000,000 B -> 792,069 tokens), ~2.5% cheaper, so never extrapolate a
# concurrency figure across geometries. Shorter requests use the spare blocks,
# and a third LONG request queues rather than being refused. Verified: three
# concurrent 67-token requests answered in parallel, ~4.1 s each.
#
# WHY NOT --max-num-seqs 2 (measured 2026-10-02, single-variable boot probes
# against the then-deployed known-good shape, production stopped for each):
#   SEQS 2 + MNBT 1024 -> Engine core initialization failed
#   SEQS 2 + MNBT 2048 -> Engine core initialization failed
#   SEQS 3 + MNBT 1024 -> healthy (792,069 tokens, 3.02x, arena 6,248)
#   SEQS 3 + MNBT 2048 -> healthy (the deployed incumbent)
# The failure is torch.fx ConstraintViolationError: query_start_loc.size()[0]
# "marked dynamic but your code specialized it to be a constant (3)", raised
# from AOT build_guards(strict_error=True) during profile_run -> _dummy_run.
# model_runner.py sizes that input buffer max_num_reqs + 1 and _dummy_run sets
# num_reqs = min(max_num_batched_tokens, max_num_reqs), so the width is
# SEQS + 1 for ANY MNBT >= SEQS: 3 at SEQS 2, 4 at SEQS 3. MNBT only moves the
# compile RANGE ((1,1024) vs (1,2048)) and cannot dodge it — the 2x2 above is
# why MNBT is not a lever here. --enforce-eager bypasses the path but loses
# CUDA graphs. Same failure class killed the 2026-10-01 bf16 2 x 262144 shape.
#
# THE ARENA LEVER IS max_model_len, NOT KV_BYTES (measured 2026-10-02). The
# planner sizes expert cache as budget - non-expert - KV reserve - graphs -
# overhead, and the KV reserve is registry-priced at 9190 B/token x max_model_len:
#   MAXLEN 262144 -> reserve 2.48 GiB -> arena 14.30 GiB / 6,248 slots
#   MAXLEN 200704 -> reserve 1.91 GiB -> arena 14.87 GiB / 6,496 slots
# d(reserve) 0.57 GiB = d(arena) 0.57 GiB = +248 slots. The 6.1e9 -> 3.15e9
# pool cut buys ZERO arena; it buys runtime HEADROOM (the shape it replaced
# died with 0 bytes free on a real 995-token request). To grow the arena,
# lower MAXLEN — not KV_BYTES.
#
# GATHER 32x64 stays: R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64
# measured +18.7% decode vs default geometry and the win grows with depth.
# Do not drop it.
# MNBT 1024 is Frank's lock: MNBT 512 doubles TTFT at 4064 ctx (2.46 s vs
# 1.36 s); 1024-4096 are interchangeable on decode; 12288 costs 2.5% and
# arena. 1024 keeps the workspace floor so the arena stays max. (Side note
# measured 2026-10-02: MNBT 2048 compiles the first graph in 26 s vs 172 s at
# 1024 — a startup-cost lever, not a correctness one.)
# PREFETCH IS ABSENT BY LAW: expert-cache telemetry measured the prefetcher at
# 36% precision (36,720 slabs prefetched, 13,369 served) — roughly half the
# gather traffic was waste. Never re-add --expert-prefetch without a fresh A/B.
# TunableOp stays OFF: CLAV_TUNABLEOP_SWEEP silently skips under expert offload
# unless CLAV_TUNABLEOP_WITH_OFFLOAD=1, and forcing it corrupts numerics.
# --expert-precise-memory stays DROPPED: it crashed the 2 x 262144 bf16 boot
# (0.13 GiB free of 31.86 GiB, NRestarts=3 at ~6.7 min each). It survives in
# the serve script as an opt-in; re-enable only with a pool/context reduction
# that pays for the 2.35 GiB it moves into the arena.
FRANK_PRIMARY_MAXLEN=200704
FRANK_PRIMARY_SEQS=3
FRANK_PRIMARY_KV_BYTES=3150000000
FRANK_PRIMARY_KV_DTYPE=fp8
FRANK_PRIMARY_MNBT=1024
FRANK_PRIMARY_GATHER_CHUNKS=32
FRANK_PRIMARY_GATHER_LANES=64
FRANK_PRIMARY_UTIL=0.98

# --- client contract: reasoning split, tools, preserve_thinking ----------------
# Without --reasoning-parser the engine leaves the thinking text INLINE in
# `content` with a dangling `</think>` and a null `reasoning` field (measured
# 2026-10-01: content = 'We need answer...Brief.\n</think>\n\n391') — the
# malformed thinking block clients choke on. The qwen3 parser splits it
# properly. The GLM rollback (d91379d) had dropped all three of these; the
# working values are the ones ecaad36 shipped for this model.
FRANK_PRIMARY_REASONING_PARSER=qwen3
# Tools: --enable-auto-tool-choice + the qwen3_xml parser is what makes
# tool_calls work here (without it, tool_choice auto returns 400).
FRANK_PRIMARY_TOOL_PARSER=qwen3_xml
# preserve_thinking keeps prior thinking blocks in the rendered prompt across
# turns; enable_thinking makes thinking the server-side default. Verified the
# payload's tokenizer_config template consumes both keys.
FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS='{"enable_thinking": true, "preserve_thinking": true}'