# frank-primary.env — FreeToken (PR #587 ROCm) serving the orcarouter
# Qwen3.8-Flash-Next-Uncensored-FP8 payload on dankserver :7770.
#
# 2026-10-05 cutover from the tcclaviger RDNA4 vLLM fork (git tag
# frank-primary-vllm-shapeG-20261003 is the pre-cutover state; the 362-line
# vLLM-era env is in that tag). The engine is FreeToken at PR head
# 77a7201f98f00d93f618c6cf8b81ffe8360e0899 (branch rocm714_torch214), built
# on the host into a local image by
# scripts/docker/frank-primary-freetoken.Dockerfile — the PR's own
# docs/install_amd.md stack (ROCm 7.14 base image, torch 2.11.0+rocm7.14,
# triton 3.7) for gfx1201.
#
# WHY FREETOKEN: the vLLM fork's expert offload planned VRAM/host tiers per
# boot from an estimator (util 0.982 law, arena 17.77 GiB / 7,762 slots,
# ~40k-token GDN state faults) and 1.0 was unbootable. FreeToken keeps ALL
# expert weights in pinned HOST RAM (~120 GiB) with an LRU GPU cache, sizes
# KV from an explicit token pool, and serves on ONE R9700 (32 GB) with the
# second card idle. RAM fit: 246 GiB total, service stopped frees the vLLM
# container's ~170 GiB; weights ~120 GiB + PLE 47.7 GiB + overhead ≈ 190 GiB.
#
# PAYLOAD: unchanged — /home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8,
# 131 safetensors shards. Quant is standard modelopt block-FP8: 75,264
# weight_scale_inv tensors (block 128x128), zero input_scale (FreeToken #462
# covers exactly this export shape), one weight_scale on the PLE ngram
# embedding. FreeToken loads HF safetensors directly; no FTW conversion.
#
# SERVING FACTS (docs/cli.md, verified against main+PR587):
# - `ft serve` loads the checkpoint directly; `--served-model-name` keeps the
#   harness id; reasoning/tool parsers are AUTO (reasoning lands in
#   reasoning_content — models.yml flips reasoningContentField to match).
# - KV: --page-size must stay 1 for GDN/SWA models (default); --cache-type
#   radix auto-picks the GDN-aware variant.
# - MoE: --moe-strategy auto -> offload (fp8 experts CANNOT use
#   --moe-cpu-layers; only bf16/nvfp4/mxfp4 have a CPU executor).
#   --moe-cache-rate holds the GPU LRU fraction of all experts.
# - `ft ctl cache --kv --moe` grows the pools LIVE on a running server
#   (shrinking needs a restart); `ft ctl stats` reports pool occupancy.
# - KV MATH: 12 full-attention layers x 2 KV heads x 256 head_dim x 2 (K+V)
#   x 2 B = 24 KiB/token (f16 KV; the 36 GDN layers use fixed per-seq state,
#   not per-token KV). See FRANK_PRIMARY_FT_NUM_TOKENS below for the pool
#   size and its measured page-size law.
FRANK_PRIMARY_IMAGE=localhost/frank-primary-freetoken:pr587-77a7201
FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8
FRANK_PRIMARY_MODEL_SHARDS=131
# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
# name across systemctl and podman ps. The vLLM-era name frank-primary-qwen
# moved into the legacy list so every deploy cleans it up.
FRANK_PRIMARY_CONTAINER=frank-primary
FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-qwen frank-primary-r9v qwen-fp8 glm-probe"
FRANK_PRIMARY_SERVED_NAME=frank-primary
FRANK_PRIMARY_PUBLISH=7770:7770

# --- FreeToken engine knobs (first-boot conservative, resize live) -----
# Single GPU: the R9700 at index 0. HIP_VISIBLE_DEVICES=0 hides card 1 so
# the in-container index 0 IS the right card; --gpu 0 selects it.
FRANK_PRIMARY_FT_GPU=0
# KV pool in TOKENS (not bytes — vLLM-era FRANK_PRIMARY_KV_BYTES is gone).
# 299,968 tokens x 24 KiB = 7.0 GiB; clears the harness worst case
# (177,356 tokens) and one NATIVE 262,144-token conversation + full budget.
# MEASURED FIRST BOOT (2026-10-05): the auto attention backend is
# qsa_sparse, which OVERRIDES page size to 64, so --num-tokens must be a
# MULTIPLE OF 64 — 300,000 was refused ("nearest valid values: 299968 or
# 300032"). Grows live with `ft ctl cache --kv <n>`; shrink needs a
# restart.
FRANK_PRIMARY_FT_NUM_TOKENS=299968
# Concurrent decode slots. 2 is the first-boot conservative value; the
# radix pool admits ~2 full-length 200,704-token conversations. Raise with
# --max-running-requests only after a real-traffic look at slot occupancy.
FRANK_PRIMARY_FT_MAX_RUNNING=2
# Per-request output budget ceiling: the harness's maxTokens (36,864).
# Requests that omit max_tokens get this default instead of the engine's
# 32,768.
FRANK_PRIMARY_FT_MAX_OUTPUT_TOKENS=36864
# GPU expert LRU cache as a FRACTION of all experts (~120 GiB fp8 total ->
# ~12 GiB VRAM at 0.10). First-boot conservative; resize live with
# `ft ctl cache --moe` after reading the boot ledger + ft ctl stats.
FRANK_PRIMARY_FT_MOE_RATE=0.10
# Escape hatch for boot-ladder experiments (YaRN, --max-prefill-length,
# --moe-cpu-threads, --sampling-defaults none, ...). Unset -> no extra
# argv. Keep multi-flag values as one string; the serve script expands it
# unquoted on purpose.
FRANK_PRIMARY_FT_EXTRA_ARGS=