# frank-large.env — tunables for the frank-large endpoint on Dankclaw's
# Strix Halo iGPU (gfx1151), host port 7770.
#
# RUNTIME (2026-10-05, Frank): halogen — the closed-source C++/HIP engine
# (github.com/peonist-ai/halogen-server, image ghcr.io/peonist-ai/halogen,
# digest-pinned below) serving peonist-ai/halogen-qwen3.8-27b, a ~6.32 bpw
# NVFP4-derivative quant of the BASE Qwen/Qwen3.8-27B (35,865,565,184 B,
# 1352 tensors), as a podman container under a root unit, driven by
# scripts/serve-frank-large.sh. This CUTS OVER the
# 2026-10-02..10-03 ROCmFPX llama.cpp shape (uncensored
# Q6_0_ROCMFPX_AGENT + MTP, 27.6-28.1 t/s): rollback = revert the swap
# commit's trio; the ROCmFPX payload, draft head and build all stay on
# disk at /home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/.
#
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
# card it "will not load in transformers, vLLM, or llama.cpp". Accepted
# consequences (Frank, 2026-10-05): TEXT-ONLY (halogen ignores the model's
# vision encoder; the fleet's vision path is frank-small) and the CENSORED
# base (this is a finetune of Qwen/Qwen3.8-27B, not the abliterated file —
# refusals may return). What it buys: vendor-measured decode DFlash2 31.7
# t/s mean (vs this endpoint's MTP 27.6-28.1 and the old halogen-era MTP
# 26.9), prefill ~566 t/s at 32k (was ~400 on the sibling engine), 1.4-2.3x
# wall-clock on prefill-heavy agent traffic, at HIGHER effective precision
# (~6.32 bpw vs ~6.6 with the old W4A4 promotion — see FLAGS.md).
#
# SHAPE — the two knobs that are NOT free:
#   HALOGEN_KV_SLOTS=1 KEEPS SPECULATION ON. halogen makes batching and
#   speculation mutually exclusive: with >1 slot the drafter is disabled
#   and per-stream decode falls to serial speed (the sibling engine
#   measured 41 t/s -> 14 t/s when slots were raised, and halogen's own
#   FLAGS.md prints the same law). frank-large was already a one-session
#   endpoint under ROCmFPX (PARALLEL=1), so slots=1 is the same contract,
#   not a regression.
#   HALOGEN_SLOT_CTX=200700 (Frank's window) is the per-conversation hard
#   ceiling — a prompt over it is a 400 naming the limit, never a silent
#   truncation. KV pool = slots x slot_ctx x 64 KiB = 12.2 GiB. Budget
#   against the 121 GiB of unified memory: weights 33.4 GiB (mmap'd — page
#   cache, not anon), KV 12.2, engine working set + auto prompt cache
#   ~20-30, host baseline ~10 -> ~75-85 GiB in use. The 2026-09-16 halogen
#   era ran the same engine class on this box with LESS headroom.
#   HALOGEN_DRAFTER=2 (DFlash2) is the shipped default and the fastest
#   exact drafter on this checkpoint; the .hgn BAKES BOTH drafters (MTP +
#   DFlash2), so unlike the Flash-Next era there is no sidecar to mount
#   and no --dflash flag — the drafter is an int knob, byte-identical
#   output to greedy verification either way.
#
# CLIENT CONTRACT deltas vs the ROCmFPX shape (the harness side lives in
# ~/.omp/agent/models.yml):
#   * NO server-side sampler defaults: a request without temperature
#     decodes GREEDY. The harness sends temp 1.0 / top_p 0.95 via
#     extraBody, so that wire recipe is load-bearing now, not cosmetic.
#   * max_tokens defaults to 8192 PER REQUEST and INCLUDES thinking; a
#     reasoning reply over a small budget returns empty content with
#     finish_reason "length". The harness maxTokens is 57344: compaction
#     fires at 70% of 200700 = 140,490 and 140,490 + 57,344 = 197,834 <=
#     200,700 keeps prompt + budget inside the pool (the 262144-era 65536
#     would overflow). Requests over HALOGEN_MAX_TOKENS_CAP are a 400
#     naming the cap; cap and queue timeout are COUPLED (see FLAGS.md
#     table) — do not move one without the other.
#   * response_format json_object/json_schema is a 400 (no constrained
#     decoding in this build).
#   * Thinking lands in reasoning_content — same field as the llama.cpp
#     era, so the harness compat block does NOT change. reasoning_effort
#     accepts none|low|xhigh on this build (template default xhigh); the
#     models.yml fold (high->xhigh) still applies.
#   * The engine's token protocol has no auth and binds loopback INSIDE
#     the container; only the frontend port is published. Keep it that way.

# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
# is what was qualified). 0.1.4 = the version the model card names.
FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea
# Container identity: the halogen container name.
FRANK_LARGE_CONTAINER=frank-large-halogen
# Old quadlet-era container identity; the serve script clears stale names.
FRANK_LARGE_LEGACY_CONTAINERS=frank-large
# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
# default); the engine port (8730) is never published.
FRANK_LARGE_PUBLISH=7770:8731

FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-qwen3.8-27b
FRANK_LARGE_CHECKPOINT=qwen3.8-27b-p1w4d-d2.hgn
FRANK_LARGE_CHECKPOINT_BYTES=35865565184
FRANK_LARGE_CHECKPOINT_SHA256=274c3fc767fb57faf025dd03c76376b1fc1b32448aef45aaf02d1898fa962089
# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer, mounted
# whole at /tokenizer — the model card ships `tokenizer/` and the engine's
# HALOGEN_TOKENIZER default is that directory (the card's own docker run sets
# no tokenizer flag). The Hub stores it as 5 files (sizes from the Hub API,
# 2026-10-05): tokenizer.json 12809320 (LFS sha256
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3),
# vocab.json 6722759, merges.txt 3353259, chat_template.jinja 8952,
# tokenizer_config.json 17928 (plain git blobs — size-checked, no LFS oid).
# The serve gate and the deployer refuse a missing file or ANY symlink: an
# HF cache snapshot symlinks each file into blobs/, and those links dangle
# inside the container mount.
FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json"

# --- engine flags (docs/FLAGS.md of the engine repo; anything unset stays
# at the image default, which is what was qualified) ------------------------
HALOGEN_MODEL_ID=frank-large
HALOGEN_SLOT_CTX=200700
HALOGEN_KV_SLOTS=1
HALOGEN_DRAFTER=2
# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
# the GPU, the timeout bounds how long the NEXT client waits. 65536 at
# ~10 t/s serial worst case needs > 6550 s queue timeout; the defaults
# honor that. Do not lower the timeout alone.
HALOGEN_MAX_TOKENS_CAP=65536
HALOGEN_QUEUE_TIMEOUT=7200
# Engine prompt cache: EMPTY = auto (sized from available memory at
# startup). Pin an explicit MB only to argue with the auto figure — a
# full-context entry is ~18 GB at 262K, so a too-small pin yields a cache
# that reports enabled and never hits (the engine warns at startup).
#HALOGEN_CACHE_MB=