frank-primary: radiance native-FP8 TP=2 cutover (engine, payload build, rollback)
Replace the FreeToken PR587 endpoint (single-GPU: its TP=2 dies in the pynccl.cu ROCm JIT) with the radiance engine on both R9700s, TP=2 via libr4d's fused one-shot all-reduce. Official Qwen/Qwen3.8-Flash-Next-FP8 payload at native block-FP8 precision: - scripts/build-frank-primary-rad.sh: staging (quantization_config drop = plugin SELECTION law) -> checkpoint reshape -> rad-convert with ONE --quant rule (PLE rtn fixed-scale); plan-only first. - scripts/stack-qwen38-official-fp8.py: the two value-preserving reshapes (experts byte-stacked into the plugin's stacked layout; PLE's 128 scaleless fp8 shards decoded to one bf16 shard with the checkpoint's own scale; the rtn rule re-encodes at that same scale - codes round-trip exactly and land the one-element scale plane the plugin declares). - scripts/hfget.sh + verified download (131/131 shards sha256-match). - froggeric v22.5 chat template vendored (--override-chat-template). - serve script (pinned ledger: 2x40 GiB pools + 47.7 GiB PLE + prefix host), TTM preflight fixed for kernels with pages_limit only (no pages_max - an absent knob is skipped, not read as zero); pages_limit raised to 52,428,800 pages = 200 GiB on the host pre-cutover. - root unit, relinked; deployer gates image digest + .rad floor 158,000 MiB + health; prometheus re-adds the /metrics job. - rollback: tag frank-primary-freetoken-pr587-20261005 (FreeToken trio + Dockerfile); payload + image stay on disk until the window closes. deploy-remote.sh also carries the parallel session's frank-small payload gates (swift15) - committed here per Frank's direction to commit the tree.
This commit is contained in:
Frank
+167
-136
@@ -1,25 +1,22 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: frank-primary — the Dankserver LLM endpoint on :7770.
|
||||
#
|
||||
# This file asserts the FreeToken (PR #587 ROCm) ROOT-UNIT shape (the
|
||||
# 2026-10-05 cutover from the tcclaviger RDNA4 vLLM fork, git tag
|
||||
# frank-primary-vllm-shapeG-20261003): the serve script drives `podman run`
|
||||
# DIRECTLY (the image carries `ft serve`, built on the host from
|
||||
# scripts/docker/frank-primary-freetoken.Dockerfile), one R9700, with every
|
||||
# engine value carried in the repo env file. The seam pins what flags alone
|
||||
# do not say: the KV-pool math (300k tokens at 24 KiB/token clears the
|
||||
# harness worst case of 177,356), the deployed single-GPU shape
|
||||
# (--tensor-parallel-size EXISTS in FreeToken — a 2-card TP shape is future
|
||||
# tuning; HIP_VISIBLE_DEVICES pins THIS shape to one card), the host-RAM
|
||||
# law (experts + PLE pin ~170 GiB — the memlock law that makes the pin
|
||||
# legal), the self-healing law (the run returns before the engine is up, so
|
||||
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
|
||||
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, the
|
||||
# OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision in a
|
||||
# rad-convert container). It replaces the FreeToken seam of 2026-10-05
|
||||
# (git tag frank-primary-freetoken-pr587-20261005) and pins what flags
|
||||
# alone do not say: the served window clears the harness worst case, the
|
||||
# pinned-host ledger has a ttm pages cap floor and the serve script
|
||||
# ENFORCES it, the served window is what models.yml must quote, the host-
|
||||
# networking contract (no --publish), the image gate on a name@sha256 ref,
|
||||
# the .rad payload gate with its size floor, the chat-template override
|
||||
# mount, the self-healing law (the run returns before the engine is up, so
|
||||
# the serve script must wait on the container for Restart=always to mean
|
||||
# anything), the image-existence gate (a local build carries no
|
||||
# RepoDigests — the old digest grep refused exactly this shape; pinning
|
||||
# lives in the Dockerfile), the deployer's payload gates with the
|
||||
# false-green MainPID check, and the client contract (parsers AUTO:
|
||||
# reasoning_content emission is the engine's native behavior — models.yml
|
||||
# maps reasoningContentField on the harness side, not here).
|
||||
# anything), the deployer's false-green MainPID check, the vendored hfget
|
||||
# fetch path (no token for this ungated repo), and the client contract
|
||||
# (reasoning auto -> reasoning_content; models.yml maps the field on the
|
||||
# harness side, not here).
|
||||
|
||||
load helpers
|
||||
|
||||
@@ -28,24 +25,33 @@ SERVE="$REPO_ROOT/scripts/serve-frank-primary.sh"
|
||||
ENV_FILE="$REPO_ROOT/scripts/frank-primary.env"
|
||||
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
|
||||
DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
BUILD="$REPO_ROOT/scripts/build-frank-primary-rad.sh"
|
||||
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
||||
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
||||
|
||||
@test "frank-primary unit, serve script, env and Dockerfile exist" {
|
||||
@test "frank-primary unit, serve script, env, fetch and build scripts and template exist" {
|
||||
[ -f "$UNIT" ]
|
||||
[ -x "$SERVE" ]
|
||||
[ -f "$ENV_FILE" ]
|
||||
[ -f "$DOCKERFILE" ]
|
||||
[ -f "$DOWNLOAD" ]
|
||||
[ -x "$BUILD" ]
|
||||
[ -x "$HFGET" ]
|
||||
[ -f "$TEMPLATE" ]
|
||||
# The FreeToken-era Dockerfile is retired with the runtime it built:
|
||||
# the radiance image is the publisher's, pulled and digest-pinned, not
|
||||
# built on the host.
|
||||
[ ! -f "$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile" ]
|
||||
}
|
||||
|
||||
@test "frank-primary unit is boot-persistent, self-healing and unprivileged" {
|
||||
# Root unit parity with frank-large/frank-small: enable is the [Install]
|
||||
# section, the restart policy survives a crash, and the pinned host RAM
|
||||
# (expert rows + PLE table) needs an unbounded memlock (a root unit
|
||||
# grants it via PID1; the dank session's own hard limit is 8 MiB).
|
||||
# (expert pools + PLE table + prefix host tier, ~132 GiB) needs an
|
||||
# unbounded memlock (a root unit grants it via PID1; the dank session's
|
||||
# own hard limit is 8 MiB).
|
||||
grep -q '^WantedBy=multi-user.target$' "$UNIT"
|
||||
# Restart=always, not on-failure: the vLLM-era lesson carries over — a
|
||||
# clean front-end exit after a worker fault reads as success to systemd,
|
||||
# so on-failure never fires.
|
||||
# Restart=always, not on-failure: a clean front-end exit after an
|
||||
# engine fault reads as success to systemd, so on-failure never fires.
|
||||
grep -q '^Restart=always$' "$UNIT"
|
||||
! grep -q '^Restart=on-failure$' "$UNIT"
|
||||
# ExecStop runs `podman stop -t 120`, so the stop timeout MUST exceed
|
||||
@@ -72,109 +78,156 @@ DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
@test "the unit stops the container child explicitly — container identity is the unit name" {
|
||||
# The serve script blocks on podman wait, which cannot stop the
|
||||
# container child; systemd stop goes through ExecStop or the engine
|
||||
# outlives the unit and holds the GPU. Frank's 2026-10-05 call: the
|
||||
# container name IS the service name (frank-primary), and the vLLM-era
|
||||
# name frank-primary-qwen moved into the legacy cleanup list.
|
||||
# outlives the unit and holds the GPU. The container name IS the
|
||||
# service name across runtimes (Frank's 2026-10-05 call).
|
||||
grep -q '^ExecStop=/usr/bin/podman stop' "$UNIT"
|
||||
grep -q 'podman stop -t 120 frank-primary$' "$UNIT"
|
||||
grep -q '^FRANK_PRIMARY_CONTAINER=frank-primary$' "$ENV_FILE"
|
||||
grep -q 'frank-primary-qwen' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "the serve shape is direct podman run, not a launcher, shim, or the vLLM fork" {
|
||||
@test "the serve shape is direct podman run, not a launcher or a predecessor runtime" {
|
||||
grep -q 'podman run' "$SERVE"
|
||||
! grep -q 'launch.sh' "$SERVE"
|
||||
! grep -q 'podman-shim' "$SERVE"
|
||||
! grep -q 'R9V_STATE_FILE' "$SERVE"
|
||||
# The tcclaviger vLLM fork is RETIRED — its image name and its
|
||||
# fork-specific flags must not survive in the FreeToken run line
|
||||
# (TP=2 is gone with the fork; FreeToken serves ONE GPU).
|
||||
# Every predecessor runtime is RETIRED — its image name and its
|
||||
# engine-specific flags must not survive in the radiance run line.
|
||||
! grep -q 'tcclaviger' "$SERVE"
|
||||
! grep -q 'tcclaviger' "$ENV_FILE"
|
||||
! grep -q 'freetoken\|ft serve' "$SERVE"
|
||||
! grep -q 'freetoken' "$ENV_FILE"
|
||||
! grep -q 'HIP_VISIBLE_DEVICES' "$SERVE"
|
||||
! grep -q -- '--tensor-parallel-size' "$SERVE"
|
||||
! grep -q -- '--kv-cache-dtype' "$SERVE"
|
||||
! grep -q -- '--max-num-seqs' "$SERVE"
|
||||
! grep -q -- '--moe-cache-rate' "$SERVE"
|
||||
! grep -q -- '--num-tokens ' "$SERVE"
|
||||
}
|
||||
|
||||
@test "the FreeToken engine shape is in the env file" {
|
||||
# KV pool: TOKENS, not bytes. 24 KiB/token (12 full-attention layers x
|
||||
# 2 KV heads x 256 head_dim x K+V x f16) — the 36 GDN layers use fixed
|
||||
# per-seq state, not per-token KV. 299,968 tokens = 7.0 GiB and clears
|
||||
# the harness worst case (140,492-token prompt + 36,864 max_tokens =
|
||||
# 177,356); one NATIVE 262,144-token conversation + full budget also
|
||||
# fits. The value must stay a MULTIPLE OF 64: the auto attention
|
||||
# backend is qsa_sparse, which overrides page size to 64 (measured
|
||||
# first boot — 300,000 was refused). The arithmetic guard keeps future
|
||||
# resizes honest.
|
||||
grep -q '^FRANK_PRIMARY_FT_NUM_TOKENS=299968$' "$ENV_FILE"
|
||||
nt=$(awk -F= '/^FRANK_PRIMARY_FT_NUM_TOKENS=/{print $2}' "$ENV_FILE")
|
||||
[ "$nt" -ge 177356 ]
|
||||
[ $((nt % 64)) -eq 0 ]
|
||||
# First-boot conservative shape: 2 decode slots, 10% expert LRU
|
||||
# (~12 GiB of the ~120 GiB fp8 expert set), output ceiling = the
|
||||
# harness maxTokens, single R9700 at index 0.
|
||||
grep -q '^FRANK_PRIMARY_FT_MAX_RUNNING=2$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_FT_MOE_RATE=0.10$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_FT_MAX_OUTPUT_TOKENS=36864$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_FT_GPU=0$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
|
||||
@test "the image is the publisher's, digest-pinned by manifest" {
|
||||
# stilldeadcode/radiance@sha256:<64 hex> — a name@digest ref exists in
|
||||
# storage only if THAT manifest was pulled. The tag form would float.
|
||||
grep -qE '^FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:[0-9a-f]{64}$' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "the radiance engine shape is in the env file" {
|
||||
# Served window must clear the harness worst case: OMP compacts at 70%
|
||||
# of contextWindow; for 200,000 that is 140,000, and a pre-compaction
|
||||
# prompt can legally reach that height — 140,492 + 36,864 = 177,356 at
|
||||
# the FreeToken-era window was the record; the same relation must hold
|
||||
# here (140,000 + 36,864 = 176,864 <= 200,000).
|
||||
grep -q '^FRANK_PRIMARY_MAX_MODEL_LEN=200000$' "$ENV_FILE"
|
||||
ml=$(awk -F= '/^FRANK_PRIMARY_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
|
||||
[ "$ml" -ge 177356 ]
|
||||
# First-boot conservative shape (Frank's call): 2 decode slots. The
|
||||
# fused TP all-reduce is declared over seqs x (1 + spec) rows = 8.
|
||||
grep -q '^FRANK_PRIMARY_MAX_SEQS=2$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_TP=2$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_TP_WIRE=wht6$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_SPEC_TOKENS=3$' "$ENV_FILE"
|
||||
# Native-precision economics: the expert pool is per rank (56.4 GiB
|
||||
# slice per rank at TP2), the PLE ngram table is pinned in host RAM per
|
||||
# Frank's law, KV is fp8, the ratio is the publisher's production
|
||||
# value.
|
||||
grep -q '^FRANK_PRIMARY_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_HOST_POOL_MIB=40960$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_EXPERT_VS_CACHE=0.82$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
||||
grep -q '^FRANK_PRIMARY_PREFIX_HOST_MIB=4096$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
|
||||
# The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
|
||||
# under the ttm pages cap, and the env states the floor the serve
|
||||
# script enforces (>= 140,000 MiB — the current cap ~123 GiB fails it,
|
||||
# which is the point).
|
||||
grep -q '^FRANK_PRIMARY_TTM_MIN_MIB=' "$ENV_FILE"
|
||||
ttmm=$(awk -F= '/^FRANK_PRIMARY_TTM_MIN_MIB=/{print $2}' "$ENV_FILE")
|
||||
pool=$(awk -F= '/^FRANK_PRIMARY_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
|
||||
ph=$(awk -F= '/^FRANK_PRIMARY_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
|
||||
# 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
|
||||
ledger=$(( 2 * pool + 48824 + ph + 2048 ))
|
||||
[ "$ttmm" -ge "$ledger" ]
|
||||
# The .rad size floor is positive (a truncated convert must fail fast).
|
||||
grep -q '^FRANK_PRIMARY_RAD_MIN_MIB=' "$ENV_FILE"
|
||||
[ "$(awk -F= '/^FRANK_PRIMARY_RAD_MIN_MIB=/{print $2}' "$ENV_FILE")" -gt 0 ]
|
||||
}
|
||||
|
||||
@test "the serve script wires the engine flags from the env" {
|
||||
grep -q 'ft serve --model /models' "$SERVE"
|
||||
grep -q -- '--model "/models/$FRANK_PRIMARY_MODEL_FILE"' "$SERVE"
|
||||
grep -q -- '--served-model-name "$FRANK_PRIMARY_SERVED_NAME"' "$SERVE"
|
||||
grep -q -- '--host 0.0.0.0 --port 7770' "$SERVE"
|
||||
grep -q -- '--gpu "$FRANK_PRIMARY_FT_GPU"' "$SERVE"
|
||||
grep -q -- '--num-tokens "$FRANK_PRIMARY_FT_NUM_TOKENS"' "$SERVE"
|
||||
grep -q -- '--max-running-requests "$FRANK_PRIMARY_FT_MAX_RUNNING"' "$SERVE"
|
||||
grep -q -- '--max-output-tokens "$FRANK_PRIMARY_FT_MAX_OUTPUT_TOKENS"' "$SERVE"
|
||||
grep -q -- '--moe-cache-rate "$FRANK_PRIMARY_FT_MOE_RATE"' "$SERVE"
|
||||
# Single-GPU law: visibility is pinned to one card so in-container
|
||||
# index 0 is the right R9700 (mirrors the env's FT_GPU).
|
||||
grep -q -- '-e HIP_VISIBLE_DEVICES="$FRANK_PRIMARY_FT_GPU"' "$SERVE"
|
||||
grep -q -- '--host :: --port "$FRANK_PRIMARY_PORT"' "$SERVE"
|
||||
grep -q -- '--override-chat-template /templates/template.jinja' "$SERVE"
|
||||
grep -q -- '--tp "$FRANK_PRIMARY_TP" --tp-wire "$FRANK_PRIMARY_TP_WIRE"' "$SERVE"
|
||||
grep -q -- '--max-num-seqs "$FRANK_PRIMARY_MAX_SEQS"' "$SERVE"
|
||||
grep -q -- '--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN"' "$SERVE"
|
||||
grep -q -- '--placement "$FRANK_PRIMARY_PLACEMENT"' "$SERVE"
|
||||
grep -q -- '--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB"' "$SERVE"
|
||||
grep -q -- '--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE"' "$SERVE"
|
||||
grep -q -- '--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE"' "$SERVE"
|
||||
grep -q -- '--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB"' "$SERVE"
|
||||
grep -q -- '--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB"' "$SERVE"
|
||||
grep -q -- '--ngram-placement "$FRANK_PRIMARY_NGRAM_PLACEMENT"' "$SERVE"
|
||||
grep -q -- '--num-speculative-tokens "$FRANK_PRIMARY_SPEC_TOKENS"' "$SERVE"
|
||||
# The boot-ladder escape hatch is opt-in: unset renders NO extra argv.
|
||||
grep -q -- '${FRANK_PRIMARY_FT_EXTRA_ARGS:+$FRANK_PRIMARY_FT_EXTRA_ARGS}' "$SERVE"
|
||||
! grep -q '^FRANK_PRIMARY_FT_EXTRA_ARGS=.' "$ENV_FILE"
|
||||
grep -q -- '${FRANK_PRIMARY_EXTRA_ARGS:+$FRANK_PRIMARY_EXTRA_ARGS}' "$SERVE"
|
||||
! grep -q '^FRANK_PRIMARY_EXTRA_ARGS=.' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "the container contract is the publisher's (host network, io_uring, pin, cards)" {
|
||||
# docs/DOCKER.md: EVERY one of these is needed — host networking (the
|
||||
# server binds the host's addresses; no --publish), --init (the engine
|
||||
# releases the cards on SIGTERM), seccomp=unconfined (the default
|
||||
# profile refuses io_uring, which the n-gram reads use, and the NUMA
|
||||
# binding calls), memlock unbounded (the pinned ledger), kfd + dri.
|
||||
grep -q -- '--network host' "$SERVE"
|
||||
! grep -q -- '--publish' "$SERVE"
|
||||
! grep -q -- '-p 7770' "$SERVE"
|
||||
grep -q '^FRANK_PRIMARY_PORT=7770$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_SERVED_NAME=frank-primary$' "$ENV_FILE"
|
||||
grep -q -- '--init' "$SERVE"
|
||||
grep -q -- '--security-opt seccomp=unconfined' "$SERVE"
|
||||
grep -q -- '--ulimit memlock=-1' "$SERVE"
|
||||
grep -q -- '--device /dev/kfd' "$SERVE"
|
||||
grep -q -- '--device /dev/dri' "$SERVE"
|
||||
grep -q -- '-v "$FRANK_PRIMARY_MODEL_DIR":/models:ro' "$SERVE"
|
||||
grep -q -- '-v "$FRANK_PRIMARY_STATE_DIR":/data' "$SERVE"
|
||||
grep -q -- '-v "$FRANK_PRIMARY_TEMPLATE":/templates/template.jinja:ro' "$SERVE"
|
||||
}
|
||||
|
||||
@test "parsers are AUTO — the engine infers them natively, the harness maps the field" {
|
||||
# FreeToken infers the qwen3.8 reasoning/tool dialects from the model
|
||||
# family and emits thinking in `reasoning_content` natively — NO
|
||||
# --reasoning-parser/--tool-call-parser override flags. The vLLM-era
|
||||
# client-contract wiring (qwen3 parser, qwen3_xml tool parser,
|
||||
# default-chat-template-kwargs) is gone; models.yml's
|
||||
# reasoningContentField flip is the harness-side contract and lives in
|
||||
# the operator's home, not this repo.
|
||||
# radiance --reasoning-format defaults to auto and emits thinking in
|
||||
# `reasoning_content`; tool calls ride the chat format. NO parser
|
||||
# override flags exist in this engine. models.yml's
|
||||
# reasoningContentField is the harness-side contract and lives in the
|
||||
# operator's home, not this repo.
|
||||
! grep -q -- '--reasoning-parser' "$SERVE"
|
||||
! grep -q -- '--tool-call-parser' "$SERVE"
|
||||
! grep -q -- '--enable-auto-tool-choice' "$SERVE"
|
||||
! grep -q -- '--default-chat-template-kwargs' "$SERVE"
|
||||
! grep -q 'FRANK_PRIMARY_REASONING_PARSER' "$ENV_FILE"
|
||||
! grep -q 'FRANK_PRIMARY_TOOL_PARSER' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "the serve script gates the image, the payload and the GPU nodes" {
|
||||
# The image gate is EXISTENCE (podman image exists): the FreeToken
|
||||
# image is built ON THE HOST — a local tag carries no RepoDigests — and
|
||||
# pinning lives in the Dockerfile (pinned PR-head SHA, base image).
|
||||
# The payload gate checks config.json and the exact shard count from
|
||||
# the env — a missing shard means a guaranteed long-timeout boot
|
||||
# instead of a fast failure.
|
||||
@test "the serve script gates the image, the .rad payload, the template and the GPU nodes" {
|
||||
# The image gate is the digest ref itself; the payload gate checks the
|
||||
# .rad file and its size floor — a truncated convert would otherwise be
|
||||
# a guaranteed long-timeout boot instead of a fast failure. The ttm
|
||||
# preflight refuses to start under a pages cap below the pinned ledger.
|
||||
grep -q 'FRANK_PRIMARY_IMAGE' "$SERVE"
|
||||
grep -q 'podman image exists' "$SERVE"
|
||||
grep -q 'FRANK_PRIMARY_MODEL_SHARDS' "$SERVE"
|
||||
grep -q 'config.json' "$SERVE"
|
||||
grep -q 'model.safetensors.index.json' "$SERVE"
|
||||
grep -q 'FRANK_PRIMARY_MODEL_FILE' "$SERVE"
|
||||
grep -q 'FRANK_PRIMARY_RAD_MIN_MIB' "$SERVE"
|
||||
grep -q 'FRANK_PRIMARY_TEMPLATE' "$SERVE"
|
||||
grep -q 'FRANK_PRIMARY_TTM_MIN_MIB' "$SERVE"
|
||||
grep -q 'pages_limit' "$SERVE"
|
||||
grep -q 'pages_max' "$SERVE"
|
||||
grep -q '/dev/kfd' "$SERVE"
|
||||
grep -q -- '--device /dev/dri' "$SERVE"
|
||||
}
|
||||
|
||||
@test "the serve script clears stale containers and self-heals via podman wait" {
|
||||
# A crashed unit can leave the container name registered forever after;
|
||||
# the legacy identities (the vLLM-era frank-primary-qwen container, the
|
||||
# retired R9V container and the ad hoc experiment containers) hold the
|
||||
# same GPU and die here too. Then block on the container so a crash is
|
||||
# a nonzero ExecStart.
|
||||
# the legacy identities hold the same GPUs and die here too. Then block
|
||||
# on the container so a crash is a nonzero ExecStart.
|
||||
grep -q 'FRANK_PRIMARY_LEGACY_CONTAINERS' "$ENV_FILE"
|
||||
grep -q 'frank-primary-qwen' "$ENV_FILE"
|
||||
grep -q 'frank-primary-r9v' "$ENV_FILE"
|
||||
@@ -182,19 +235,19 @@ DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
grep -q 'exec podman wait "$FRANK_PRIMARY_CONTAINER"' "$SERVE"
|
||||
}
|
||||
|
||||
@test "frank-primary holds the single fleet port under its own alias" {
|
||||
# LAN publish of :7770, the harness's model id.
|
||||
grep -q '^FRANK_PRIMARY_PUBLISH=7770:7770$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_SERVED_NAME=frank-primary$' "$ENV_FILE"
|
||||
}
|
||||
|
||||
@test "the model id matches the payload's served alias wiring" {
|
||||
# The payload dir and its shard count are the fetch script's output
|
||||
# layout; the fetch script must write where the env points.
|
||||
grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
|
||||
grep -q 'Qwen3.8-Flash-Next-Uncensored-FP8' "$DOWNLOAD"
|
||||
grep -q 'orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8' "$DOWNLOAD"
|
||||
@test "the model id matches the payload's build and fetch scripts" {
|
||||
# The payload dir/file are the build script's output layout; the fetch
|
||||
# script must name the repo the build stages from.
|
||||
grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
|
||||
grep -q '^FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad$' "$ENV_FILE"
|
||||
grep -q 'qwen4exp-fp8-native.rad' "$BUILD"
|
||||
grep -q 'Qwen/Qwen3.8-Flash-Next-FP8' "$DOWNLOAD"
|
||||
grep -q 'hfget.sh' "$DOWNLOAD"
|
||||
# The repo is UNGATED — no token may be required on the fetch path.
|
||||
! grep -q 'HF_TOKEN' "$DOWNLOAD"
|
||||
# The staging step drops quantization_config (the descriptor fix) —
|
||||
# document it where it happens.
|
||||
grep -q 'quantization_config' "$BUILD"
|
||||
}
|
||||
|
||||
@test "no bare HIP-reserved env var is defined" {
|
||||
@@ -205,37 +258,16 @@ DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
! grep -qE '^(export )?(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$SERVE"
|
||||
}
|
||||
|
||||
@test "the Dockerfile pins the FreeToken source and follows the ROCm install law" {
|
||||
# The engine is the PR #587 head, NOT a moving branch: a future push to
|
||||
# rocm714_torch214 would otherwise silently change the deployed engine.
|
||||
grep -q '77a7201f98f00d93f618c6cf8b81ffe8360e0899' "$DOCKERFILE"
|
||||
grep -q 'pull/587/head' "$DOCKERFILE"
|
||||
# docs/install_amd.md's build law: --no-build-isolation --no-deps (a
|
||||
# plain `pip install -e .` on ROCm can REPLACE the ROCm torch with a
|
||||
# CUDA build via its NVIDIA-authored dependency set), curated portable
|
||||
# deps, flashlib --no-deps.
|
||||
grep -q -- '--no-build-isolation --no-deps -e .' "$DOCKERFILE"
|
||||
grep -q 'flashlib==0.3.0' "$DOCKERFILE"
|
||||
# The CUDA-only backends have no ROCm builds and are intentionally
|
||||
# absent (the PR's 12 pytest failures are exactly those backends;
|
||||
# FreeToken falls back to its Triton kernels).
|
||||
! grep -q 'flashinfer' "$DOCKERFILE"
|
||||
! grep -q 'sgl_kernel' "$DOCKERFILE"
|
||||
! grep -q 'nvidia-cutlass-dsl' "$DOCKERFILE"
|
||||
# The base image is digest-pinned (the registry tag is not trusted).
|
||||
grep -qE '^FROM rocm/pytorch:.*@sha256:[0-9a-f]{64}$' "$DOCKERFILE"
|
||||
}
|
||||
|
||||
@test "the deployer gates the image, payload and containers, then health-checks" {
|
||||
# The deployer refuses when the image is absent from rootless storage
|
||||
# or the payload is missing/wrong-sharded — then clears stale containers
|
||||
# (including the legacy identities), waits out the port,
|
||||
# The deployer refuses when the image is absent from dank's rootless
|
||||
# storage or the .rad is missing/too small — then clears stale
|
||||
# containers (including the legacy identities), waits out the port,
|
||||
# relinks/enables/restarts the root unit and health-gates it.
|
||||
grep -q 'LLM_PRIMARY_CONTAINER=frank-primary$' "$REMOTE_DEPLOY"
|
||||
grep -q 'FRANK_PRIMARY_IMAGE' "$REMOTE_DEPLOY"
|
||||
grep -q 'podman image exists' "$REMOTE_DEPLOY"
|
||||
grep -q 'FRANK_PRIMARY_MODEL_SHARDS' "$REMOTE_DEPLOY"
|
||||
grep -q 'config.json' "$REMOTE_DEPLOY"
|
||||
grep -q 'FRANK_PRIMARY_MODEL_FILE' "$REMOTE_DEPLOY"
|
||||
grep -q 'FRANK_PRIMARY_RAD_MIN_MIB' "$REMOTE_DEPLOY"
|
||||
grep -q 'frank-primary-qwen' "$REMOTE_DEPLOY"
|
||||
grep -q 'qwen-fp8' "$REMOTE_DEPLOY"
|
||||
grep -q 'glm-probe' "$REMOTE_DEPLOY"
|
||||
@@ -243,13 +275,12 @@ DOCKERFILE="$REPO_ROOT/scripts/docker/frank-primary-freetoken.Dockerfile"
|
||||
grep -q 'relink_unit frank-primary' "$REMOTE_DEPLOY"
|
||||
grep -q 'systemctl enable frank-primary.service' "$REMOTE_DEPLOY"
|
||||
grep -q 'systemctl restart frank-primary.service' "$REMOTE_DEPLOY"
|
||||
# The health gate reads the BODY, not just the HTTP code: FreeToken
|
||||
# returns 200 with {"status":"loading"} while the engine loads (the
|
||||
# FastAPI front end binds before the scheduler is up — a bare 200
|
||||
# false-greened a deploy at 6 s while 131 shards were still paging in,
|
||||
# measured 2026-10-06). Only status:"ok" counts.
|
||||
# The health gate is a bare curl: the HTTP listener binds AFTER the
|
||||
# full load (main.cpp — engine.start() then engine.serve()), so during
|
||||
# load the port refuses connections and a bare curl is honest. The
|
||||
# FreeToken-era body read ('status ok') is gone with that engine.
|
||||
grep -q 'curl -fsS http://127.0.0.1:7770/health' "$REMOTE_DEPLOY"
|
||||
grep -q 'never reached /health status ok' "$REMOTE_DEPLOY"
|
||||
! grep -q '"status":"ok"' "$REMOTE_DEPLOY"
|
||||
}
|
||||
|
||||
@test "the deployer will not report green on a stray port-holder" {
|
||||
|
||||
Reference in New Issue
Block a user