feat(frank-medium): cut the endpoint over to radiance TP=2
The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:
arm I llama.cpp layer split 32:33 + DFlash2, Q8_0 geo decode 45.7 t/s, TTFT 0.72 s, burst-4 71.7 t/s
arm L llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4 54.3 t/s
arm R radiance vLLM TP=2 + DFlash2-FP8, MXFP4 geo decode 95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s
Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.
- un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
The replica pair and its affinity proxy stay archived; one process owns
both cards now.
- retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
stay: they are the tooling that rollback needs.
- deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
the drafter, the pinned patch clone and the patched libr4d; retires the
llama root unit into the home trash instead of relinking it; then
install_quadlet + wait_container_healthy 1800s (first start compiles).
retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
a/b/proxy trio only, since frank-medium is installed again.
- planner + test-changed map dankserver/frank-medium/** to frank-medium
and its seam; the scripts-trio mapping is gone with the files.
- tests/frank_medium_unit.bats rewritten for the container contract: boot
persistence of a GENERATED unit (no enable on this host's v1 generator),
TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
contract, and the archive invariants.
- homepage card + docs updated, and they were stale twice over: they still
described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.
Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
This commit is contained in:
+8
-11
@@ -369,11 +369,14 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
_add(services, "node-exporter")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "frank-medium":
|
||||
# frank-medium is a ROOT llama.cpp unit since 2026-09-16: its
|
||||
# runtime is a native gfx1201 binary, not a container, so there is
|
||||
# nothing under dankserver/frank-medium/ to plan any more. The
|
||||
# retired vllm-radiance quadlets live in .AGENTTRASH/, which is
|
||||
# archive material and plans nothing.
|
||||
# frank-medium is a CONTAINER again since 2026-09-20: the TP=2
|
||||
# radiance Quadlet, its entrypoint and its healthcheck all live
|
||||
# under this directory, and the container mounts the latter two
|
||||
# straight out of this checkout — so any file here changes deployed
|
||||
# state. (It was a root llama.cpp unit from 2026-09-16, whose trio
|
||||
# lived in scripts/; that shape is archived in
|
||||
# scripts/.AGENTTRASH/20260920-llama-layersplit/ and plans nothing.)
|
||||
_add(services, "frank-medium")
|
||||
return services
|
||||
return services
|
||||
|
||||
@@ -405,12 +408,6 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
||||
# which is archive material and plans nothing.
|
||||
_add(services, "frank-small")
|
||||
if name in {"frank-medium.env", "frank-medium.service", "serve-frank-medium.sh"}:
|
||||
# llama.cpp frank-medium (2026-09-16) keeps its unit, env and serve
|
||||
# files here the same way. The files that BUILD and FETCH its
|
||||
# runtime and payload are host-side one-offs: they change no
|
||||
# deployed state, so they select nothing.
|
||||
_add(services, "frank-medium")
|
||||
if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
|
||||
# CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and
|
||||
# serve files here the same way: a root unit on a host-built
|
||||
|
||||
Reference in New Issue
Block a user