frank-medium: swap ROCmFPX llama.cpp runtime for vllm-radiance MXFP4
Rootless Quadlet on dankserver serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq (the radiance stack's designated alternate target; ships its fp8 MTP head, no rewrite step) with the DFlash2-FP8 drafter at SPEC=7 on the 2x R9700s (TP=2, host network, :7770, alias frank-medium unchanged). - dankserver/frank-medium/: quadlet unit, container entrypoint (fork of serve-mxfp4.sh's patch block at pinned clone commit 97353293), healthcheck - dispatcher retires the root llama.cpp unit, re-tags the registry image to the localhost alias, installs the quadlet, waits on /health - planner maps dankserver/frank-medium/* to frank-medium; scripts-era mappings removed - ROCmFPX-era env/serve/unit/download scripts staged in scripts/.AGENTTRASH with .rocmfpx suffix; seams updated (frank_medium_quadlet.bats new)
This commit is contained in:
1 parent
db29c4d227
commit
6b647c1558
13 files changed
+566
-47
No files matched your search
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env python3
|
||||
"""frank-medium container healthcheck: vLLM /health must answer 200.
|
||||
|
||||
Runs INSIDE the container. The vLLM /health endpoint only answers once the
|
||||
engine is up (weights loaded, cudagraphs captured), so a healthy container is
|
||||
a serving container, not merely a live process. The HealthCmd in the quadlet
|
||||
splits on spaces into an argv array (podman 5.x v1 generator), which is why
|
||||
this lives in a file rather than an inline python one-liner.
|
||||
"""
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen("http://127.0.0.1:7770/health", timeout=5) as r:
|
||||
sys.exit(0 if r.status == 200 else 1)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
Reference in new issue
Block a user