frank-primary: Lucebox DeepSeek-V4.1-Flash-ROCMFP23 quadlet + deploy handler
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB 43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the MLA KV cache on the R9700, routed experts computed on CPU from the page cache (246 GiB RAM). The unit bypasses the image entrypoint (Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch shape; --ds4-fused-decode's monolithic-load fallback IS the serving mode here. deploy_llm_primary gates on image + model byte count, stops/masks the radiance unit, waits for :7770, installs and starts. Runbook: frank-primary-lucebox-ds4.md.
This commit is contained in:
+14
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
"""frank-primary health probe: 200 on /health only once the engine serves.
|
||||
|
||||
File-based (not an inline HealthCmd) because the podman 5.x v1 quadlet
|
||||
generator splits HealthCmd on spaces into an argv array.
|
||||
"""
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen("http://127.0.0.1:7770/health", timeout=10) as r:
|
||||
sys.exit(0 if r.status == 200 else 1)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
Reference in New Issue
Block a user