frank-primary: radiance native-FP8 TP=2 cutover (engine, payload build, rollback)
Replace the FreeToken PR587 endpoint (single-GPU: its TP=2 dies in the pynccl.cu ROCm JIT) with the radiance engine on both R9700s, TP=2 via libr4d's fused one-shot all-reduce. Official Qwen/Qwen3.8-Flash-Next-FP8 payload at native block-FP8 precision: - scripts/build-frank-primary-rad.sh: staging (quantization_config drop = plugin SELECTION law) -> checkpoint reshape -> rad-convert with ONE --quant rule (PLE rtn fixed-scale); plan-only first. - scripts/stack-qwen38-official-fp8.py: the two value-preserving reshapes (experts byte-stacked into the plugin's stacked layout; PLE's 128 scaleless fp8 shards decoded to one bf16 shard with the checkpoint's own scale; the rtn rule re-encodes at that same scale - codes round-trip exactly and land the one-element scale plane the plugin declares). - scripts/hfget.sh + verified download (131/131 shards sha256-match). - froggeric v22.5 chat template vendored (--override-chat-template). - serve script (pinned ledger: 2x40 GiB pools + 47.7 GiB PLE + prefix host), TTM preflight fixed for kernels with pages_limit only (no pages_max - an absent knob is skipped, not read as zero); pages_limit raised to 52,428,800 pages = 200 GiB on the host pre-cutover. - root unit, relinked; deployer gates image digest + .rad floor 158,000 MiB + health; prometheus re-adds the /metrics job. - rollback: tag frank-primary-freetoken-pr587-20261005 (FreeToken trio + Dockerfile); payload + image stay on disk until the window closes. deploy-remote.sh also carries the parallel session's frank-small payload gates (swift15) - committed here per Frank's direction to commit the tree.
This commit is contained in:
@@ -98,12 +98,13 @@ scrape_configs:
|
||||
# service down. Same host, same rule, so both local endpoints use the
|
||||
# gateway.
|
||||
#
|
||||
# (2026-10-05) frank-primary left this file: FreeToken (the PR587 ROCm
|
||||
# engine now serving it) has NO /metrics endpoint — the job scraped 404s
|
||||
# forever, which is a structural never-green, not an outage. The
|
||||
# netavark published-port quirk above is same-host only; plain LAN
|
||||
# routing REACHES 192.168.0.235:7770 (health answers) — only the metrics
|
||||
# path is absent. If FreeToken grows a /metrics, re-add the job here.
|
||||
# (2026-10-05) frank-primary LEFT the llamacpp family with FreeToken
|
||||
# (the PR587 ROCm engine then serving it had NO /metrics — the job
|
||||
# scraped 404s forever, a structural never-green) and RE-JOINED the same
|
||||
# day on the radiance engine: its exporter speaks vLLM-compatible names
|
||||
# plus radiance:* families, a different namespace — hence the separate
|
||||
# job below, not a re-add to the llamacpp pair. The netavark quirk above
|
||||
# is same-host only; plain LAN routing REACHES 192.168.0.235:7770.
|
||||
- job_name: "frank-small"
|
||||
metrics_path: "/metrics"
|
||||
static_configs:
|
||||
@@ -118,3 +119,15 @@ scrape_configs:
|
||||
labels:
|
||||
service: frank-large
|
||||
|
||||
# frank-primary (dankserver) — the radiance engine since 2026-10-05.
|
||||
# A DIFFERENT host, so plain LAN routing is correct here (the
|
||||
# host.containers.internal trick is same-host only): health answers at
|
||||
# 192.168.0.235:7770 from this bridge container. Radiance exports
|
||||
# vLLM-compatible names plus radiance:* families — NOT the llamacpp:*
|
||||
# namespace, so llm-fleet rules must not be pointed at this target.
|
||||
- job_name: "frank-primary"
|
||||
metrics_path: "/metrics"
|
||||
static_configs:
|
||||
- targets: ["192.168.0.235:7770"]
|
||||
labels:
|
||||
service: frank-primary
|
||||
|
||||
Reference in New Issue
Block a user