docs(vault): one fact, one owner — repair the doubled pages, single-source the facts
Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
This commit is contained in:
1 parent
3880503057
commit
a5fe045b58
122 files changed
+2412
-2037
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
type: reference
|
||||
title: frank-medium vllm-radiance MXFP6 Runbook
|
||||
description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
|
||||
description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -14,6 +14,7 @@ tags:
|
||||
- rccl
|
||||
- pcie-atomics
|
||||
- quadlet
|
||||
- retired-endpoint
|
||||
timestamp: 2026-09-21
|
||||
---
|
||||
|
||||
@@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
|
||||
`--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
|
||||
`ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
|
||||
DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
|
||||
drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
|
||||
decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
|
||||
`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
|
||||
back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
|
||||
vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
|
||||
drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
|
||||
below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
|
||||
rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
|
||||
`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
|
||||
rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
|
||||
`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
|
||||
image back to `localhost/radiance-vllm:0.9.3` (re-pull
|
||||
`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
|
||||
486,464 tokens with bf16 KV and held the full 262,144 window).
|
||||
|
||||
**Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
|
||||
@@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
|
||||
with growing history) shows what happens at that limit: turns alternate between
|
||||
**~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
|
||||
and must be prefilled again, because two families per card plus their cached
|
||||
prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
|
||||
both replicas steady — and per-family pinning kept the fast turns on the replica
|
||||
that holds the prefix.
|
||||
prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
|
||||
turns on the replica that holds the prefix.
|
||||
|
||||
Read that as a capacity budget, not a defect: two concurrent large-context
|
||||
sessions per card are comfortable, three or more of that size trade turns
|
||||
@@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here.
|
||||
the same image and kernel 7.0.0-30 failing identically.
|
||||
- `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
|
||||
this page compares against.
|
||||
---
|
||||
Reference in new issue
Block a user