docs: frank-primary runs FreeToken (PR587 ROCm) — runbook, index, AGENTS, log

New runbook frank-primary-freetoken-pr587 (build laws, shape, measured
cutover numbers, TP=2 non-boot signature, rollback); LLM/index.md intro +
mermaid + active/retired entries; AGENTS.md fleet row + section + bench
and model-config mentions + runbook index; log.md entry. Seam pins the
measured KV pool 299968 with the multiple-of-64 guard.

NOTE: intended PGP signature skipped — key 0x76EA601A11B178D7 has no
secret key in this machine's keyring (fresh ~/.gnupg); the PGP identity
lives elsewhere.
This commit is contained in:
Frank committed 2026-10-05 19:35:31 -07:00
1 parent 6ba19f6f48
commit e2308dff71
5 files changed
+295 -139

No files matched your search

+59 -116
View File
@@ -24,7 +24,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 2048 (arena-free: the registry prices overhead flat up to its nbt 4096 anchor), MoE gather 32×64, **`--expert-precise-memory`** (fitted 5.25 GiB overhead, not the 7.60 GiB default), **`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`** (the plan's CUDA-graph term priced at the measured capture — 0.31 GiB then a NEGATIVE −0.06 GiB delta, so the 256 MiB analytic base was fiction → graph term 0.07 GiB), **`GPU_MAX_HW_QUEUES=2`**, **mounted cache dir** at `/root/.cache` (the image ships no `/root/.cache` at all; unmounted, every boot re-pays `torch.compile took 181.20 s`), **3 slots × 200704 ctx** at KV_BYTES 3,166,500,000/rank (~400,700 KV tokens at the engine's 7,891.8 B/token here) — the **pool, not the slot count, caps resident conversations** (measured live: `GPU KV cache size: 401,408 tokens, Maximum concurrency for 200,704 tokens per request: 2.00×`; the 3.15e9 pool below it measured 398,658/1.99×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. **util stays 0.982 BY LAW**: at `le=1` the worker init refuses `budget == total` with `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks before a single weight loads — measured crash loop 2026-10-03. Arena at shape G: **17.77 GiB / 7,762 slots measured**, KV 401,408 tokens / 2.00× concurrency, 0.89 GiB of real free VRAM per card held through a live 4,096-token generation (the ~3.5 GiB of driver-reported free VRAM at shape E was estimator slack — the registry's 9190 B/token KV reserve vs the real 7,892, and the 5.25 GiB overhead vs far less — NOT allocatable), and the client contract wired: `--reasoning-parser …
| Dankserver | `frank-primary` | **FreeToken PR #587** (branch `rocm714_torch214`, head `77a7201`; local image `frank-primary-freetoken:pr587-77a7201` built on the host from `scripts/docker/frank-primary-freetoken.Dockerfile` — digest-pinned rocm/pytorch rocm7.14 torch-2.11.0 base, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards, modelopt block-FP8, loaded straight from safetensors), **ONE R9700 (second card idle)**, all experts in **pinned host RAM** (~120 GiB) behind a GPU LRU (`--moe-cache-rate 0.10`), PLE table 47.7 GiB pinned (root unit + `LimitMEMLOCK=infinity`), **KV pool `--num-tokens 299968`** (24 KiB/token, multiple-of-64 law of the `qsa_sparse` auto backend), 2 slots, `--max-output-tokens 36864`, **parsers AUTO** (thinking → `reasoning_content`, tools qwen3.8 dialect), vision tower host-streamed. Measured: boot ~2.3 min cold / 92 s warm, decode **17.1 tok/s** single-stream; **TP=2 does not boot on ROCm** (pynccl JIT); `/metrics` 404 (Prometheus scrape reads down). Deploy gate reads the `/health` BODY (`status:"ok"`) — FreeToken 200s while loading. Rollback tag `frank-primary-vllm-shapeG-20261003` |7770|
| Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (35,865,565,184 B; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (measured: 8 slots at serial ~6 t/s/stream), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot AND `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` are both 400s naming the limit, never truncations. Serve passes NO `--ulimit nofile` (rootless runc cannot raise Dankclaw's 524288 hard limit — see Key Lessons). Cutover 2026-10-05|7770|
| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
@@ -260,7 +260,7 @@ merely wedged kernel usually still answers ARP.
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
- Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
`sudo systemctl stop frank-small && sudo systemctl mask frank-small`
- Dankserver `frank-primary` (root unit, tcclaviger vLLM on the 2x R9700):
- Dankserver `frank-primary` (root unit, FreeToken on one R9700):
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
2. Bench.
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
@@ -295,8 +295,9 @@ a role pointing at a provider the file does not define fails at call time. As
of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
`frank-primary` takes them through a BF16 mmproj loaded explicitly (this build never
autoloads projectors), and `frank-small` through
`frank-primary` takes them natively (FreeToken builds the qwen3.8 vision
tower and streams its block weights from host RAM two at a time by default),
and `frank-small` through
`mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the MiMo
GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
**200704** (resliced from 180224 on 2026-10-03), the per-slot window, not the
@@ -443,122 +444,63 @@ pairs with Q5 only; SGLang rollback in
`/home/dank/GitPackages/sglang-cu128/`, INT8 payload staged to
`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`).
### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
### frank-primary — FreeToken (PR #587 ROCm), Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
The **tcclaviger RDNA4 vLLM fork** (`docker.io/tcclaviger/vllm@sha256:da4029666…`,
digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
(~172.8 GiB, 131 shards) on **both** Radeon AI PRO R9700s (gfx1201), **TP=2**,
**fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
2026-10-03 the **shape-G** geometry (shape E + MNBT 2048 + the CUDA-graph plan
term at the measured floor): `--max-model-len 200704` × 3 slots with
`--kv-cache-memory-bytes 3,166,500,000`/rank (~400,700 KV tokens — the engine
prices this pool at 7,891.8 B/token here), util **0.982** (the law — see the
unbootable-1.0 lesson below), `--expert-precise-memory`,
`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`,
`GPU_MAX_HW_QUEUES=2` and a mounted `/root/.cache`. `models.yml` carries
`contextWindow: 200704` for this endpoint — the served window, not the model's
native 262144, so the harness compacts before vLLM answers a longer prompt
with a 400.
The **FreeToken engine at PR #587 head `77a7201`** (branch
`rocm714_torch214`, built on the host into the local image
`localhost/frank-primary-freetoken:pr587-77a7201` by
`scripts/docker/frank-primary-freetoken.Dockerfile` — the PR's own
docs/install_amd.md stack: digest-pinned
`rocm/pytorch:rocm7.14_ubuntu24.04_py3.12_pytorch_release_2.11.0` base =
torch 2.11.0+rocm7.14/triton 3.7, `--no-build-isolation --no-deps -e .`,
curated portable deps, `flashlib==0.3.0 --no-deps`, and a `libhiprtc.so`
dev-symlink the pip-ROCm wheel lacks) serving
**`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`** (~172.8 GiB, 131 shards,
standard modelopt block-FP8 — 75,264 `weight_scale_inv` tensors, no
`input_scale`, loaded directly from HF safetensors, no FTW conversion) on
**ONE R9700 (gfx1201)** — the second card sits idle. Plain `podman run`
through `scripts/serve-frank-primary.sh` under the root systemd unit
(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the
container name is the unit name (`frank-primary`; the vLLM-era
`frank-primary-qwen` is in the legacy cleanup list). Cutover 2026-10-05;
rollback tag `frank-primary-vllm-shapeG-20261003` (the tcclaviger vLLM trio
and its 362-line env lore live in that tag; the tcclaviger image stays in
dank's rootless storage).
**`--max-num-seqs 2` is impossible on this model.** Measured 2026-10-02 by
single-variable boot probe against the then-deployed shape: SEQS 2 fails at
BOTH MNBT 1024 and 2048 with `torch.fx.experimental.symbolic_shapes.Constraint
ViolationError: Constraints violated (L['query_start_loc'].size()[0]) …
specialized it to be a constant (3)`; SEQS 3 passes at both. `model_runner.py`
sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
`SEQS + 1` for any `MNBT ≥ SEQS` — MNBT only moves the compile RANGE
((1,1024) vs (1,2048)). `--enforce-eager` bypasses the path but drops CUDA
graphs. The same failure class killed the 2026-10-01 bf16 2×262144 shape.
**Shape**: all expert weights in **pinned host RAM** (~120 GiB) behind a
GPU LRU (`--moe-cache-rate 0.10`, ~12 GiB — the single-card ceiling given
the KV contract), the ~47.7 GiB PLE ngram table pinned in host RAM
(→ root unit + memlock law), **KV pool `--num-tokens 299968`**
(24 KiB/token f16 across the 12 full-attention layers ≈ 7.0 GiB; **must be
a multiple of 64** — the auto attention backend is `qsa_sparse`, which
overrides page size to 64; 300,000 was refused at first boot), 2 decode
slots, `--max-output-tokens 36864` (the harness maxTokens; requests that
omit one get this default). **Parsers are AUTO**: thinking lands in
`reasoning_content` natively, tool calls in the qwen3.8 dialect — models.yml
flips `reasoningContentField` to `reasoning_content` on the harness side.
Vision works (models.yml keeps `input: [text, image]`); the vision tower's
blocks stream from host RAM two at a time by default.
**The expert arena has THREE reachable levers: `max_model_len`, the overhead
term, and the CUDA-graph estimate.**
The fork's planner sizes expert cache as `budget − non-expert − KV reserve −
graphs − overhead`, and the KV reserve is registry-priced at 9190 B/token ×
`max_model_len`: 262144 → 2.48 GiB reserve → arena 14.30 GiB / **6,248 slots**;
200704 → 1.91 GiB → 14.87 GiB / **6,496 slots**. Δreserve 0.57 GiB = Δarena
0.57 GiB. The 6.1e9 → 3.15e9 pool cut bought **zero** arena; it buys runtime
headroom (the shape it replaced died with `0 bytes free` on a real 995-token
request). The second lever is `--expert-precise-memory`, **ON since
2026-10-02**: it swaps the registry's default overhead (7.60 GiB = fitted +
compile/graph-capture headroom) for the fitted figure, and the 2.35 GiB
difference lands in the arena — measured plan lines at 200704 × 3: overhead
5.25 GiB, expert cache **17.22 GiB / 7,522 slots** (+1,026 / +16 %), swap region
13.7 GiB / 5,986 slots, boot healthy in 420 s. The pool stayed at 3.15e9
deliberately: because the pool is not an arena lever, cutting it would cost
concurrency (a 2.2e9 probe reports 277,686 tokens / **1.38×**) and buy no slots.
The arena (2.344 MiB per slot) is the decode-critical region — cutting it for KV
headroom collapses the offload hit rate.
**Measured at cutover**: boot ~2.3 min cold-ish / 92 s warm → /health
status ok; decode **17.1 tok/s** single-stream temp 0 (the vLLM incumbent
measured ~30 tok/s on TWO cards; the KV contract caps the single-card LRU,
and **TP=2 exists but does not boot on ROCm** — the TP allreduce JIT
(`pynccl.cu`) can't resolve `cuda_runtime.h` for gfx1201; probed
2026-10-06, signature in the runbook). `/metrics` is a 404 — the Dankclaw
Prometheus llm-fleet scrape of :7770 reads down until repointed.
`ft ctl cache --kv/--moe` resizes pools live (grow only; shrink needs a
restart); `ft ctl stats` reports occupancy.
**Free VRAM on this endpoint is NOT arena headroom (2026-10-03 audit).**
`rocm-smi` reads ~3.5 GiB free per card after hours of real traffic, but the
planner charges `budget = total × util` against terms that overstate reality:
the KV reserve is registry-priced at 9190 B/token (`1.91 GiB` at 200704) while
the real pool bills 2.93 GiB at 7,892 B/token, and the precise overhead (5.25
GiB) is far above the idle-measured figure. So the visible free space is the
estimator gap, not allocatable arena. `gpu_memory_utilization` is a real arena
lever (each +0.01 = +0.32 GiB) but **0.982 is the law, and 1.00 is UNBOOTABLE
on this fork — measured 2026-10-03**: the worker init demands
`free > budget`, and at `le=1` budget == device total, so both ranks raise
`ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)` before a
single weight loads, twice (2-restart crash loop, zero payload progress). The
CLI field accepting `le=1` is the trap; Frank's 0.982 is where it always
ships. What IS reachable from flags: `VLLM_CUDAGRAPH_MEM_BASE_MB` /
`_PER_GRAPH_MB`, read at call time by `estimate_cudagraph_memory()` — the
analytic default (256 MiB + 52 MiB/graph) charged 0.55 GiB where the engine's
own post-capture line reported 0.31 GiB, and a second boot (with the plan
already charging 0.32 GiB) reported **-0.06 GiB**, so the 256 MiB base pool is
pure over-charge: shape G charges `0 base + 12 MiB/graph` = 0.07 GiB, which
still prices per graph for a future 18-graph spec-decode shape (0.21 GiB).
Verify every deploy by comparing the `Estimated CUDA graph memory`
line with `Graph capturing finished … took X GiB`; if capture exceeds the
estimate, raise `_PER_GRAPH_MB` back toward 52. MNBT 1024 → 2048 costs ZERO
arena: the registry overhead is `base + 0.22 GiB × max(0,(nbt−4096)/1024)`
anchored at nbt 4096, so every rung ≤4096 charges the same 5.25 GiB — and the
fitted figure was fitted AT 4096.
**Deploy gate law (changed for FreeToken)**: `/health` returns **200 with
`{"status":"loading"}`** while the engine loads — the FastAPI front end
binds before the scheduler is up, so a bare `curl` false-greens (measured:
"healthy after 6 s" mid-page-in). The deployer loops until the body says
`"status":"ok"`; the MainPID stray-port guard is unchanged. The image gate
is `podman image exists` (a local build carries no RepoDigests — the old
digest grep refused exactly this shape; pinning lives in the Dockerfile).
**A container's compile cache does NOT survive `podman rm -f`, and on this
endpoint that is 181 s per boot.** Measured 2026-10-02: the image ships **no
`/root/.cache` at all** (fresh container: `No such file or directory`); after a
boot it holds `/root/.cache/vllm` at **1,154,714,333 B in 4,665 files** plus
`/root/.cache/comgr` at 229 MB, all in the ephemeral layer, and the engine
prints `torch.compile took 181.20 s in total` (173.19 s of it one graph,
`Compiling a graph for compile range (1, 1024)`). Every deploy, restart and
reboot re-pays it. The serve script now mounts
`FRANK_PRIMARY_CACHE_DIR` at `/root/.cache` — the parent, so the vLLM AOT /
torch_compile tree (`VLLM_CACHE_ROOT`, whose default is
`/root/.cache/vllm`) and the comgr HIP-kernel JIT cache both persist, and
nothing in the image is hidden because the image has no such directory. The
compile RANGE is part of the cache key, so an MNBT change costs one
recompilation, not a permanent tax. Same class of fix as the recipe's
`/cache/{triton,vllm,inductor}` volumes. **Proven live 2026-10-02 on
consecutive boots:** cold `Compiling a graph for compile range (1, 1024) takes
179.24 s` / `torch.compile took 188.26 s`; warm `Directly load the compiled
graph(s) for compile range (1, 1024) from the cache, took 2.143 s` /
`torch.compile took 3.97 s in total` / `init engine (profile, create kv cache,
warmup model) took 41.13 s (compilation: 3.98 s)`. Time to `/health` is
dominated by the 172.8 GiB payload load, so the visible win is the ~184 s of
inductor/AOT build, not a proportional boot cut.
Measured 2026-09-29 tuning campaign (fp8 + MTP4 era, since retired): decode
**37.4-40.6 tok/s, acceptance ~54 %**; the 256k ctx arm measured 34.1-36.5;
2-way concurrency sums BELOW parity (25.49 vs 26.8 single) — slots are a
capacity feature, not a throughput one. V-only q8 KV is NOT possible via
config: stock `--kv-cache-dtype` is K+V together; a V-only quant needs a fork
patch.
Unit and self-healing: root systemd unit (`scripts/frank-primary.service`,
`User=dank`, `LimitMEMLOCK=infinity`), `ExecStop=podman stop -t 120
frank-primary-qwen`. The serve script drives `podman run` directly, gates
the digest-pinned image, the payload's `config.json` and its exact
131-shard count and the GPU device nodes, clears stale container
identities, then blocks on `podman wait`. `deploy_llm_primary` gates the
same, waits out :7770, relinks/enables/restarts the unit and health-gates
with the MainPID check. Boot cost ~7 min warm (the 172.8 GiB payload load; the
~184 s inductor/AOT build inside it is what the mounted cache now skips — see
the compile-cache lesson below); health returns 503 while loading.
Runbook: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Runbook: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
## Key Lessons Learned
@@ -1005,7 +947,7 @@ Active:
|---------|---------|
| frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
| frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q6_K, 2×200,704 f16 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md` |
| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
| frank-primary (FreeToken PR #587 ROCm, Qwen3.8-Flash-Next FP8, 1×R9700, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1014,6 +956,7 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook |
|---------|------|--------|---------|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |