docs: frank-large vLLM MiMo-BF16 runbook; retire the gufo runbook
New runbook carries the roof math, the measured three-arm spec table, the greedy-text numerics trap, the reasoning-field contract change and the rollback tag. LLM index and AGENTS.md fleet table, bench policy and runbook index follow; the frank-small index entries catch up to the SGLang cutover (entry was missed on 2026-10-01). Work log records the cutover.
This commit is contained in:
Frank committed
1 parent
dab483c79f
commit
1127a52078
5 files changed
+330
-133
No files matched your search
@@ -12,7 +12,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
|
||||
- **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
|
||||
via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
|
||||
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (stock llama.cpp on
|
||||
Alloy, Pi-hole) plus TWO LLM services: `frank-large` (vLLM TheRock on
|
||||
the gfx1151 iGPU) and `frank-small` (SGLang on the RTX 3090). Caddy on
|
||||
Dankclaw reverse-proxies both hosts.
|
||||
- **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
|
||||
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
| Target | Service | Runtime | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **bf16 KV** (`--kv-cache-dtype auto`), **no speculative decode**, MNBT 2048, MoE gather 32×64, **2 seqs × 262144 ctx (the model's full native window)** at KV_BYTES 7,300,000,000/rank (bf16 measures 13,839 B/token), with **`--expert-precise-memory`** — the fitted 5.25 GiB overhead instead of the 7.60 GiB default, which hands the compile/graph-capture headroom to the expert arena (~7,278 slots vs 6,248) — util 0.98, and the client contract wired: `--reasoning-parser qwen3` (thinking in `reasoning`, never inline), `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, default chat-template kwargs `enable_thinking` + `preserve_thinking`. Capacity history: five full-length contexts boot but leave ~0 bytes free at runtime (a real request with `max_tokens=261149` killed the worker: `Tried to allocate 50.00 MiB ... 0 bytes free`); six never fit (6×241664 asked 10.70 GiB vs 9.18 GiB free). DCP cannot help here — it requires TP > the model's 2 KV heads, i.e. four GPUs. fp8+MTP4 shape restorable from git tag `frank-primary-fp8-mtp-20261001` | 7770 |
|
||||
| Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | `unsloth/Qwen3.8-27B-GGUF` **UD-Q4_K_XL** (27B dense, **vanilla — no longer abliterated**, 16.35 GiB, Apache-2.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), **4 sessions x 163840 ctx, unquantized (f16) KV** — gufo has no KV-quant lever — greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only. Swapped 2026-10-01 from ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB): ~13 GiB less weights is what funds four f16 sessions, at the cost of the uncensored behaviour | 7770 |
|
||||
| Dankclaw | `frank-large` | vLLM 0.30 in the digest-pinned `docker.io/kyuz0/vllm-therock-gfx1151` image (TheRock: ROCm 10 / torch 2.11, Triton Gated DeltaNet kernels; plain `podman run`, root systemd unit) | `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` **full BF16** (9B dense, 17.53 GiB, 4 shards — the arch frank-small serves as INT8 on the 3090), **cross-model MTP spec decode from `Qwen/Qwen3.5-9B` (2 draft tokens; 19.8 t/s decode measured, +51% over the 13.1 t/s base, which is ~94% of the LPDDR5x BF16 roof)**, 4 seqs x 262144 native ctx at util 0.45, qwen3 reasoning + qwen3_coder tool parsers, text-only (`--language-model-only`). Cut over 2026-10-01 from gufo; rollback tag `frank-large-gufo-q4kxl-20261001` | 7770 |
|
||||
| Dankclaw | `frank-small` | SGLang in a self-contained python3.12 venv (root systemd unit; the venv carries its own CUDA 13 stack — see the runbook) | letechlead MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid, 9B dense, 12.0 GiB, INT8 W8A16 AutoRound, in-checkpoint vision tower, NO MTP weights shipped so no spec decode) on the RTX 3090, fp8 KV, pinned 606,208-token pool / 2 slots of 262,144 (native window; 2026-10-01 reslice from 4 x 151,552 for compaction headroom), bf16 mamba states, eager prefill, 83.9 tok/s decode | 7771 |
|
||||
|
||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||
@@ -209,14 +209,14 @@ archive.
|
||||
**STOP the production service BEFORE benchmarking** — these servers serve
|
||||
real users, and a bench beside a live service starves the GPU and corrupts
|
||||
results. Use the runtime's own harness: `llama-bench` for the llama.cpp
|
||||
endpoints, `gufo bench` for frank-large. Rolling your own bench
|
||||
endpoints; frank-large's vLLM shape is measured by replaying real text through the live server and reading `Server-Timing`. Rolling your own bench
|
||||
requires asking first.
|
||||
Scratch ports are not used on LLM hosts — a
|
||||
test process cannot share the single GPU anyway; validate against the service
|
||||
itself.
|
||||
|
||||
**`gufo bench` cannot rank speculative-decode settings, and will invert the
|
||||
ranking.** Measured 2026-09-28 on frank-large at depth 32768: its corpus
|
||||
(A gufo-era law; frank-large is vLLM since 2026-10-01, but the rollback tag keeps it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
|
||||
Measured 2026-09-28 on frank-large at depth 32768: its corpus
|
||||
reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
|
||||
draft widths, i.e. the same highly-predictable text every run, and 93 % of
|
||||
every proposal accepted. It reported 29–37 t/s decode where real traffic
|
||||
@@ -241,7 +241,7 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
|
||||
merely wedged kernel usually still answers ARP.
|
||||
|
||||
1. Stop and mask (self-healing would restart it):
|
||||
- Dankclaw `frank-large` (root unit, gufo on the iGPU):
|
||||
- Dankclaw `frank-large` (root unit, vLLM podman on the iGPU):
|
||||
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
|
||||
- Dankclaw `frank-small` (root unit, SGLang on the 3090):
|
||||
`sudo systemctl stop frank-small && sudo systemctl mask frank-small`
|
||||
@@ -281,8 +281,8 @@ of 2026-10-01 both `frank-small` and `frank-primary` accept images
|
||||
(`input: [text, image]`) — frank-primary loads the BF16 mmproj explicitly
|
||||
(this build never autoloads projectors), and frank-small's MiMo payload
|
||||
bakes its vision tower INTO the checkpoint (no projector file at all);
|
||||
`frank-large` remains text-only because its gufo deployment loads no
|
||||
vision projector. Note also that
|
||||
`frank-large` stays text-only because its vLLM deployment passes
|
||||
`--language-model-only` — the MiMo vision tower is in the checkpoint; the flag is the lever. Note also that
|
||||
the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
|
||||
parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
|
||||
models.yml sets `reasoningContentField` per endpoint to match.
|
||||
@@ -292,111 +292,75 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
|
||||
audio encoder, and left again with the Gemma4 retirement on 2026-10-01
|
||||
(the MiMo tower is vision-only).
|
||||
|
||||
### frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (Dankclaw, ACTIVE)
|
||||
### frank-large — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (Dankclaw, ACTIVE)
|
||||
|
||||
**gufo** (`gufo-org/gufo`) v0.1.1 — "the Strix Halo inference engine", a
|
||||
vertical engine built for exactly this hardware (Ryzen AI MAX+ 395 /
|
||||
Radeon 8060S, `gfx1151`) — as a **root systemd unit**
|
||||
(`scripts/frank-large.service`, `User=dank`), serving
|
||||
`ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s `UD-Q8_K_XL`
|
||||
(27B dense, 29.3 GiB, sha256-verified; **abliterated/uncensored**, and under
|
||||
the *non-OSI* SWIFT OPEN LICENSE 1.0) on port 7770 from the gfx1151 iGPU.
|
||||
One session, full 262,144 ctx (the model's native window, `SESSIONS=1`),
|
||||
DFlash2 spec decode (the z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` drafter, gufo's
|
||||
**adaptive** block policy, `--draft-tokens 7`), greedy sampling, reasoning
|
||||
extracted to `reasoning_content`, and **no vision projector**.
|
||||
**vLLM 0.30** in the **digest-pinned** `docker.io/kyuz0/vllm-therock-gfx1151`
|
||||
image (image ID `2d24a8c5a3fd`, digest `sha256:98c1f3ac9833…` — TheRock
|
||||
build: ROCm 10.0.0 / torch 2.11.0, Triton Gated DeltaNet kernels,
|
||||
`vllm/v1/spec_decode/dflash.py` present) — as a **root systemd unit**
|
||||
(`scripts/frank-large.service`, `User=dank`, `Restart=always`,
|
||||
`ExecStop=podman stop -t 120`, `TimeoutStopSec` 180 > the stop grace),
|
||||
serving `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` in **full BF16** (9B dense
|
||||
qwen3_5 hybrid, 17.53 GiB, 4 sha256-verified shards) on port 7770 from the
|
||||
gfx1151 iGPU. Official vLLM images are CUDA-only; this fork is the fleet's
|
||||
gfx1151 path. `scripts/serve-frank-large.sh` gates image-by-repo-digest →
|
||||
payload (config + index + exact shard count) → spec draft (accepts BOTH
|
||||
weight layouts: single `model.safetensors` for dflash, sharded index for
|
||||
the MTP head) → whole `/dev/dri` (never individual renderD nodes), sweeps
|
||||
the bench identity `vllm-mimo`, then `exec podman wait` so a container
|
||||
death trips `Restart=always`. Boot ~200–240 s.
|
||||
|
||||
**Two swaps landed on 2026-09-28**, both same day: the engine (from the
|
||||
unsloth llama.cpp b11030 prebuilt, which had replaced Kairic Edge ROCmFPX)
|
||||
and then the payload (from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on
|
||||
disk and the payload-only rollback). The two targets are structurally
|
||||
interchangeable for gufo — same `qwen35` architecture, the same 866 tensors
|
||||
in the same order, the same per-tensor Q8_0/F32/BF16 layout, the same MTP
|
||||
head at `blk.64` — verified by parsing and diffing both GGUF headers before
|
||||
either was loaded. **The DFlash2 drafter did not move**, so it has its own
|
||||
`DRAFTER_DIR` and the deployer's stray-run sweep keys on the drafter's
|
||||
filename, not the target's.
|
||||
**The shape is chosen by roof math, and the spec decode is the
|
||||
performance feature.** BF16 decode is LPDDR5x-bandwidth-bound here:
|
||||
13.1 t/s measured = ~94% of the roof for 17.5 GiB of weights — a quant,
|
||||
not a bug. The only way past it is multiple tokens per weight pass.
|
||||
Measured 2026-10-01 (one session, temp 0, 28,647-token prefill / 512
|
||||
decode): base **17,876 t/s prefill / 13.1 decode**; +DFlash-8 7,976 /
|
||||
17.4 (acceptance 59%/26% per position); **+MTP-2 7,380 / 19.8
|
||||
(acceptance 71%/44%, token rate 57.4%)** — deployed arm.
|
||||
`num_speculative_tokens` beyond 2 is UNMEASURED; both spec arms pay
|
||||
~55–59% of prefill. KV grows only on the 8 full-attention layers
|
||||
(32 KiB/token bf16); the 24 GDN layers hold fixed ~25 MiB state per
|
||||
sequence — so 4 x 262,144 at util 0.45 fits the GTT aperture easily.
|
||||
|
||||
**The runtime is built from source on the host** —
|
||||
`/home/dank/GitPackages/gufo-v0.1.1/`, reproducible with
|
||||
`scripts/build-gufo.sh` (`--verify` re-checks an installed tree). gufo ships
|
||||
no prebuilt Linux binaries and the v0.1.0/v0.1.1 release assets are empty.
|
||||
Unlike the retired llama.cpp prebuilt, which bundled its own ROCm, **gufo
|
||||
links the host ROCm 7.2.4 through its RPATH**, so the HIP dev packages must
|
||||
stay installed. Two host workarounds the build script encodes:
|
||||
**The MTP head is CROSS-MODEL — the fleet's first.** MiMo ships ZERO
|
||||
`mtp.*` tensors (the reason frank-small's SGLang NEXTN is a 0.00 ghost:
|
||||
SGLang reads the head from the TARGET checkpoint); vLLM reads the draft
|
||||
from a separate path, so `--speculative-config
|
||||
'{"method":"mtp","model":"/models/Qwen3.5-9B","num_speculative_tokens":2}'`
|
||||
drafts from the distill's SOURCE model, and 71%/44% acceptance is the
|
||||
measured proof that the distillation preserved Qwen3.5 hidden-state
|
||||
structure. The alternative drafter (`z-lab/Qwen3.5-9B-DFlash`, fetched;
|
||||
its "PR #40898 unmerged" card claim is false — dflash ships in vLLM 0.30)
|
||||
stays on disk: `FRANK_LARGE_SPEC=dflash` + repoint `SPEC_DIR`.
|
||||
|
||||
- **Ubuntu's `librocwmma-dev` must be absent.** It installs a truncated
|
||||
rocWMMA header set into `/usr/include` that shadows AMD's complete one
|
||||
and breaks the build with `'internal/accessors.hpp' file not found`. A
|
||||
`-I` cannot fix it — in HIP mode the host pass searches the system
|
||||
directories first. The package has no reverse dependencies.
|
||||
- **`--gcc-install-dir` must name GCC 15.** Ubuntu 26.04 ships
|
||||
`/usr/include/c++/16`, which the ROCm clang prefers; its `<atomic_wait.h>`
|
||||
needs a `std::__detail::__notify_impl` symbol version the installed
|
||||
`libstdc++.so.6` (GLIBCXX_3.4.35) does not export, and the link fails.
|
||||
**Spec decode does NOT preserve greedy text here (measured).**
|
||||
Temperature-0 completions differ across arms (bf16 batched verification
|
||||
reorders reductions; near-tie argmaxes diverge and cascade). vLLM spec
|
||||
decode is distribution-preserving, not bitwise-greedy-preserving — do not
|
||||
chase text drift between arms as a regression.
|
||||
|
||||
`gufo diagnose` is the preflight and must report `[PASS]` with `gfx1151`.
|
||||
It also self-caps compute queues (`GPU_MAX_HW_QUEUES`, 2 resident for
|
||||
`serve llm`) after reading `/sys/class/kfd/kfd/proc/*/queues/*/type`, which
|
||||
matters because ComfyUI holds queues on the same device; four HIP processes
|
||||
is the device ceiling.
|
||||
**Client contract differs from every llama.cpp-era endpoint:** vLLM emits
|
||||
thinking in **`reasoning`**, not `reasoning_content` — `models.yml` sets
|
||||
`reasoningContentField: "reasoning"` for this provider (miss it and
|
||||
thinking vanishes from transcripts). Tools ride
|
||||
`--enable-auto-tool-choice --tool-call-parser qwen3_coder`. The harness
|
||||
extraBody moved to the Qwen3.5-family recipe (0.7/0.8) with
|
||||
`chat_template_kwargs.{enable_thinking,preserve_thinking}` — a top-level
|
||||
`preserve_thinking` is ignored by vLLM. Env keys stay
|
||||
`FRANK_LARGE_`-prefixed (HIP reads bare `TEMP`/`HOST`/`GPU_MAX_HEAP_SIZE`).
|
||||
|
||||
**The client contract changed, and the chat template is the sharp edge.**
|
||||
**gufo has no Jinja engine at all** — `src/models/qwen/chat_template.cpp` is
|
||||
587 lines of hand-written C++ that appends strings, with no
|
||||
`--chat-template-file`, no env override, and no template body to point at.
|
||||
`scripts/qwen-sharp-v22.4.0.jinja` therefore **cannot** be supplied to
|
||||
frank-large: that is a port, not a config line. Given the choice on
|
||||
2026-09-28 the decision was to ship on gufo's stock renderer and accept the
|
||||
loss — the `terse` system-prompt injection, the in-band `<|think_*|>`
|
||||
control, the `json|xml` `tool_call_format` switch, the tool-argument
|
||||
truncation and `add_vision_id` prefixes are all gone. gufo's own renderer
|
||||
does keep the effort instruction strings and `preserve_thinking`, which are
|
||||
what the harness depends on. **This is the thing to watch**: the endpoint
|
||||
now runs an *abliterated* model without the prompt that tells it to stop
|
||||
waffling. The cheap fix, if verbosity shows up, is to send the `terse` text
|
||||
as an ordinary system message from `models.yml` — not a template change.
|
||||
`reasoning_effort` accepts the full OpenAI set (all six levels verified
|
||||
200), so `models.yml`'s `reasoningEffortMap` fold is no longer *required* —
|
||||
it is kept deliberately, so that removing it does not silently move every
|
||||
`high` role from `xhigh` to `high` effort inside a swap. And there is no
|
||||
KV-quantization or prompt-cache-RAM flag; prompt reuse is on by default and
|
||||
`--cache-disk` is opt-in and off.
|
||||
Fetch: `scripts/download-frank-large-mimo.sh` (target),
|
||||
`scripts/download-frank-large-qwen35.sh` (MTP source),
|
||||
`scripts/download-frank-large-dflash.sh` (dflash alternative) — all
|
||||
`-f` + byte-count + LFS sha256 verified, `--verify-only` supported, no
|
||||
token on the command line.
|
||||
|
||||
**Read the perf trade before assuming a win.** gufo's own Q8_K_XL tables
|
||||
(vs llama.cpp b11069, DFlash2) show prefill +37 % to +80 %, cold load 4.5×
|
||||
faster, decode +7 % at zero depth and +40 % at 4 users — but **−14 % at
|
||||
65,536 tokens of depth and −24 % at 131,072**. On the 8-bit target the
|
||||
crossover is ~32k of cached context, and frank-large runs one 262,144-token
|
||||
conversation, so long agent sessions sit on the losing side of that line.
|
||||
The Q4_K_XL target shows no such regression; revisiting the quantization is
|
||||
a separate decision from the runtime. Peak memory is 50.58 GiB at the full
|
||||
262,144 context against the iGPU's 114 GiB GTT aperture.
|
||||
|
||||
The serve script (`scripts/serve-frank-large.sh`) validates runtime → target
|
||||
GGUF → drafter and exits with a pointer message if any is missing, then
|
||||
`exec`s `gufo serve --host 0.0.0.0 --port 7770 --sessions 1 llm --model
|
||||
…--served-model-name frank-large --context 262144 --speculative dflash2 …`.
|
||||
The deployer's `deploy_llm_large` gates on the **runtime binary** as well
|
||||
as both payload paths, and returns before it stops anything.
|
||||
|
||||
Fetch: `scripts/download-frank-large-swift.sh` (the target; per-file byte
|
||||
count + sha256 against the Hub LFS oid, no auth header — the repo is public
|
||||
and a token on the curl command line is world-readable) and
|
||||
`scripts/download-frank-large.sh` (the DFlash2 drafter, which did not move).
|
||||
Both take `--verify-only`.
|
||||
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md`.
|
||||
Rollback, two ways: **payload only** — re-fetch with
|
||||
`scripts/download-frank-large.sh` and repoint `MODEL_FILE` at
|
||||
`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`; that
|
||||
29.3 GiB target was **reclaimed on 2026-09-28** (staged to
|
||||
`~/.AGENTTRASH/20260928-retired-models/`, restorable from there), so it is
|
||||
no longer a one-minute flip. **Engine** —
|
||||
[[frank-large-unsloth-qwen38-27b-q8xl]], the b11030 prebuilt at
|
||||
`/home/dank/GitPackages/unsloth-llama-b11030/`; the only way to get
|
||||
Qwen-Sharp back. The DFlash2 drafter was never reclaimed and still lives in
|
||||
`Qwen3.8-27B-UD-Q8_K_XL/`.
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md`.
|
||||
Rollback: git tag **`frank-large-gufo-q4kxl-20261001`** — gufo v0.1.1
|
||||
(`scripts/build-gufo.sh` stays on disk for it), the Q4_K_XL Qwen3.8-27B
|
||||
payload and the DFlash2 drafter all remain on the host; restore the tag's
|
||||
unit/env/serve trio and redeploy.
|
||||
|
||||
### frank-small — SGLang on the RTX 3090 (Dankclaw, ACTIVE)
|
||||
|
||||
@@ -827,7 +791,7 @@ Active:
|
||||
|
||||
| Service | Runbook |
|
||||
|---------|---------|
|
||||
| frank-large (gufo Qwen3.8-27B UD-Q8_K_XL, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md` |
|
||||
| frank-large (vLLM TheRock MiMo-V2.6 BF16 + cross-model MTP, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md` |
|
||||
| frank-small (SGLang MiMo-V2.6 INT8, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md` |
|
||||
| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8 + MTP4, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
@@ -837,6 +801,7 @@ Retired (rollback targets and history):
|
||||
|
||||
| Runtime | Host | Window | Runbook |
|
||||
|---------|------|--------|---------|
|
||||
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01 | `frank-large-gufo-qwen38-27b.md` |
|
||||
|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
|
||||
| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
|
||||
| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
|
||||
|
||||
Reference in new issue
Block a user