docs: frank-primary runs FreeToken (PR587 ROCm) — runbook, index, AGENTS, log
New runbook frank-primary-freetoken-pr587 (build laws, shape, measured cutover numbers, TP=2 non-boot signature, rollback); LLM/index.md intro + mermaid + active/retired entries; AGENTS.md fleet row + section + bench and model-config mentions + runbook index; log.md entry. Seam pins the measured KV pool 299968 with the multiple-of-64 guard. NOTE: intended PGP signature skipped — key 0x76EA601A11B178D7 has no secret key in this machine's keyring (fresh ~/.gnupg); the PGP identity lives elsewhere.
This commit is contained in:
1 parent
6ba19f6f48
commit
e2308dff71
5 files changed
+295
-139
No files matched your search
@@ -24,7 +24,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
|
||||
| Target | Service | Runtime | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 2048 (arena-free: the registry prices overhead flat up to its nbt 4096 anchor), MoE gather 32×64, **`--expert-precise-memory`** (fitted 5.25 GiB overhead, not the 7.60 GiB default), **`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`** (the plan's CUDA-graph term priced at the measured capture — 0.31 GiB then a NEGATIVE −0.06 GiB delta, so the 256 MiB analytic base was fiction → graph term 0.07 GiB), **`GPU_MAX_HW_QUEUES=2`**, **mounted cache dir** at `/root/.cache` (the image ships no `/root/.cache` at all; unmounted, every boot re-pays `torch.compile took 181.20 s`), **3 slots × 200704 ctx** at KV_BYTES 3,166,500,000/rank (~400,700 KV tokens at the engine's 7,891.8 B/token here) — the **pool, not the slot count, caps resident conversations** (measured live: `GPU KV cache size: 401,408 tokens, Maximum concurrency for 200,704 tokens per request: 2.00×`; the 3.15e9 pool below it measured 398,658/1.99×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. **util stays 0.982 BY LAW**: at `le=1` the worker init refuses `budget == total` with `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks before a single weight loads — measured crash loop 2026-10-03. Arena at shape G: **17.77 GiB / 7,762 slots measured**, KV 401,408 tokens / 2.00× concurrency, 0.89 GiB of real free VRAM per card held through a live 4,096-token generation (the ~3.5 GiB of driver-reported free VRAM at shape E was estimator slack — the registry's 9190 B/token KV reserve vs the real 7,892, and the 5.25 GiB overhead vs far less — NOT allocatable), and the client contract wired: `--reasoning-parser …
|
||||
| Dankserver | `frank-primary` | **FreeToken PR #587** (branch `rocm714_torch214`, head `77a7201`; local image `frank-primary-freetoken:pr587-77a7201` built on the host from `scripts/docker/frank-primary-freetoken.Dockerfile` — digest-pinned rocm/pytorch rocm7.14 torch-2.11.0 base, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards, modelopt block-FP8, loaded straight from safetensors), **ONE R9700 (second card idle)**, all experts in **pinned host RAM** (~120 GiB) behind a GPU LRU (`--moe-cache-rate 0.10`), PLE table 47.7 GiB pinned (root unit + `LimitMEMLOCK=infinity`), **KV pool `--num-tokens 299968`** (24 KiB/token, multiple-of-64 law of the `qsa_sparse` auto backend), 2 slots, `--max-output-tokens 36864`, **parsers AUTO** (thinking → `reasoning_content`, tools qwen3.8 dialect), vision tower host-streamed. Measured: boot ~2.3 min cold / 92 s warm, decode **17.1 tok/s** single-stream; **TP=2 does not boot on ROCm** (pynccl JIT); `/metrics` 404 (Prometheus scrape reads down). Deploy gate reads the `/health` BODY (`status:"ok"`) — FreeToken 200s while loading. Rollback tag `frank-primary-vllm-shapeG-20261003` |7770|
|
||||
| Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (35,865,565,184 B; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (measured: 8 slots at serial ~6 t/s/stream), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot AND `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` are both 400s naming the limit, never truncations. Serve passes NO `--ulimit nofile` (rootless runc cannot raise Dankclaw's 524288 hard limit — see Key Lessons). Cutover 2026-10-05|7770|
|
||||
| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
|
||||
|
||||
@@ -260,7 +260,7 @@ merely wedged kernel usually still answers ARP.
|
||||
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`
|
||||
- Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
|
||||
`sudo systemctl stop frank-small && sudo systemctl mask frank-small`
|
||||
- Dankserver `frank-primary` (root unit, tcclaviger vLLM on the 2x R9700):
|
||||
- Dankserver `frank-primary` (root unit, FreeToken on one R9700):
|
||||
`sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
|
||||
2. Bench.
|
||||
NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
|
||||
@@ -295,8 +295,9 @@ a role pointing at a provider the file does not define fails at call time. As
|
||||
of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
|
||||
image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
|
||||
ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
|
||||
`frank-primary` takes them through a BF16 mmproj loaded explicitly (this build never
|
||||
autoloads projectors), and `frank-small` through
|
||||
`frank-primary` takes them natively (FreeToken builds the qwen3.8 vision
|
||||
tower and streams its block weights from host RAM two at a time by default),
|
||||
and `frank-small` through
|
||||
`mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the MiMo
|
||||
GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
|
||||
**200704** (resliced from 180224 on 2026-10-03), the per-slot window, not the
|
||||
@@ -443,122 +444,63 @@ pairs with Q5 only; SGLang rollback in
|
||||
`/home/dank/GitPackages/sglang-cu128/`, INT8 payload staged to
|
||||
`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`).
|
||||
|
||||
### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
|
||||
### frank-primary — FreeToken (PR #587 ROCm), Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
|
||||
|
||||
The **tcclaviger RDNA4 vLLM fork** (`docker.io/tcclaviger/vllm@sha256:da4029666…`,
|
||||
digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
|
||||
(~172.8 GiB, 131 shards) on **both** Radeon AI PRO R9700s (gfx1201), **TP=2**,
|
||||
**fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
|
||||
2026-10-03 the **shape-G** geometry (shape E + MNBT 2048 + the CUDA-graph plan
|
||||
term at the measured floor): `--max-model-len 200704` × 3 slots with
|
||||
`--kv-cache-memory-bytes 3,166,500,000`/rank (~400,700 KV tokens — the engine
|
||||
prices this pool at 7,891.8 B/token here), util **0.982** (the law — see the
|
||||
unbootable-1.0 lesson below), `--expert-precise-memory`,
|
||||
`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`,
|
||||
`GPU_MAX_HW_QUEUES=2` and a mounted `/root/.cache`. `models.yml` carries
|
||||
`contextWindow: 200704` for this endpoint — the served window, not the model's
|
||||
native 262144, so the harness compacts before vLLM answers a longer prompt
|
||||
with a 400.
|
||||
The **FreeToken engine at PR #587 head `77a7201`** (branch
|
||||
`rocm714_torch214`, built on the host into the local image
|
||||
`localhost/frank-primary-freetoken:pr587-77a7201` by
|
||||
`scripts/docker/frank-primary-freetoken.Dockerfile` — the PR's own
|
||||
docs/install_amd.md stack: digest-pinned
|
||||
`rocm/pytorch:rocm7.14_ubuntu24.04_py3.12_pytorch_release_2.11.0` base =
|
||||
torch 2.11.0+rocm7.14/triton 3.7, `--no-build-isolation --no-deps -e .`,
|
||||
curated portable deps, `flashlib==0.3.0 --no-deps`, and a `libhiprtc.so`
|
||||
dev-symlink the pip-ROCm wheel lacks) serving
|
||||
**`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`** (~172.8 GiB, 131 shards,
|
||||
standard modelopt block-FP8 — 75,264 `weight_scale_inv` tensors, no
|
||||
`input_scale`, loaded directly from HF safetensors, no FTW conversion) on
|
||||
**ONE R9700 (gfx1201)** — the second card sits idle. Plain `podman run`
|
||||
through `scripts/serve-frank-primary.sh` under the root systemd unit
|
||||
(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the
|
||||
container name is the unit name (`frank-primary`; the vLLM-era
|
||||
`frank-primary-qwen` is in the legacy cleanup list). Cutover 2026-10-05;
|
||||
rollback tag `frank-primary-vllm-shapeG-20261003` (the tcclaviger vLLM trio
|
||||
and its 362-line env lore live in that tag; the tcclaviger image stays in
|
||||
dank's rootless storage).
|
||||
|
||||
**`--max-num-seqs 2` is impossible on this model.** Measured 2026-10-02 by
|
||||
single-variable boot probe against the then-deployed shape: SEQS 2 fails at
|
||||
BOTH MNBT 1024 and 2048 with `torch.fx.experimental.symbolic_shapes.Constraint
|
||||
ViolationError: Constraints violated (L['query_start_loc'].size()[0]) …
|
||||
specialized it to be a constant (3)`; SEQS 3 passes at both. `model_runner.py`
|
||||
sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
|
||||
`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
|
||||
`SEQS + 1` for any `MNBT ≥ SEQS` — MNBT only moves the compile RANGE
|
||||
((1,1024) vs (1,2048)). `--enforce-eager` bypasses the path but drops CUDA
|
||||
graphs. The same failure class killed the 2026-10-01 bf16 2×262144 shape.
|
||||
**Shape**: all expert weights in **pinned host RAM** (~120 GiB) behind a
|
||||
GPU LRU (`--moe-cache-rate 0.10`, ~12 GiB — the single-card ceiling given
|
||||
the KV contract), the ~47.7 GiB PLE ngram table pinned in host RAM
|
||||
(→ root unit + memlock law), **KV pool `--num-tokens 299968`**
|
||||
(24 KiB/token f16 across the 12 full-attention layers ≈ 7.0 GiB; **must be
|
||||
a multiple of 64** — the auto attention backend is `qsa_sparse`, which
|
||||
overrides page size to 64; 300,000 was refused at first boot), 2 decode
|
||||
slots, `--max-output-tokens 36864` (the harness maxTokens; requests that
|
||||
omit one get this default). **Parsers are AUTO**: thinking lands in
|
||||
`reasoning_content` natively, tool calls in the qwen3.8 dialect — models.yml
|
||||
flips `reasoningContentField` to `reasoning_content` on the harness side.
|
||||
Vision works (models.yml keeps `input: [text, image]`); the vision tower's
|
||||
blocks stream from host RAM two at a time by default.
|
||||
|
||||
**The expert arena has THREE reachable levers: `max_model_len`, the overhead
|
||||
term, and the CUDA-graph estimate.**
|
||||
The fork's planner sizes expert cache as `budget − non-expert − KV reserve −
|
||||
graphs − overhead`, and the KV reserve is registry-priced at 9190 B/token ×
|
||||
`max_model_len`: 262144 → 2.48 GiB reserve → arena 14.30 GiB / **6,248 slots**;
|
||||
200704 → 1.91 GiB → 14.87 GiB / **6,496 slots**. Δreserve 0.57 GiB = Δarena
|
||||
0.57 GiB. The 6.1e9 → 3.15e9 pool cut bought **zero** arena; it buys runtime
|
||||
headroom (the shape it replaced died with `0 bytes free` on a real 995-token
|
||||
request). The second lever is `--expert-precise-memory`, **ON since
|
||||
2026-10-02**: it swaps the registry's default overhead (7.60 GiB = fitted +
|
||||
compile/graph-capture headroom) for the fitted figure, and the 2.35 GiB
|
||||
difference lands in the arena — measured plan lines at 200704 × 3: overhead
|
||||
5.25 GiB, expert cache **17.22 GiB / 7,522 slots** (+1,026 / +16 %), swap region
|
||||
13.7 GiB / 5,986 slots, boot healthy in 420 s. The pool stayed at 3.15e9
|
||||
deliberately: because the pool is not an arena lever, cutting it would cost
|
||||
concurrency (a 2.2e9 probe reports 277,686 tokens / **1.38×**) and buy no slots.
|
||||
The arena (2.344 MiB per slot) is the decode-critical region — cutting it for KV
|
||||
headroom collapses the offload hit rate.
|
||||
**Measured at cutover**: boot ~2.3 min cold-ish / 92 s warm → /health
|
||||
status ok; decode **17.1 tok/s** single-stream temp 0 (the vLLM incumbent
|
||||
measured ~30 tok/s on TWO cards; the KV contract caps the single-card LRU,
|
||||
and **TP=2 exists but does not boot on ROCm** — the TP allreduce JIT
|
||||
(`pynccl.cu`) can't resolve `cuda_runtime.h` for gfx1201; probed
|
||||
2026-10-06, signature in the runbook). `/metrics` is a 404 — the Dankclaw
|
||||
Prometheus llm-fleet scrape of :7770 reads down until repointed.
|
||||
`ft ctl cache --kv/--moe` resizes pools live (grow only; shrink needs a
|
||||
restart); `ft ctl stats` reports occupancy.
|
||||
|
||||
**Free VRAM on this endpoint is NOT arena headroom (2026-10-03 audit).**
|
||||
`rocm-smi` reads ~3.5 GiB free per card after hours of real traffic, but the
|
||||
planner charges `budget = total × util` against terms that overstate reality:
|
||||
the KV reserve is registry-priced at 9190 B/token (`1.91 GiB` at 200704) while
|
||||
the real pool bills 2.93 GiB at 7,892 B/token, and the precise overhead (5.25
|
||||
GiB) is far above the idle-measured figure. So the visible free space is the
|
||||
estimator gap, not allocatable arena. `gpu_memory_utilization` is a real arena
|
||||
lever (each +0.01 = +0.32 GiB) but **0.982 is the law, and 1.00 is UNBOOTABLE
|
||||
on this fork — measured 2026-10-03**: the worker init demands
|
||||
`free > budget`, and at `le=1` budget == device total, so both ranks raise
|
||||
`ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)` before a
|
||||
single weight loads, twice (2-restart crash loop, zero payload progress). The
|
||||
CLI field accepting `le=1` is the trap; Frank's 0.982 is where it always
|
||||
ships. What IS reachable from flags: `VLLM_CUDAGRAPH_MEM_BASE_MB` /
|
||||
`_PER_GRAPH_MB`, read at call time by `estimate_cudagraph_memory()` — the
|
||||
analytic default (256 MiB + 52 MiB/graph) charged 0.55 GiB where the engine's
|
||||
own post-capture line reported 0.31 GiB, and a second boot (with the plan
|
||||
already charging 0.32 GiB) reported **-0.06 GiB**, so the 256 MiB base pool is
|
||||
pure over-charge: shape G charges `0 base + 12 MiB/graph` = 0.07 GiB, which
|
||||
still prices per graph for a future 18-graph spec-decode shape (0.21 GiB).
|
||||
Verify every deploy by comparing the `Estimated CUDA graph memory`
|
||||
line with `Graph capturing finished … took X GiB`; if capture exceeds the
|
||||
estimate, raise `_PER_GRAPH_MB` back toward 52. MNBT 1024 → 2048 costs ZERO
|
||||
arena: the registry overhead is `base + 0.22 GiB × max(0,(nbt−4096)/1024)`
|
||||
anchored at nbt 4096, so every rung ≤4096 charges the same 5.25 GiB — and the
|
||||
fitted figure was fitted AT 4096.
|
||||
**Deploy gate law (changed for FreeToken)**: `/health` returns **200 with
|
||||
`{"status":"loading"}`** while the engine loads — the FastAPI front end
|
||||
binds before the scheduler is up, so a bare `curl` false-greens (measured:
|
||||
"healthy after 6 s" mid-page-in). The deployer loops until the body says
|
||||
`"status":"ok"`; the MainPID stray-port guard is unchanged. The image gate
|
||||
is `podman image exists` (a local build carries no RepoDigests — the old
|
||||
digest grep refused exactly this shape; pinning lives in the Dockerfile).
|
||||
|
||||
**A container's compile cache does NOT survive `podman rm -f`, and on this
|
||||
endpoint that is 181 s per boot.** Measured 2026-10-02: the image ships **no
|
||||
`/root/.cache` at all** (fresh container: `No such file or directory`); after a
|
||||
boot it holds `/root/.cache/vllm` at **1,154,714,333 B in 4,665 files** plus
|
||||
`/root/.cache/comgr` at 229 MB, all in the ephemeral layer, and the engine
|
||||
prints `torch.compile took 181.20 s in total` (173.19 s of it one graph,
|
||||
`Compiling a graph for compile range (1, 1024)`). Every deploy, restart and
|
||||
reboot re-pays it. The serve script now mounts
|
||||
`FRANK_PRIMARY_CACHE_DIR` at `/root/.cache` — the parent, so the vLLM AOT /
|
||||
torch_compile tree (`VLLM_CACHE_ROOT`, whose default is
|
||||
`/root/.cache/vllm`) and the comgr HIP-kernel JIT cache both persist, and
|
||||
nothing in the image is hidden because the image has no such directory. The
|
||||
compile RANGE is part of the cache key, so an MNBT change costs one
|
||||
recompilation, not a permanent tax. Same class of fix as the recipe's
|
||||
`/cache/{triton,vllm,inductor}` volumes. **Proven live 2026-10-02 on
|
||||
consecutive boots:** cold `Compiling a graph for compile range (1, 1024) takes
|
||||
179.24 s` / `torch.compile took 188.26 s`; warm `Directly load the compiled
|
||||
graph(s) for compile range (1, 1024) from the cache, took 2.143 s` /
|
||||
`torch.compile took 3.97 s in total` / `init engine (profile, create kv cache,
|
||||
warmup model) took 41.13 s (compilation: 3.98 s)`. Time to `/health` is
|
||||
dominated by the 172.8 GiB payload load, so the visible win is the ~184 s of
|
||||
inductor/AOT build, not a proportional boot cut.
|
||||
|
||||
Measured 2026-09-29 tuning campaign (fp8 + MTP4 era, since retired): decode
|
||||
**37.4-40.6 tok/s, acceptance ~54 %**; the 256k ctx arm measured 34.1-36.5;
|
||||
2-way concurrency sums BELOW parity (25.49 vs 26.8 single) — slots are a
|
||||
capacity feature, not a throughput one. V-only q8 KV is NOT possible via
|
||||
config: stock `--kv-cache-dtype` is K+V together; a V-only quant needs a fork
|
||||
patch.
|
||||
|
||||
Unit and self-healing: root systemd unit (`scripts/frank-primary.service`,
|
||||
`User=dank`, `LimitMEMLOCK=infinity`), `ExecStop=podman stop -t 120
|
||||
frank-primary-qwen`. The serve script drives `podman run` directly, gates
|
||||
the digest-pinned image, the payload's `config.json` and its exact
|
||||
131-shard count and the GPU device nodes, clears stale container
|
||||
identities, then blocks on `podman wait`. `deploy_llm_primary` gates the
|
||||
same, waits out :7770, relinks/enables/restarts the unit and health-gates
|
||||
with the MainPID check. Boot cost ~7 min warm (the 172.8 GiB payload load; the
|
||||
~184 s inductor/AOT build inside it is what the mounted cache now skips — see
|
||||
the compile-cache lesson below); health returns 503 while loading.
|
||||
|
||||
Runbook: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||
Runbook: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`.
|
||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
|
||||
|
||||
## Key Lessons Learned
|
||||
@@ -1005,7 +947,7 @@ Active:
|
||||
|---------|---------|
|
||||
| frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
|
||||
| frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q6_K, 2×200,704 f16 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md` |
|
||||
| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| frank-primary (FreeToken PR #587 ROCm, Qwen3.8-Flash-Next FP8, 1×R9700, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||
|
||||
@@ -1014,6 +956,7 @@ Retired (rollback targets and history):
|
||||
| Runtime | Host | Window | Runbook |
|
||||
|---------|------|--------|---------|
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
|
||||
| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
|
||||
| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
|
||||
|
||||
Reference in new issue
Block a user