diff --git a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md index 7a7c8b7..448190c 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md @@ -148,17 +148,17 @@ Deployed flags: | Flag | Value | Why | |---|---|---| -| `--split-mode` / `--device` / `--tensor-split` | `layer` / `ROCm0,ROCm1` / `35,30` | both cards, one model, no collective; the split is skewed because ROCm1 carries the drafter, and it is one LAYER of granularity — 0.43 GiB on a DeltaNet boundary, 1.43 GiB on an attention one. 35:30 since 2026-09-17 (part 3): ROCm0 is CPU-root-port Gen4 x16, ROCm1 rides the chipset at Gen4 x1, and the drafter-bound card on the x1 path ran 98% against the x16 card's 40%. 36:29 fails both the 1 GiB floor (card 0 free 0.45) and the fit slack (gap 2.22) — this is the farthest safe shift | +| `--split-mode` / `--device` / `--tensor-split` | `layer` / `ROCm0,ROCm1` / `32,33` | both cards, one model, no collective. Balanced (operator call 2026-09-19): 65 layers split as evenly as they can be, ROCm1 (the last-layers card that also carries the drafter and output layers) holding the extra one. The predecessor's 35:30 skew (2026-09-17, part 3) leaned work onto the x16 card because ROCm1's Gen4 x1 chipset path made it the 98 % bottleneck; the balanced point trades that lean for symmetry. Split granularity is one LAYER — 0.43 GiB on a DeltaNet boundary, 1.43 on an attention one | | `--gpu-layers` / `--fit` | `999` / `off` | everything on the cards; auto-fit must not resize a context we set | -| `--ctx-size` / `--parallel` | `393216` / `2` | the pool over two slots of 196608; `--parallel` **divides** the context. 1.5x the trained 262144 window, and the GGUF has no rope scaling — see the 2026-09-17 section | +| `--ctx-size` / `--parallel` | `327680` / `2` | the pool over two slots of 163840; `--parallel` **divides** the context. 163840 is inside the checkpoint's trained 262144 window, so the whole pool is trained territory — no rope extrapolation (unlike the predecessor's 393216) | | `--batch-size` / `--ubatch-size` | `6144` / `512` | measured: ubatch 512 is the prefill optimum *and* the cheapest point on the curve; the logical batch is throughput- and VRAM-neutral. See [[frank-medium-batch-ubatch]] | -| `--ctx-checkpoints` / `--cache-ram` | `0` / `16384` | no per-slot KV snapshots (the build's default is **32** — this is a deliberate deviation, not an inherited default); the RAM-side prompt cache is capped at 16 GiB, which is what lets a full 196608-token slot state be cached at all (2026-09-18, see below) | +| `--ctx-checkpoints` / `--cache-ram` | `0` / `16384` | no per-slot KV snapshots (the build's default is **32** — this is a deliberate deviation, not an inherited default); the RAM-side prompt cache is capped at 16 GiB, which is what lets a full 163840-token slot state be cached at all (2026-09-18, see below) | | `--flash-attn` / cache types | `on` / `f16` K and V | an agentic checkpoint is the wrong place to spend KV precision | -| `--spec-type` / `--spec-draft-n-max` | `draft-mtp` / `3` | the model's own head, `shares_model` — no sidecar file, no second model copy; n_max 3 is the measured MTP winner (32.4 tok/s vs 26.7 at n_max 7). The DFlash2 sidecar stays on disk: `SPEC_TYPE=draft-dflash` brings the 43.5 tok/s chain back at ~1.1 GiB on ROCm1 | +| `--spec-type` / `--spec-draft-n-max` | `draft-dflash` / `7` | the z-lab DFlash2 `Q4_K_M` sidecar (1.06 GiB), the measured throughput winner (43.5 tok/s vs the model's own MTP head at 32.4); n_max 7 is the DFlash2 optimum. The target file also carries its own MTP head (`nextn_predict_layers=1`), so `SPEC_TYPE=draft-mtp` is one change away — the drafter file stays on disk | | `--spec-draft-device` | `ROCm1` | the drafter rides the card that ends up holding the target's last layers | | `--temp` / `--top-p` / `--top-k` | `0.7` / `0.95` / `20` | the values the vLLM predecessor served on this endpoint | | `--reasoning-format` / `--reasoning-budget` | `deepseek` / `-1` | trace into `reasoning_content`, unlimited | -| `--mmproj` | `mmproj-BF16.gguf` (888 MiB, beside the target) | vision enabled since 2026-09-17; this build has NO autoload, so the flag is explicit and the serve script refuses to start without the file | +| `--mmproj` | `mmproj-Qwen3.8-27B-Uncensored-f16.gguf` (beside the target) | vision enabled; this build has NO autoload, so the flag is explicit and the serve script refuses to start without the file | ## The DSpark excursion (2026-09-16, tried and reverted) diff --git a/serverAppConfigs-docs/LLM/halogen-flash-server.md b/serverAppConfigs-docs/LLM/halogen-flash-server.md index 8d7f374..fc7e6d2 100644 --- a/serverAppConfigs-docs/LLM/halogen-flash-server.md +++ b/serverAppConfigs-docs/LLM/halogen-flash-server.md @@ -1,7 +1,7 @@ --- type: reference title: halogen-flash-server on Dankclaw (frank-large) -description: The active frank-large runbook — peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b) on the Dankclaw gfx1151 box: container shape, deployed flags, the memory model, the client contract, and the failure modes worth knowing. +description: The RETIRED frank-large runbook (2026-09-16 → 09-19, rollback target) — peonist-ai/halogen-flash-server 0.5.8 serving Qwen3.8-Flash-Next (w4b) on the Dankclaw gfx1151 box: container shape, deployed flags, the memory model, the client contract, and the failure modes worth knowing. Replaced by [[frank-large-ciru-orca]] on 2026-09-19. tags: - llm - halogen @@ -16,14 +16,15 @@ timestamp: 2026-09-13 # halogen-flash-server on Dankclaw (frank-large) -> **ACTIVE again since 2026-09-16, and retuned.** It replaced the llama.cpp / -> kingjones30-ROCmFPX STRIX deployment on 2026-09-11 (see +> **RETIRED 2026-09-19 — replaced by [[frank-large-ciru-orca]]** (the CIRU v4.4.1 +> runtime serving jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca). This runbook is the +> rollback target. It was ACTIVE again since 2026-09-16: it replaced the +> llama.cpp / kingjones30-ROCmFPX STRIX deployment on 2026-09-11 (see > [[kingjones-strix-rocmfp4]]), was displaced for two days by ik_llama.cpp (see -> [[frank-large-ik-uncensored]], the current rollback), and came back as the -> shape the fleet asked for: two 200k-token conversations rather than the -> native-context pair. Deployed as a rootless Quadlet container, -> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770. The RTX 3090 -> it does not touch now belongs to [[frank-small-qwen38-gsq-rco]]. +> [[frank-large-ik-uncensored]]), and came back as the shape the fleet asked for: +> two 200k-token conversations rather than the native-context pair. It was +> deployed as a rootless Quadlet container, +> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770. The engine is a from-scratch serving runtime for the Qwen3.8-Flash-Next architecture on AMD's gfx1151 (Strix Halo) — the same silicon this host runs.