Commit Graph
10 Commits
Author SHA1 Message Date
Frank 132e1478cf frank-default-large: upgrade to CIRU ActiveFPX PromptForge v3 (Output-K8 runtime, full-W8 views, BF16 vision) 2026-08-21 20:00:31 -07:00
Frank 583efd8cf2 Revert frank-default-large to CIRU runtime (DFlash2 too slow)
Restore the CIRU ROCmFPX setup that DFlash2 replaced:

- Restore jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2 runtime
  (native depth-4 MTP, PromptForge DualView, ROCm 7.15)
- Archive the DFlash2 Vulkan deployment files in .AGENTTRASH
- DFlash2 was retired the same day it shipped (2026-08-21) for speed
- Revert docs (deployment.md, index.md, AGENTS.md); mark dflash2 page archived
- Restored scripts are byte-identical to the pre-swap CIRU versions
2026-08-21 10:15:23 -07:00
Frank fd0d9dd3cb Swap frank-default-large to DFlash2 on Vulkan (PR #27342)
Replace the retired CIRU ROCmFPX runtime with upstream llama.cpp PR #27342
DFlash2 on the Vulkan backend:

- Target: unsloth Qwen3.8-27B-UD-Q8_K_XL.gguf (31.4 GB, already on disk)
- Drafter: z-lab Qwen3.8-27B-DFlash2-Q4_K_M.gguf (block size 8, top-k 16)
- Runtime: /home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server
  (GGML_VULKAN=ON; HIP lacks TOP_K so the stack runs on Vulkan0/RADV)
- spec: --spec-type draft-dflash --spec-draft-n-max 8 (clamps to 7)

Validated 2026-08-21 on scratch port 52626: mean acceptance 5.27,
health + reasoning completion OK.

Archive the retired CIRU .env/.serve script in .AGENTTRASH.
Update deployment.md, index.md, AGENTS.md; add docs/dflash2-large-vulkan.md.
2026-08-21 09:40:08 -07:00
Frank cd6202f4a0 frank-default-large: batch=4096, ubatch=4096 (OOM fix) 2026-08-20 19:58:45 -07:00
Frank 8bf9462f21 frank-default-large: add --no-mmap to avoid teardown OOM 2026-08-20 14:49:45 -07:00
Frank 8c391fe120 Reduce ctxcp from 32 to 16 to free ~16 GiB VRAM headroom and prevent OOM crashes during prompt processing 2026-08-18 16:11:58 -07:00
Frank a08be1641f frank-default-large: full native 262144 context per slot (524288 total)
llama.cpp splits -c across -np slots; to give each of the 2 slots the full
262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB
unified pool. Update env default, serve-script fallback, and the deployment
doc line.
2026-08-18 08:01:17 -07:00
Frank 1ba9618fcc Swap fleet naming + restore Qwen3.6-35B-A3B Q6_K_P on Dankserver
- Restore the pre-2026-08-17 Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
  Q6_K_P + ngram-mod setup as frank-default-small on Dankserver (port 7770,
  upstream llama.cpp b10121, MoE experts on CPU via --n-cpu-moe 36).
- Rename the CIRU Qwen3.8-27B ActiveFPX setup from frank-default-small to
  frank-default-large (Dankclaw, port 52625); bump to CTX_SIZE=262144
  (native max) and PARALLEL=2 (96 GiB unified pool).
- Retire the Qwen3.8-27B IQ4_XS FastMTP frank-default-large (Dankserver) to
  .AGENTTRASH.
- deploy.sh: SERVICE[dankserver]=frank-default-small,
  SERVICE[dankclaw]=frank-default-large; HEALTH_WAIT[dankserver]=600.
2026-08-18 00:32:26 -07:00
Frank 27788f6f2c fix: make serve scripts + deploy.sh executable 2026-08-17 15:24:40 -07:00
Frank d9af7f1d19 fleet redo 2026-08-17: retire PP+NPU, new topology
- Move all previous scripts to old/scripts/ (retired PP/NPU-era)
- frank-default-large: llama.cpp FastMTP on dankserver (HauhauCS
  Qwen3.8-27B IQ4_XS + FastMTP 32K draft + ngram-mod, K q8_0/V f16,
  128K ctx, batch 2048/ubatch 1024, port 7770)
- frank-default-small: vLLM ROCm on dankclaw (Avesed Qwen3.6-35B-A3B
  INT8 W8A8, full 262144 ctx, no KV quant, 4 seqs, batch 16384,
  ngram spec, port 52625)
- retarget deploy.sh (dankserver/dankclaw, inline sudo, no helpers)
- docs: deployment + index + AGENTS.md + README for new topology,
  superseded banners on retired docs, add frank-redo-2026-08-17 runbook
2026-08-17 15:23:46 -07:00