Commit Graph
18 Commits
Author SHA1 Message Date
Frank 0150333a6a serve-frank-large: restore Qwen-Sharp chat template + reformat for readability
The v6 restore commits dropped --chat-template-file and
--chat-template-kwargs while the artifact check still validated the
template file, so the Sharp template was checked but never applied.
Re-add both flags (with --jinja) so reasoning is emitted in DeepSeek
format and preserve_thinking/preserve_reasoning are honoured.

Reformat the script one-flag-per-line with long flag aliases
(--gpu-layers, --ctx-size, --ubatch-size, --cache-type-k/v) like
serve-frank-small, and update the 2 frank_large_config assertions
that grep the old short-flag forms (-ctk/-ctv, -ub).

- scripts/serve-frank-large.sh
- tests/frank_large_config.bats
2026-08-26 00:38:39 -07:00
Frank 8110884e8e Document canonical Caddy migration 2026-08-25 21:59:59 -07:00
Frank 863897f92a Clarify llama-bench harness scope 2026-08-25 19:07:46 -07:00
Frank dc69e2b4b4 Support fixed ubatch batch sweep at 16K context 2026-08-25 18:54:09 -07:00
Frank 96f5ae6fe1 Raise ubatch benchmark context to 32K 2026-08-25 17:44:45 -07:00
Frank 4cb67d7b33 Set ubatch sweep context and batch to 16K 2026-08-25 17:32:12 -07:00
Frank c072c75fa9 Make ubatch benchmark cleanup failure safe 2026-08-25 17:11:42 -07:00
Frank d6ea71b9a4 Configure DeepSeek reasoning and large-model ubatch sweep 2026-08-25 16:52:45 -07:00
Frank 6ee195dd8f Restore historical Q6 large model runtime 2026-08-25 14:23:47 -07:00
Frank ddfc64013f Fix Q6 AGENT projector path 2026-08-25 13:58:50 -07:00
Frank 86d5f38142 Restore Q6 AGENT large model 2026-08-25 13:36:07 -07:00
Frank 54b2d84613 Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.

- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
  REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
  preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
  Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
  (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled

The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
2026-08-25 07:12:02 -07:00
Frank b457457bfe Switch large service to Kairic Edge v1.2 2026-08-25 04:41:04 -07:00
Frank f86ed70c71 Use conservative four-token MTP drafts 2026-08-25 04:20:15 -07:00
Frank 3cf6353050 Reduce large model context to 64K 2026-08-25 04:13:00 -07:00
Frank b0eb571321 Reduce large model context to 128K 2026-08-25 04:09:12 -07:00
Frank ee3c3de750 Reduce frank-default-large context for KV headroom 2026-08-25 04:05:33 -07:00
Frank 7ca9dc248e frank-default-large: swap to caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP
Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with
cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the
charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo
iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file);
strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary
/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir
/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env,
ubatch bench, large-config bats test, docs, AGENTS.md.
2026-08-25 03:33:29 -07:00