Frank
b2a74004df
Replace the unsloth llama.cpp prebuilt (b11030, Qwen3.8-Flash-Next UD-IQ4_XS, 125B MoE) with the Kairic Edge runtime (ciru-ai/ROCmFPX, branch kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (27B dense, IU4 W4A4). - Runtime: built from source (scripts/build-kairic-edge-gfx1151.sh), TheRock 7.15.0a20260718 toolchain, CK fdf4bb7f + gfx1151-iu4 patch. - Shape: one stream, full 262144 context (the model's native window), 1 slot, 8 GiB prompt-cache budget. - Spec method: native MTP depth 4 + n-gram drafting (24/64/64). DFlash is NOT available for this model (no DRAFT_DFLASH spec type). - Compatibility mode: the harness needs sampling, tools and a reasoning budget, which fast-greedy rejects. COMPATIBILITY_MODE=1 enables them. - Reasoning: enabled (auto + deepseek format, thoughts land in reasoning_content). - Payload: 4 files (15.5 GiB GGUF + 7.99 GiB FFN + 1.88 GiB GDN + 0.70 GiB GDN-Output), sha256-verified (scripts/download-frank-large-kairic.sh).
22 lines
727 B
Desktop File
22 lines
727 B
Desktop File
[Unit]
|
|
Description=Kairic Edge (ROCmFPX v1.2, gfx1151) Qwen3.8-27B IU4 W4A4 (dankclaw, frank-large, Strix Halo iGPU)
|
|
After=network-online.target
|
|
Wants=network-online.target
|
|
|
|
[Service]
|
|
Type=simple
|
|
User=dank
|
|
Group=dank
|
|
# The engine pins staging buffers and the PLE pager's hot window; keep the
|
|
# memlock budget unbounded so an RLIMIT smaller than the pinned set cannot
|
|
# fail the load. Parity with the halogen quadlet's Ulimit=memlock=-1:-1 and
|
|
# the other LLM units on this fleet.
|
|
LimitMEMLOCK=infinity
|
|
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
|
Restart=on-failure
|
|
RestartSec=10
|
|
|
|
[Install]
|
|
WantedBy=multi-user.target
|