dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-small: Swift-1.5-27B Uncensored RCO-IQ3_XXS with MTP back on (two 131072 slots, ubatch 512, q8_0 KV, publisher's BF16 mmproj)

Browse Source
This commit is contained in:
Frank
2026-10-06 01:10:43 -07:00
parent 713b126639
commit da36fa32cf
6 changed files with 151 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+6 -7
View File
@@ -1207,14 +1207,13 @@ deploy_llm_large() {
#
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
# not shared: frank-large's engine runs on the gfx1151 iGPU.
LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/Swift15-IQ3S/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf}"
LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
# The projector IS the vision capability here: no Swift repo ships one (the
# GGUF carries no vision tensors), so the file is unsloth's mmproj built
# from the shared Qwen3.8-27B base — proven byte-identical to Swift's own
# vision stack. A deploy that starts without this file silently ships a
# text-only endpoint. Checked unconditionally, like frank-large's.
LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf}"
# The projector IS the vision capability here: the GGUF carries no vision
# tensors, and the publisher ships mmproj-BF16.gguf in the same repo, so a
# deploy that starts without this file silently ships a text-only
# endpoint. Checked unconditionally, like frank-large's.
LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
deploy_llm_small() {
# Refuse rather than restart-loop: a unit whose runtime or payload is