dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-small: complete the Ornith swap in seam + deployer twins

Browse Source 
The rogue session's Ornith commits (3ee88d7/77decb5) swapped the env and
fetch script but left the deploy handler gates and the seam asserting
the MiMo-Q6K paths — the seam failures 195-203 and a deploy that gates
a file the unit no longer reads. Flip both to Ornith-Q8 (Q8_0 GGUF +
BF16 mmproj), keeping the anti-drift twin structure intact.
This commit is contained in:
Frank
2026-10-05 18:10:59 -07:00
parent 9c508218e9
commit 8f53df7b4f
2 changed files with 10 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+4 -4
View File
@@ -1207,12 +1207,12 @@ deploy_llm_large() {
# #
# It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is # It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
# not shared: frank-large's engine runs on the gfx1151 iGPU. # not shared: frank-large's engine runs on the gfx1151 iGPU.
LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf}" LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf}"
LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}" LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
# The projector IS the vision capability here: the Q6_K GGUF ships no `v.*` # The projector IS the vision capability here: the Ornith GGUF ships no
# tensors, so a deploy that starts without this file silently ships a # `v.*` tensors, so a deploy that starts without this file silently ships a
# text-only endpoint. Checked unconditionally, like frank-large's. # text-only endpoint. Checked unconditionally, like frank-large's.
LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/MiMo-Q6K/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf}" LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf}"
deploy_llm_small() { deploy_llm_small() {
# Refuse rather than restart-loop: a unit whose runtime or payload is # Refuse rather than restart-loop: a unit whose runtime or payload is
tests/frank_small_unit.bats
+6 -6
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env bats #!/usr/bin/env bats
# Seam: the frank-small llama.cpp root unit on dankclaw. # Seam: the frank-small llama.cpp root unit on dankclaw.
# frank-small is the RTX 3090 endpoint; since 2026-10-01 night it serves # frank-small is the RTX 3090 endpoint; since 2026-10-05 it serves
# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q6_K via a CUDA (sm_86) build of # ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 via a CUDA (sm_86) build of
# llama.cpp master, chosen because it is the only runtime here that moves K and # llama.cpp master, chosen because it is the only runtime here that moves K and
# V independently. Since 2026-10-03 it runs TWO unquantized contexts of # V independently. Since 2026-10-03 it runs TWO unquantized contexts of
# 200,704 tokens each and carries a vision projector, and the card must keep # 200,704 tokens each and carries a vision projector, and the card must keep
@@ -16,9 +16,9 @@ SERVE="$REPO_ROOT/scripts/serve-frank-small.sh"
ENV_FILE="$REPO_ROOT/scripts/frank-small.env" ENV_FILE="$REPO_ROOT/scripts/frank-small.env"
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh" REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
ENV_LARGE="$REPO_ROOT/scripts/frank-large.env" ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
TARGET_STUB="MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf" TARGET_STUB="Ornith-1.5-9B-Q8_0.gguf"
# Same basename as the shipped projector: the test env repoints it into $STAGE. # Same basename as the shipped projector: the test env repoints it into $STAGE.
MM_STUB="mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf" MM_STUB="mmproj-Ornith-1.5-9B-BF16.gguf"
setup() { setup() {
STAGE="$(mktemp -d)" STAGE="$(mktemp -d)"
@@ -81,9 +81,9 @@ argv_value() {
grep -q 'FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-small.env' "$UNIT" grep -q 'FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-small.env' "$UNIT"
} }
@test "the shipped env points at the llama.cpp runtime and the Q6_K GGUF" { @test "the shipped env points at the llama.cpp runtime and the Ornith-1.5-9B Q8_0 GGUF" {
grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server$' "$ENV_FILE" grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server$' "$ENV_FILE"
grep -qE "^MODEL_FILE=/home/dank/models/MiMo-Q6K/$TARGET_STUB$" "$ENV_FILE" grep -qE "^MODEL_FILE=/home/dank/models/Ornith-Q8/$TARGET_STUB$" "$ENV_FILE"
} }
@test "the deployer gates on exactly the payload paths the env file ships" { @test "the deployer gates on exactly the payload paths the env file ships" {