dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-primary: swap Dankserver to llama.cpp Qwen3.8-Flash-Next Q8_0

Browse Source 
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).

The deployed contract:

- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
  and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
  -ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
  over PCIe loses at 3.7 t/s, computing in RAM does not care).
  per_layer_token_embd.weight=CPU is required by the shared MTP head's
  borrow path (a GPU-resident borrow crashes the draft load).
  mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
  (mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
  draftless ngram-mod matcher chained behind it — the only legal way to
  layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
  autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
  Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
  by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
  with Qwen-recommended sampling riding compat.extraBody.

The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.

Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.

Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).

models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].

Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).

Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
This commit is contained in:
Frank
2026-09-26 19:45:28 -07:00
parent d89813363e
commit 8500600d09
25 changed files with 856 additions and 544 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-plan.py
+12 -7
View File
@@ -371,13 +371,11 @@ def services_for_path(path: str, target: str) -> set[str]:
if name == "node-exporter":
_add(services, "node-exporter")
return services
if len(parts) >= 2 and parts[1] == "frank-primary":
# frank-primary is the Lucebox DeepSeek V4 Quadlet (2026-09-26
# cutover, replacing frank-medium on :7770): its entrypoint and
# healthcheck live under this directory and the container mounts
# them straight out of this checkout, so any file here changes
# deployed state.
_add(services, "frank-primary")
# frank-primary's quadlet-era directory left the tree on 2026-09-26
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
# the trio in scripts/ now maps it). Nothing lives under
# dankserver/frank-primary/ except dated trash, and trash plans
# nothing (the retire rule above the service blocks).
return services
if parts[0] == "frankbook-pro":
@@ -421,6 +419,13 @@ def services_for_path(path: str, target: str) -> set[str]:
# what the live frank-large endpoint renders, so it plans
# frank-large.
_add(services, "frank-large")
if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}:
# llama.cpp frank-primary (2026-09-26, replacing the Lucebox
# DS4.1 quadlet) keeps its unit, env and serve files here the
# same way: a root unit on the unsloth b11030-mix prebuilt
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
# the payload, which is host-side state, so it selects nothing.
_add(services, "frank-primary")
# Deployment/test/benchmark scripts and other repo tooling do not
# restart a runtime service.
return services
scripts/deploy-remote.sh
+87 -41
View File
@@ -1178,52 +1178,98 @@ deploy_llm_small() {
wait_for_health frank-small http://127.0.0.1:7771/health 900
}
# deploy_llm_primary — frank-primary is the Lucebox DeepSeek V4 Quadlet
# (the 2026-09-26 cutover that replaces frank-medium on :7770). The model
# is a 377 GiB GGUF and the image is a local build, so the gates are:
# image present, model file present at its declared byte count (a partial
# download must never be served).
# deploy_llm_primary — frank-primary is unsloth llama.cpp (the b11030-mix
# prebuilt, ROCm gfx120X, bundled runtime) serving Qwen3.8-Flash-Next at
# Q8_0 on the two R9700s, as a ROOT unit — the same shape as frank-large on
# Dankclaw: a host binary, no container image, no generated unit.
# relink_unit points /etc/systemd/system/frank-primary.service at the repo
# file and boot persistence is that unit's [Install] section.
#
# The 2026-09-26 swap that replaced the Lucebox DS4.1 Quadlet on this port.
# The payload is six Q8_0 split parts plus the shared MTP head and the
# projector, so the gate walks every artifact at its declared byte count (a
# partial download must never be served — and stat -c %s, never wc -c, which
# read the whole 377 GiB lucebox file for minutes on 2026-09-26).
LLM_PRIMARY_RUNTIME=/home/dank/GitPackages/unsloth-llama-b11030/llama-server
LLM_PRIMARY_ROOT=/home/dank/models/Qwen3.8-Flash-Next-GGUF
LLM_PRIMARY_PART_SIZES=(
10946624
682434912
54400261312
49446841216
49668930400
34015618784
)
LLM_PRIMARY_DRAFT="$LLM_PRIMARY_ROOT/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf"
LLM_PRIMARY_DRAFT_SIZE=2786568256
LLM_PRIMARY_MMPROJ="$LLM_PRIMARY_ROOT/mmproj-BF16.gguf"
LLM_PRIMARY_MMPROJ_SIZE=907542944
# The lucebox quadlet held this port until the 2026-09-26 cutover. Its
# generated user unit is host state, not repo state: stop it and move the
# installed .container into dated home trash (the same retire pattern as
# retire_halogen_large_quadlet). The image and the 377 GiB payload stay on
# disk as the rollback target.
retire_lucebox_primary_quadlet() {
local trash="$HOME/.AGENTTRASH/20260926-llamacpp-cutover/quadlet"
systemctl --user stop frank-primary.service 2>/dev/null || true
if [[ -e "$quadlet_unit_dir/frank-primary.container" ]]; then
mkdir -p "$trash"
mv -- "$quadlet_unit_dir/frank-primary.container" "$trash/frank-primary.container"
echo "retired quadlet frank-primary.container -> $trash"
fi
systemctl --user daemon-reload
}
deploy_llm_primary() {
local missing=0
local model="/home/dank/models/DeepSeek-V4.1-Flash/DeepSeek-V4.1-Flash-ROCMFP23.gguf"
local want_size=405349859328
if ! podman image exists localhost/lucebox-rocm:gfx1201; then
echo "ERROR: lucebox image absent (localhost/lucebox-rocm:gfx1201)" >&2
echo "frank-primary: podman build --build-arg LUCE_HIP_ARCHES=gfx1201 --build-arg ROCM_VERSION=7.14.1-full -t localhost/lucebox-rocm:gfx1201 -f Dockerfile.rocm . (see runbook)" >&2
# Refuse rather than restart-loop. A unit whose runtime or payload is
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the deploy log to say why.
local missing=0 i part want_size got_size
if [[ ! -x "$LLM_PRIMARY_RUNTIME" ]]; then
echo "ERROR: runtime absent at $LLM_PRIMARY_RUNTIME" >&2
echo "frank-primary: fetch the unsloth b11030-mix gfx120X prebuilt (see the runbook)" >&2
missing=1
fi
# stat -c %s, never `wc -c < FILE`. wc falls back to reading the whole
# stream whenever its fstat shortcut does not apply, and this payload is
# 377 GiB: on 2026-09-26 a deploy sat in wc for minutes (30 s of CPU,
# NVMe-bound) before it could even reach the cutover.
got_size="$(stat -c %s "$model" 2>/dev/null || echo 0)"
if [[ "$got_size" != "$want_size" ]]; then
echo "ERROR: model absent or incomplete at $model (have $got_size, want $want_size)" >&2
missing=1
fi
[[ "$missing" -eq 0 ]] || return 1
# Cutover COMPLETED 2026-09-26: frank-primary took :7770 and the radiance
# unit is retired. The host-side unit, its mask and the repo files were all
# moved to trash by the parent session, so there is nothing left to stop or
# mask here — masking a name that no longer exists would only resurrect it
# in the unit search path. The port gate stays, because 7770 is a
# host-network bind: nothing new can be created until the holder released it.
# 7770 is a host-network bind: nothing new can be created until the old
# holder released it (the same gate as the 2026-09-16 cutover).
sleep 2
wait_port_free 7770 || return 1
install_quadlet frank-primary dankserver/frank-primary/quadlet frank-primary
# No enable: generated units cannot be enabled on this host (v1 quadlet
# generator); boot persistence is the unit's [Install] WantedBy=default.target.
systemctl --user start frank-primary.service
# First start page-ins the dense weights and builds the graphs;
# HealthStartPeriod in the unit is 1800s to match.
wait_container_healthy frank-primary 1800 || {
podman logs --tail 40 frank-primary 2>&1 || true
for i in 1 2 3 4 5 6; do
part="$LLM_PRIMARY_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-0000$i-of-00006.gguf"
want_size="${LLM_PRIMARY_PART_SIZES[$((i-1))]}"
got_size="$(stat -c %s "$part" 2>/dev/null || echo 0)"
if [[ "$got_size" != "$want_size" ]]; then
echo "ERROR: part $i absent or incomplete at $part (have $got_size, want $want_size)" >&2
missing=1
fi
done
for part in "$LLM_PRIMARY_DRAFT:$LLM_PRIMARY_DRAFT_SIZE" \
"$LLM_PRIMARY_MMPROJ:$LLM_PRIMARY_MMPROJ_SIZE"; do
local file="${part%%:*}" want="${part##*:}"
got_size="$(stat -c %s "$file" 2>/dev/null || echo 0)"
if [[ "$got_size" != "$want" ]]; then
echo "ERROR: artifact absent or incomplete at $file (have $got_size, want $want)" >&2
missing=1
fi
done
[[ "$missing" -eq 0 ]] || {
echo "frank-primary: run scripts/download-frank-primary.sh on this host" >&2
return 1
}
retire_lucebox_primary_quadlet
# On a re-deploy the predecessor is THIS unit: stop it before waiting on
# the port, or the wait deadlocks on the server it just stopped serving.
run_privileged systemctl stop frank-primary.service 2>/dev/null || true
# Nothing starts until the port is released: the lucebox cutover on
# 2026-09-26 and the frank-large cutover on 2026-09-16 both failed
# exactly here when the next engine bound 7770 a second too early.
wait_port_free 7770 || return 1
relink_unit frank-primary
run_privileged systemctl daemon-reload
run_privileged systemctl enable frank-primary.service
run_privileged systemctl restart frank-primary.service
# Cold start page-ins 175 GiB of expert pages and builds the graphs;
# 1800s parity with the lucebox HealthStartPeriod this port has always
# needed.
wait_for_health frank-primary http://127.0.0.1:7770/health 1800
}
wait_for_health() {
scripts/frank-primary.env
+78
View File
@@ -0,0 +1,78 @@
# frank-primary — unsloth llama.cpp (b11030-mix prebuilt, ROCm gfx120X)
# serving unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 on the two Radeon AI PRO
# R9700s (gfx1201), with the shared Q8_0 MTP drafter head, the BF16 vision
# projector, and the draft-mtp + ngram-mod spec chain.
#
# These are the tunables read by scripts/serve-frank-primary.sh. The flag
# list is owned by the serve script; this file only sets fleet values.
#
# DO NOT name a new tunable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP
# runtime reads those bare (the 2026-08-13 TEMP incident). The bind address
# is BIND_HOST, not HOST — frank-large's env predates that lesson and gets
# away with it; do not copy the shape.
# The payload (scripts/download-frank-primary.sh): the serve script refuses
# to start without all six Q8_0 parts, the MTP head and the projector.
MODEL_ROOT=/home/dank/models/Qwen3.8-Flash-Next-GGUF
# The runtime: the unsloth b11030-mix prebuilt for gfx120X (the same
# b11030-mix build frank-large runs on gfx1151 — it bundles its own ROCm
# runtime, no host SDK gate).
RUNTIME_DIR=/home/dank/GitPackages/unsloth-llama-b11030
# Fleet endpoint: the port and alias the lucebox DS4.1 quadlet held before
# the 2026-09-26 swap, so no client, probe or models.yml entry changes
# address.
BIND_HOST=0.0.0.0
PORT=7770
# One conversation, the model's full native 262144 window.
CONTEXT_SIZE=262144
PARALLEL_SLOTS=1
# Spec chain: draft-mtp (the shared Q8_0 head) at depth 2, with the
# draftless ngram-mod matcher chained behind it. Depth 2 is the measured
# optimum class for this pair at long contexts (the retired Dankclaw
# Flash-Next deployment ran exactly this head at depth 2 with 0.83-0.92
# acceptance); each extra verified position re-batches the CPU expert reads,
# so depth buys less than bandwidth here.
SPEC_DRAFT_N_MAX=2
# Device placement. -ngl 999 puts every layer's non-overridden tensors on
# the GPUs; the big overrides then move the bandwidth-huge tensors to host
# RAM (computed there — the x4-pair measurement: streaming over PCIe loses,
# computing in RAM does not). OT_OVERRIDES is comma-separated
# <tensor-pattern>=<backend> pairs passed as repeated --override-tensor
# flags:
# * exps=CPU — the routed experts. ~100 GiB at Q8, read once per token;
# the CPU backend computes them from page cache.
# * the n-gram embedding block (the 51B-param PLE table) — sparse row
# lookups, tiny PCIe traffic from CPU RAM; NEVER pin with a GPU target
# (the 23 -> 13.4 t/s regression).
# * per_layer_token_embd.weight=CPU — REQUIRED by the shared MTP head's
# borrow path: GPU-resident, it crashes the draft load with
# "invalid vector subscript" (llama.cpp PR #28243).
# The exact tensor pattern for the n-gram block was discovered at load time
# on 2026-09-26 (see the runbook); this list is the deployed set.
NGL=999
OT_OVERRIDES=exps=CPU,ngram_embeddings=CPU,per_layer_token_embd.weight=CPU
TENSOR_SPLIT=1,1
# The drafter lives on the first R9700 (the card the load log shows holding
# the shared embedding borrow path).
SPEC_DRAFT_DEVICE=ROCm0
# Batch geometry: logical batch 8192 with ubatch 512 — the measured optimum
# class on this fleet's ROCm stack (a bigger logical batch is free; compute
# buffers reserve at max graph size). ubatch 512 balances prefill
# throughput against the asymmetric compute-buffer cost on the card that
# holds the output layers and the drafter.
BATCH_SIZE=8192
UBATCH_SIZE=512
# Prompt-cache budget: 16 GiB. The KV itself is small (the hybrid
# DeltaNet/QSA pattern keeps a 262144 window a fraction of the full-ctx
# x all-layers number), so the prompt cache is the RAM-side spend that
# actually moves prefill on repeat traffic. Host RAM budget: ~175 GiB of
# expert pages + this cache + the server's own arenas inside 246 GiB.
CACHE_RAM=16384
scripts/frank-primary.service
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=unsloth llama.cpp (b11030-mix, ROCm gfx120X) Qwen3.8-Flash-Next Q8_0, shared-Q8_0 MTP + ngram-mod spec chain, BF16 vision, 262144 ctx / 1 slot, root systemd unit (dankserver, frank-primary, 2x Radeon AI PRO R9700)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=dank
Group=dank
# The model is mmap'd (175 GiB of expert pages + the 51B-param n-gram
# table live in page cache); an RLIMIT_MEMLOCK smaller than what the
# runtime pins would fail the load mid-start. Parity with the other LLM
# units on this fleet.
LimitMEMLOCK=infinity
Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
scripts/serve-frank-primary.sh
Executable
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env bash
# Serve unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 as frank-primary on
# Dankserver's two Radeon AI PRO R9700s (gfx1201), using the unsloth
# b11030-mix prebuilt (ROCm gfx120X, bundled runtime).
#
# Shape: one stream, the model's full 262144 context, all non-overridden
# tensors on the GPUs (--n-gpu-layers 999 across both cards), with the
# routed experts and the 51B-param n-gram embedding table computed in host
# RAM via --override-tensor (the x4-pair law: streaming over PCIe loses,
# computing in RAM does not care). mmap stays ON (the PLE table must stay
# page-cache-backed; --no-mmap forces it into anonymous memory and the
# cgroup kill is silent).
#
# Spec chain: draft-mtp (the shared Q8_0 head, loaded via -md) chained with
# the draftless ngram-mod matcher. The shared head borrows the target's
# embedding tensors, which is why per_layer_token_embd.weight must be
# CPU-resident (see frank-primary.env). depth 2.
#
# Vision: the BF16 projector, loaded explicitly via --mmproj (this build
# never autoloads projectors — the frank-medium 2026-09-17 trap: a server
# started without it comes up healthy and text-only, and answers image
# requests with 500).
#
# Template: the GGUF's embedded chat template via --jinja. The Qwen3.8
# family template gates reasoning_effort to {xhigh, medium, low} — the
# harness folds its levels onto that set in models.yml
# (reasoningEffortMap). If the embedded template ever falls back to
# peg-native and bloats prompts (~73,600 tokens for a ~250-token request),
# switch to --chat-template-file scripts/qwen-sharp-v22.4.0.jinja the way
# frank-large does.
#
# Tunables come from frank-primary.env unless FRANK_PRIMARY_ENV_FILE
# overrides it; anything not set there falls through to the defaults below.
set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}"
if [[ -f "$ENV_FILE" ]]; then
set -a
source "$ENV_FILE"
set +a
fi
# --- defaults (used when the env file does not set them) ------------------
: "${RUNTIME_DIR:=/home/dank/GitPackages/unsloth-llama-b11030}"
: "${MODEL_ROOT:=/home/dank/models/Qwen3.8-Flash-Next-GGUF}"
: "${BIND_HOST:=0.0.0.0}"
: "${PORT:=7770}"
: "${CONTEXT_SIZE:=262144}"
: "${PARALLEL_SLOTS:=1}"
: "${SPEC_DRAFT_N_MAX:=2}"
: "${NGL:=999}"
: "${OT_OVERRIDES:=exps=CPU,ngram_embeddings=CPU,per_layer_token_embd.weight=CPU}"
: "${TENSOR_SPLIT:=1,1}"
: "${SPEC_DRAFT_DEVICE:=ROCm0}"
: "${BATCH_SIZE:=8192}"
: "${UBATCH_SIZE:=512}"
: "${CACHE_RAM:=16384}"
SERVER_BIN="$RUNTIME_DIR/llama-server"
MODEL_PARTS=(
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00001-of-00006.gguf"
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00002-of-00006.gguf"
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00003-of-00006.gguf"
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00004-of-00006.gguf"
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00005-of-00006.gguf"
"$MODEL_ROOT/Q8_0/Qwen3.8-Flash-Next-Q8_0-00006-of-00006.gguf"
)
DRAFTER_FILE="$MODEL_ROOT/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf"
MMPROJ_FILE="$MODEL_ROOT/mmproj-BF16.gguf"
if [[ ! -x "$SERVER_BIN" ]]; then
echo "runtime not found at $SERVER_BIN" >&2
echo " fetch the unsloth b11030-mix gfx120X prebuilt (see the runbook)" >&2
exit 1
fi
# Every part is checked: part 1 is the -m target and parts 2-6 are followed
# by the split-file loader, so a missing tail part fails at LOAD time with
# a confusing partial-mmap error unless it is caught here.
for f in "${MODEL_PARTS[@]}" "$DRAFTER_FILE" "$MMPROJ_FILE"; do
if [[ ! -f "$f" ]]; then
echo "missing asset: $f" >&2
echo " run scripts/download-frank-primary.sh on this host" >&2
exit 1
fi
done
# --- env (the unsloth build's bundled ROCm runtime) -----------------------
# Both R9700s (gfx1201, HIP devices 0 and 1); the host's Raphael iGPU is
# device 2 and must stay hidden from the enumeration.
export HIP_VISIBLE_DEVICES=0,1
export LD_LIBRARY_PATH="$RUNTIME_DIR:${LD_LIBRARY_PATH:-}"
# --- the serve command ---------------------------------------------------
ARGS=(
--host "$BIND_HOST"
--port "$PORT"
--model "${MODEL_PARTS[0]}"
--alias frank-primary
--ctx-size "$CONTEXT_SIZE"
--parallel "$PARALLEL_SLOTS"
--n-gpu-layers "$NGL"
--tensor-split "$TENSOR_SPLIT"
--jinja
--reasoning auto
--reasoning-format deepseek
--reasoning-budget -1
--flash-attn on
--batch-size "$BATCH_SIZE"
--ubatch-size "$UBATCH_SIZE"
--cache-ram "$CACHE_RAM"
# Vision: explicit, never autoloaded by this build
--mmproj "$MMPROJ_FILE"
# Spec chain: the shared MTP head + the draftless ngram matcher
--spec-type draft-mtp,ngram-mod
--model-draft "$DRAFTER_FILE"
--spec-draft-n-max "$SPEC_DRAFT_N_MAX"
--spec-draft-device "$SPEC_DRAFT_DEVICE"
# Greedy server default: deterministic e2e, and the harness overrides
# sampling per-request (models.yml compat.extraBody carries Qwen's
# recommended temp 1.0 / top_p 0.95 / top_k 20).
--temp 0
--top-p 1.0
--top-k 0
--min-p 0.0
)
# Comma-separated <pattern>=<backend> pairs -> repeated --override-tensor.
IFS=',' read -ra _ots <<< "$OT_OVERRIDES"
for ot in "${_ots[@]}"; do
ARGS+=(--override-tensor "$ot")
done
exec "$SERVER_BIN" "${ARGS[@]}"
scripts/test-changed.sh
+8 -8
View File
@@ -187,16 +187,16 @@ for path in "${CHANGED[@]}"; do
# as additions, with "no focused-test mapping".
.AGENTTRASH/*|*/.AGENTTRASH/*)
;;
# frank-primary is the lucebox DS4.1 Quadlet (2026-09-26 cutover on
# :7770): its entrypoint and healthcheck live under
# dankserver/frank-primary/ and the container mounts them straight out
# of the checkout, so any file here changes deployed state. The
# radiance quadlet that held this endpoint before that date is archive
# material in dankserver/.AGENTTRASH/ and maps to nothing.
dankserver/frank-primary/*)
# frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
# live in scripts/, which is why this case has to precede the generic
# scripts/*.service and scripts/* fallbacks below.
scripts/frank-primary.*|scripts/serve-frank-primary.sh)
add_test tests/frank_primary_unit.bats
add_test tests/llm_units.bats
add_test tests/deployment_scoping.bats
add_test tests/docs_completeness.bats
add_test tests/scripts_syntax.bats
;;
# frank-large's chat template lives in scripts/ (moved there on
# 2026-09-26 when its old home was retired): a template edit changes