3.2 KiB
LLM Endpoint Inventory
frank-secondary (Dankclaw, port 7771)
Current: Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K (llama.cpp, RTX 3090, 200,704-token slots, vision via BF16 mmproj).
Target: Underdog-Saluki-27B-1.0 (llama.cpp, RTX 3090, 400,000-token slots, vision via BF16 mmproj).
Both run on the same hardware with the same runtime stack. The swap is model-only; container image and systemd unit remain unchanged. Vision support persists via the publisher's BF16 mmproj pass-through (--mmproj).
Context window law: 400,000 tokens is the per-slot hard ceiling (--ctx-size 401408 --parallel 2). The harness clamps to min(45056, contextWindow − est − 4096) to keep room for the MTP head. With 400K context, a 45K token request leaves 355K for MTP head + estimation buffer.
frank-primary (Dankserver, port 7770)
Radiance 1.0.13, ukisai/Swift1.5-Qwen3.8-Flash-Next BF16, 49,152 experts GPTQ w4 + 24 bf16-protected experts, 200K max-model-len, 47.7 GiB n-gram pinned to host RAM, 2 R9700s TP=2 (--tp-wire wht6), MTP spec decode depth 2, --max-num-seqs 3, --max-model-len 200000.
frank-large (Dankclaw, port 7770)
Halogen-flash-server 0.16.2, Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, 262,144 context per-slot, 2 slots × 262,144 ctx, MTP spec decode ALWAYS-ON (HALOGEN_SPEC_ADAPT=0), vision ON (v2 tower loads beside checkpoint), 524,288 positions KV pool.
Runtime Stack
- frank-primary: Radiance 1.0.13 (StillDeadcode, digest-pinned
stilldeadcode/radiance@sha256:4c3f484564…), plainpodman run, root systemd unit, 2 R9700s TP=2. - frank-large: Halogen-flash-server 0.16.2 (
ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…), digest-pinned, plainpodman runviascripts/serve-frank-large.sh, root systemd unit, Strix Halo iGPU. - frank-secondary: llama.cpp master (0.5.0-dev, commit
5fc4f3c), CUDA sm_86 build, root systemd unit, RTX 3090.
All LLM services are root systemd units with Restart=always. Container exit codes drive service restarts (ExecStop=podman stop -t 60 <name>). Boot persistence requires quadlet units in ~/.config/containers/systemd/<name>.container, enabled in user manager, with linger on for the deploy user.
Model Config Resolution
Official model ids are StarCraft unit names: frank-archon → 192.168.0.235:7770 (FRANK_PRIMARY_SERVED_NAME), frank-dragoon → 192.168.0.143:7770 (HALOGEN_MODEL_ID), frank-zealot → 192.168.0.143:7771 (ALIAS).
Per-slot vs pool: llama.cpp reports per-slot window (--ctx-size 401408 --parallel 2), not the pool. A 2×200K pool means 200K per slot, not 400K total.
Reasoning field: llama.cpp and SGLang emit reasoning_content; vLLM emits reasoning. models.yml sets reasoningContentField per endpoint to match.
Vision: All three endpoints accept input: [text, image]. Radiance runs the checkpoint's 27-layer Qwen-VL tower (patching, pad-token expansion, 3-part rotary, --mm-max-patches available). Halogen v2 loads the vision tower beside the checkpoint. Llama.cpp passes the model's BF16 mmproj via --mmproj.
Audio: No audio input anywhere in the fleet (left with FLM NPU endpoint 2026-09-11, returned 2026-09-17, left again 2026-10-01).