dank/command-center
1
0 Fork 0
Code Issues 11 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

refactor(omp): rename fleet models to unit-name callsigns and refit windows

Browse Source 
frank-medium -> frank-archon (dankserver), frank-large -> frank-dragoon
(dankclaw-gpu), frank-small -> frank-zealot (dankclaw-3090). The Dankclaw
service/unit names keep the old frank-large spelling; only the OMP model ids
move, per the comments at each entry.

Windows refit to the served context minus the compaction headroom:
  archon  196608 -> 200000 ctx, 32768 -> 16384 maxTokens
  dragoon 200192 -> 217088 ctx, 57344 -> 45056 maxTokens, input +image
  zealot  262144 -> 200704 ctx, 32768 -> 16384 maxTokens
All three now carry compactionModel frank-zealot, reasoningContentField
reasoning_content, and Qwen-recommended sampling in compat.extraBody
(merged last, so it beats the global -1 sampler settings).
This commit is contained in:
Frank committed 2026-10-09 13:17:40 -07:00
1 parent 40c7d3c752
commit 0cb002123b
1 file changed
+189 -110
Download Patch File Download Diff File Expand all files Collapse all files
agent/models.yml
+189 -110
View File
@@ -4,92 +4,81 @@
# - apiKey "EMPTY" -> auth: none (keyless local endpoint) # - apiKey "EMPTY" -> auth: none (keyless local endpoint)
# - thinkingLevelMap {max: xhigh} -> compat.reasoningEffortMap # - thinkingLevelMap {max: xhigh} -> compat.reasoningEffortMap
providers: providers:
# The provider NAMES are historical (they were llama.cpp endpoints when the # Provider NAMES are labels, and every reference to them lives in config.yml
# fleet ran llama.cpp): frank-medium is vllm-radiance now, and frank-large is # (modelRoles, agentModelOverrides) or in another model's compactionModel —
# halogen-flash-server again. Renaming is deliberately avoided because every # rename one and a missed reference fails at call time. So each label is
# modelRoles entry in config.yml embeds the provider name, and a partial # migrated with its model in the same edit. One provider per HOST:PORT, so
# rename fails at call time — the label is cosmetic, the references are not. # Dankclaw's second endpoint (:7771) is a separate block rather than a second
# One provider per HOST:PORT, so Dankclaw's second endpoint (:7771) is a # model inside the :7770 one.
# separate block rather than a second model inside the :7770 one.
"llamacpp (dankserver)": "llamacpp (dankserver)":
baseUrl: http://192.168.0.235:7770/v1 baseUrl: http://192.168.0.235:7770/v1
api: openai-completions api: openai-completions
auth: none auth: none
models: models:
- id: frank-medium - id: frank-archon
name: "frank-medium (dankserver)" # Model id renamed from frank-primary on 2026-10-08 (official
# Dankserver: both Radeon AI PRO R9700 (gfx1201) serving one model — # unit-name scheme; the systemd service, scripts and runbooks keep
# mainline llama.cpp, `--split-mode layer`, on # the frank-primary name — service names are unchanged).
# ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP at UD-Q8_K_XL with # Server-side compaction routes to compaction.remoteEndpoint in
# the z-lab DFlash2 Q8_0 drafter. Tensor parallelism is impossible on # config.yml (192.168.0.143:7771 — frank-zealot's 3090); the
# this host (the second card's chipset path has no PCIe atomic routing, # compactionModel candidate keeps the wire model id valid there.
# so it cannot dispatch an RCCL collective — and vLLM's pipeline compactionModel: llamacpp (dankclaw-3090)/frank-zealot
# parallelism needs the same p2p primitive), which is why one process name: "frank-archon (dankserver)"
# layer-splits across both cards instead. Since 2026-09-16 this is the # Dankserver: BOTH Radeon AI PRO R9700s (gfx1201, TP=2 over the
# box's only LLM runtime; the vllm-radiance replica pair is retired. # 2026-09-20 board swap's root ports, wht6 wire). Engine:
contextWindow: 196608 # radiance (StillDeadcode, Apache-2.0) from the digest-pinned
# The KV pool is 393216 positions over TWO slots, so one conversation # stilldeadcode/radiance image (index digest sha256:4c3f4845..., 287
# carries 196608 -- the house rule (see frank-large) is to declare the # MiB, kernels prebuilt for gfx1201), plain `podman run` through
# per-conversation window, not the pool. The checkpoint's trained # scripts/serve-frank-primary.sh, root unit (User=dank, dank's
# window is 262144 and the GGUF has no rope scaling, so the top 131k # rootless podman storage). Serving the OFFICIAL
# of a conversation extrapolates on the base rope. 32768 keeps prompt # Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE modelopt block-FP8
# + budget comfortably inside it; thinking counts against the budget. # precision — a rad-convert no-recipe pure copy into
maxTokens: 32768 # qwen4exp-fp8-native.rad (172.76 GiB measured by plan-only), built by
# scripts/build-frank-primary-rad.sh against a staging dir whose
# config.json drops quantization_config so the shipped qwen4exp
# plugin's (qwen4exp, "") selection key matches; per-weight
# encodings are untouched. Experts stream from a 40 GiB/rank
# pinned host pool; the 47.7 GiB PLE ngram table is pinned in RAM
# (Frank's call); KV fp8; MTP depth 3 (the checkpoint's own mtp.*
# head); prefix cache 4 GiB host + 64 GiB disk. Chat template:
# froggeric's fixed qwen3.8 v22.5
# (scripts/qwen38-froggeric-v22.5.jinja) via
# --override-chat-template. Rollback: git tag
# frank-primary-freetoken-pr587-20261005 (FreeToken, one card).
contextWindow: 200000
# maxTokens must survive the harness compaction gap AND the chars/4
# token-estimate under-run on code-heavy prompts. The clamp is
# max_tokens = min(maxTokens, contextWindow − estimate − 4096); the
# estimate under-runs by ~3 % on code. Compaction fires at 85 % of
# the window = 170,000 est; worst-case prompt + budget ≈
# 170,000 × 1.03 + 16,384 ≈ 191,500 ≤ 200,000 — the engine
# refuses a longer request, never truncates.
maxTokens: 16384
reasoning: true reasoning: true
# Text + image: the official FP8 checkpoint carries the vision
# tower (vision_config depth 27; the first blocks stay bf16 in
# modules_to_not_convert) and the plugin declares it
# (arch/common/rad_block_vit.h). Video preprocessing configs ship
# too; images are the contract that matters here.
input: [text, image] input: [text, image]
compat: compat:
# Qwen3.8's chat template accepts ONLY xhigh, medium and low for # radiance's --reasoning-format defaults to auto: thinking
# reasoning_effort and RAISES on anything else — verified against this # comes back in `reasoning_content` (config.cpp). No parser
# endpoint on 2026-09-16: low/medium/xhigh answer 200, `high` returns # flags exist or are needed.
# a 500 from the template. The harness's levels must therefore be reasoningContentField: "reasoning_content"
# folded onto that set, exactly as the sibling frank-small entry does, # The froggeric template does not gate reasoning_effort the way
# or any role whose level is unmapped fails on its first request # Qwen's shipped template did, and radiance special-cases only
# (subagents run at `high`). # none/minimal (core/server/oai.cpp:609). The map stays as the
# zero-cost safety net, same as frank-zealot's.
reasoningEffortMap: reasoningEffortMap:
minimal: low minimal: low
low: low low: low
medium: medium medium: medium
high: xhigh high: xhigh
max: xhigh max: xhigh
# Qwen recommended sampling. extraBody is merged last and wins.
"llamacpp (dankclaw)": # Every key is inside radiance's known-parameter list
baseUrl: http://192.168.0.143:7770/v1 # (core/server/oai.cpp:194-205); unknown keys are refused 400.
api: openai-completions
auth: none
models:
- id: frank-large
name: "frank-large (dankclaw-gpu)"
# Server-side compaction always runs on frank-medium (see
# compaction.remoteEndpoint in config.yml); this steers the
# compaction model candidate so the wire model id stays valid
# on the frank-medium endpoint even from a frank-large session.
compactionModel: llamacpp (dankserver)/frank-medium
# halogen-flash-server. The model's native window is 262,144, but this
# deployment caps a conversation at 200,192 — exactly half of its
# 400,384-position KV pool, so two 200k conversations are resident at
# once. Declaring the half rather than the native window is what keeps
# prompt + budget inside one conversation's region of the pool.
contextWindow: 200192
# HALOGEN_MAX_TOKENS_CAP: above this a request is a 400 rather than a
# silent truncation, and thinking counts against the budget.
# maxTokens must also survive the harness compaction gap: OMP compacts at
# 70% of contextWindow = 140,134, so a pre-compaction prompt can legally
# reach that height. 140,134 + 57,344 = 197,478 <= 200,192, which closes
# the dead zone that 400'd a request on 2026-09-17 (prompt 138,898 +
# budget 64,000 = 202,898 > 200,192: above the server's fit check, below
# the compaction trigger). Any budget > 60,058 reopens the zone.
maxTokens: 57344
reasoning: true
# Text only as of 2026-09-16: the vision tower moved to frank-small and
# this server refuses image content parts.
input: [text]
compat:
reasoningEffortMap:
max: xhigh
# Qwen3.8-Flash-Next recommended sampling. Halogen has no server-side
# sampler defaults (a request that sends no temperature decodes
# greedily), so this override is what makes ordinary calls sample.
# extraBody is merged last (after compat policy) and wins.
extraBody: extraBody:
temperature: 1.0 temperature: 1.0
top_p: 0.95 top_p: 0.95
@@ -97,57 +86,147 @@ providers:
min_p: 0.0 min_p: 0.0
presence_penalty: 0.0 presence_penalty: 0.0
repetition_penalty: 1.0 repetition_penalty: 1.0
# Keep prior thinking blocks in the rendered prompt across turns; chat_template_kwargs:
# halogen honours this key top-level (verified: 275-token prompt enable_thinking: true
# delta with it false). preserve_thinking: true
preserve_thinking: true
"llamacpp (dankclaw-3090)": "llamacpp (dankclaw)":
baseUrl: http://192.168.0.143:7771/v1 baseUrl: http://192.168.0.143:7770/v1
api: openai-completions api: openai-completions
auth: none auth: none
models: models:
- id: frank-small - id: frank-dragoon
name: "frank-small (dankclaw-3090)" # Model id renamed from frank-large on 2026-10-08 (official
# See frank-large: server-side compaction always runs on # unit-name scheme; the service/unit/scripts keep the frank-large
# frank-medium, so the wire model id stays valid from a # name — service names are unchanged).
# frank-small session. name: "frank-dragoon (dankclaw-gpu)"
compactionModel: llamacpp (dankserver)/frank-medium # Server-side compaction runs on the compaction.remoteEndpoint in
# Dankclaw's RTX 3090: mainline llama.cpp serving # config.yml (192.168.0.143:7771 — frank-zealot's 3090); this steers the
# HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M. # compaction model candidate so the wire model id stays valid
# The MTP head ships as a separate assistant sidecar # there even from a frank-dragoon session.
# (mtp-gemma-4-12B-it.gguf) loaded via --model-draft and chained compactionModel: llamacpp (dankclaw-3090)/frank-zealot
# behind the draftless ngram-mod matcher; q8_0 KV with the model's # halogen-flash-server 0.16.2 (ghcr.io/peonist-ai/
# sliding-window pattern keeps each full 262144-token window cheap, # halogen-flash-server@sha256:0c61bf84, digest-pinned) serving the
# and the explicit CTX_SIZE=1048576 / PARALLEL=4 gives FOUR full # Quat3rnion ABLITERATED Swift1.5 finetune of peonist's
# windows (measured 22.3 GiB of the 24 GiB card at load; a 5th slot # Qwen3.8-Flash-Next v2 checkpoint (qwen38-flash-next-v2-swift15-
# does not fit). The BF16 projector (with audio encoder) is loaded # abliterated.hgn, ~62 GiB resident, + REQUIRED n-gram table +
# too, so this stays multimodal (frank-medium gained vision on # vision tower) on the Strix Halo iGPU: TWO slots at
# 2026-09-17 as well; frank-large has no vision tower). # HALOGEN_CTX=262144 each with the MTP head ALWAYS ON
contextWindow: 262144 # (HALOGEN_SPEC_ADAPT=0 — the default adaptive policy halves
# Each slot IS the full trained window: prompt + budget must fit # decode). Card: 53-57 t/s greedy, ~1,640 t/s prefill at 34k, ~72%
# inside it, and thinking counts against the budget. # fewer thinking tokens than the censored 27B. ABLITERATED base
maxTokens: 32768 # (refusals should not return) and VISION ON (OpenAI image_url
# parts; >=256 px images or it answers "video"). Runbook:
# serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md.
# Per-SLOT window the harness may fill — deliberately BELOW the
# engine ceiling (HALOGEN_CTX=262144): the clamp below leaves a
# 45k gap that absorbs the estimate error.
contextWindow: 217088
# The clamp is max_tokens = min(maxTokens, contextWindow − est −
# 4096); the chars/4 est UNDER-counts the rendered prompt by ~3 %
# on code/agent traffic (measured 2026-10-07: est 152,938 ->
# server 157,388). Worst-case request = ctx − 4096 + 3 % est
# error ≈ 219k, inside the 262,144 engine ceiling with ~43k
# margin (robust to a 10 % under-count). Compaction fires at 85 %
# of the window = 184,524 est ≈ 190k measured (Frank's 180k
# target) and rides on frank-zealot: input ≈ threshold −
# keepRecent(20k) ≈ 170k, inside its 200,704 slot.
# HALOGEN_MAX_TOKENS_CAP in frank-large.env must stay >= this.
maxTokens: 45056
reasoning: true reasoning: true
# VISION ON as of 2026-10-06: the v2 vision tower loads beside the
# checkpoint (HALOGEN_VISION_TOWER; red-PNG probe verified on the
# card). image_url content parts only — the legacy top-level
# images field is the frank-secondary trap, do not repeat it here.
input: [text, image] input: [text, image]
compat: compat:
# Gemma4's chat template accepts every reasoning_effort level as of # flash-server emits thinking as `reasoning_content` — same
# 2026-09-17 (all seven probed: 200), so this map is inert. Kept # field as every engine this endpoint has run, no harness
# anyway: it cost nothing, and it is the safety net if the template # change needed.
# ever gates again the way Qwen3.8's did (subagents run at `high`). reasoningContentField: "reasoning_content"
# Same Qwen3.8 template family (it ships inside the tokenizer
# bundle): any reasoning_effort outside {xhigh, medium, low} is
# rejected, not clamped — the fold stays mandatory.
reasoningEffortMap: reasoningEffortMap:
minimal: low minimal: low
low: low low: low
medium: medium medium: medium
high: xhigh high: xhigh
max: xhigh max: xhigh
# Gemma's reference local-recipe sampling (temp 1.0 / top_p 0.95); # The card's thinking recipe, on the wire. The flash-server
# the server's own default is greedy for clients that stay silent. # engine HAS a server-side sampler (HALOGEN_TEMPERATURE/TOP_P/
# Thinking is already retained across turns by the server's # TOP_K = 1.0 / 0.95 / 20 in frank-large.env) — these match it,
# --reasoning-preserve default, and the key below is accepted # so per-request values are consistent, not load-bearing.
# top-level (verified 200, not a 400), so it is passed to pin the
# behaviour at the request rather than rely on a server default.
extraBody: extraBody:
temperature: 1.0 temperature: 1.0
top_p: 0.95 top_p: 0.95
preserve_thinking: true
"llamacpp (dankclaw-3090)":
baseUrl: http://192.168.0.143:7771/v1
api: openai-completions
auth: none
models:
- id: frank-zealot
# Model id renamed from frank-secondary on 2026-10-08 (official
# unit-name scheme; the service/unit/scripts keep the
# frank-secondary name — service names are unchanged).
name: "frank-zealot (dankclaw-3090)"
# Server-side compaction routes to compaction.remoteEndpoint in
# config.yml (192.168.0.143:7771 — this endpoint); the
# compactionModel candidate keeps the wire model id valid there.
compactionModel: llamacpp (dankclaw-3090)/frank-zealot
# Dankclaw's RTX 3090: llama.cpp (master, CUDA sm_86 build) serving
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K
# (7,359,259,008 B) — qwen35 dense hybrid (32 layers: 8 full-attention
# + 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144), TWO
# conversations (--ctx-size 401408 / --parallel 2) at Frank's 200k cap
# each. Exact KV: f16 K + f16 V. Vision via the publisher's OWN
# BF16 mmproj (921,704,448 B) — the model is natively multimodal.
# **MTP IS OFF** because the file carries no nextn/mtp tensors —
# SPEC_TYPE=none strips both spec flags. Priced from the MiMo Q6_K
# boots at this geometry (21,276/2,849 at ubatch 1024; 21,846/2,279 at
# 2048). 2026-10-07 (late) swap from Gemma4-26B-A4B Q4_K_M (1x200,704).
# Runbook:
# serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md.
# contextWindow is the PER-SLOT window (Frank's cap): a longer
# prompt needs compaction.
contextWindow: 200704
# Prompt + budget must fit inside the slot window; thinking counts
# against the budget. Compaction fires at 85 % of the window =
# 170,598 est; worst-case prompt + budget ≈ 170,598 × 1.03 +
# 16,384 ≈ 192,100 ≤ 200,704 — a longer prompt is a 400 naming
# the limit, never a truncation.
maxTokens: 16384
reasoning: true
input: [text, image]
compat:
# llama.cpp --reasoning-format deepseek emits thinking as
# `reasoning_content` — the same field the MiMo/Ornith eras used;
# harness reads unchanged.
reasoningContentField: "reasoning_content"
# The Qwen3.5 chat template has NO reasoning_effort variable (unlike
# the Qwen3.8-template endpoints, where an unmapped level is a hard
# Jinja raise). llama.cpp passes chat_template_kwargs through to
# Jinja, and Jinja ignores variables a template never reads — so the
# map is inert here, kept so role-level overrides resolve identically
# across the fleet.
reasoningEffortMap:
minimal: low
low: low
medium: medium
high: xhigh
max: xhigh
# The Qwen3.5 publisher recipe (model card, thinking mode):
# temperature 0.6 / top_p 0.95 / top_k 20 / min_p 0. Non-thinking
# mode is 0.7/0.8/20/0 — the harness runs thinking mode.
# extraBody is merged last and wins over the harness default.
extraBody:
temperature: 0.6
top_p: 0.95
top_k: 20
min_p: 0.0
presence_penalty: 0.0
chat_template_kwargs:
enable_thinking: true
# (no preserve_thinking: that kwarg belongs to the Qwen3.8
# templates; nothing in this template reads it)