feat(frank-medium): serve the Qwen-Sharp v22.4.0 chat template (froggeric)
The template was staged in the working tree (then stashed by a peer for the prometheus deploy) but never committed, so the live container kept serving /patches/qwen-fixed-v22.3.jinja. Commit the repo-tracked template and flip the quadlet: CHAT_TEMPLATE now points at it, mounted read-only at /templates/qwen-sharp-v22.4.0.jinja. The entrypoint already passes --chat-template on the shared exec path, so this applies to the FP8 shape. Rollback: one env flip back to /patches/qwen-fixed-v22.3.jinja.
This commit is contained in:
1 parent
5c1006dda6
commit
6bccff3cd2
2 files changed
+463
-4
No files matched your search
@@ -48,9 +48,9 @@
|
||||
# its patches are applied at EVERY container start by the entrypoint), /r4d
|
||||
# (the patched libr4d build; the image's shipped r4d.so NaNs the
|
||||
# gated-delta-net and must never be used), and the entrypoint + healthcheck
|
||||
# scripts from this checkout (mounted read-only straight out of
|
||||
# /home/dank/serverAppConfigs — deploy.sh git-pulls the checkout before
|
||||
# dispatching).
|
||||
# scripts and the chat template from this checkout (mounted read-only
|
||||
# straight out of /home/dank/serverAppConfigs — deploy.sh git-pulls the
|
||||
# checkout before dispatching).
|
||||
#
|
||||
# IMAGE. The unit runs the localhost/-aliased tag: this host's podman quadlet
|
||||
# path cannot resolve registry-prefixed refs from local storage at create
|
||||
@@ -85,9 +85,14 @@ Environment=HIP_VISIBLE_DEVICES=0,1
|
||||
# The FP8 checkpoint carries its own single-layer MTP head (mtp.layers.0),
|
||||
# so MTP self-speculation needs no external drafter and no fp8_mtp.py
|
||||
# rewrite step. The MXFP4 checkpoint (rollback) ships its fp8 MTP head too.
|
||||
# Chat template: the Qwen-Sharp variant (froggeric v22.4.0) — the terse
|
||||
# system prompt (opt out per-request with chat_template_kwargs {"terse":
|
||||
# false}), message.reasoning support (vLLM's thinking field), and the
|
||||
# tightened tool-call rendering. The predecessor (qwen-fixed-v22.3.jinja in
|
||||
# the /patches clone) remains there as the rollback target: flip CHAT_TEMPLATE.
|
||||
Environment=SNAP=/models/Qwen3.8-27B-Uncensored-FP8
|
||||
Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
|
||||
Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
|
||||
Environment=CHAT_TEMPLATE=/templates/qwen-sharp-v22.4.0.jinja
|
||||
# Fleet identity: the id at /v1/models and in every response body. Every
|
||||
# client (Hermes, the harness, services.json) points at frank-medium.
|
||||
Environment=SERVED_NAMES=frank-medium
|
||||
@@ -140,6 +145,7 @@ Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
|
||||
# Entrypoint + healthcheck from this checkout, mounted read-only.
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
|
||||
Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/qwen-sharp-v22.4.0.jinja:/templates/qwen-sharp-v22.4.0.jinja:ro
|
||||
# The entrypoint must be the container command; the image's own entrypoint
|
||||
# (which the patched script execs at the end) must not run unpatched.
|
||||
PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
|
||||
|
||||
Reference in new issue
Block a user