Commit Graph
3 Commits
Author SHA1 Message Date
Frank 69f87c8483 feat(llm): rename inference services dropping 'default' from names
frank-default-large -> frank-large (Dankclaw :7770), frank-default-small ->
frank-small (Dankserver :7770), frank-default-sightsound -> frank-npu
(Dankclaw :7771). Units, env files, serve scripts, deploy-plan mappings,
deploy-remote dispatch (stop/disable old units + relink + enable new), and the
test seams move together. Retired frank-default-npu.* artifacts moved to
scripts/.AGENTTRASH/ (tracked-in-history; ignored on disk).

Served aliases become frank-large / frank-small via --alias; the NPU endpoint
serves the whole FLM registry, so its rename changes no served tags.
2026-09-03 16:15:43 -07:00
Frank 214bad946f bench: fix spec-decode script — use local commands, not nested SSH
The script runs ON Dankclaw but was calling ssh() to SSH back to
dankclaw for service management. Nested SSH fails (no key auth for
localhost). Replaced ssh() with local() using subprocess.run(shell=True)
for systemctl and sed commands.
2026-08-28 00:28:02 -07:00
Frank 0ff35e6620 bench: spec decode sweep + fix serve script ngram-only path
serve-frank-large.sh: only pass --model-draft when SPEC_TYPE contains
'draft-mtp'. ngram-only configs (ngram-mod, ngram-simple, etc.) do not
need a draft model file — loading one wastes VRAM and may error.

bench-spec-decode.py: new benchmark script. Sweeps 4 configs (none,
ngram-mod, draft-mtp, draft-mtp+ngram-mod), generates 16k tokens per
config at temp 0 from a minimal prompt. Uses HTTP API (llama-bench
cannot target --spec-type). Follows benchmarking policy: stop+mask
the production service, bench, unmask+restart, health check.
2026-08-28 00:04:21 -07:00