Frank
7c77326c33
The 2026-10-08 rename moved the harness model ids and the homepage cards; the served aliases on the wire stayed frank-primary/frank-large/ frank-secondary. This closes the gap: the alias each endpoint announces is now the family name, while the services, units, scripts, env files and runbooks keep the old names (container identity stays stable). - scripts/frank-secondary.env ALIAS, scripts/frank-large.env HALOGEN_MODEL_ID, scripts/frank-primary.env FRANK_PRIMARY_SERVED_NAME (+ the two serve-script defaults/gate messages) - seams re-pinned: frank_secondary/frank_large/frank_primary unit bats - fleet harnesses retargeted: bench-probe, deep-probe, calib-push, fleet-probes-swift15, autoresearch(+measure), bench-threads-batch-http, bench-decode - shared/homepage/docs.html: api-id annotations and the "switchover will rename the ids" footnote removed — the page's names are now the ids; opencode/Pi/curl examples send frank-zealot - dankclaw/hermes/config.yaml.example: frank-large -> frank-dragoon and the live 262144 context (Hermes routes by served id; see Hermes.md) Consumers with live state re-keyed out-of-band in the same window: OpenWebUI (model row, access_grant, openai.api_configs model_ids) and the Hermes config.yaml.
49 lines
1.7 KiB
Python
Executable File
49 lines
1.7 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""bench-probe.py — RUNS ON DANKSERVER. The campaign's decode probe: one
|
|
/completion request, a prompt assembled from calib.jsonl to ~TARGET tokens,
|
|
temperature 0, max_tokens 512. Reports the engine's own timings: decode rate,
|
|
prefill rate, draft accept. Run 3x; the median is the arm's number."""
|
|
import json, sys, urllib.request
|
|
|
|
TARGET = 43600 # tokens
|
|
BYTES_PER_TOK = 3.8
|
|
URL = "http://localhost:7770/v1/completions"
|
|
|
|
def build_prompt():
|
|
want = int(TARGET * BYTES_PER_TOK)
|
|
parts = []
|
|
n = 0
|
|
with open("/home/dank/corpora/out/calib.jsonl") as f:
|
|
for line in f:
|
|
d = json.loads(line)
|
|
p = d.get("prompt") or d.get("messages", [{}])[0].get("content", "")
|
|
parts.append(p)
|
|
n += len(p.encode())
|
|
if n >= want:
|
|
break
|
|
return "".join(parts)
|
|
|
|
def probe(salt):
|
|
prompt = f"[probe {salt}]\n" + build_prompt()
|
|
body = json.dumps({
|
|
"model": "frank-archon",
|
|
"prompt": prompt, "temperature": 0.0, "max_tokens": 512,
|
|
}).encode()
|
|
req = urllib.request.Request(URL, data=body,
|
|
headers={"Content-Type": "application/json"})
|
|
with urllib.request.urlopen(req, timeout=1800) as r:
|
|
d = json.load(r)
|
|
t = d.get("timings", {})
|
|
return {
|
|
"prompt_n": t.get("prompt_n"),
|
|
"prefill_tps": round(t.get("prompt_per_second", 0), 1),
|
|
"pred_n": t.get("predicted_n"),
|
|
"decode_tps": round(t.get("predicted_per_second", 0), 2),
|
|
"draft_n": t.get("draft_n"),
|
|
"accept": (round(t["draft_n_accepted"] / t["draft_n"], 4)
|
|
if t.get("draft_n") else None),
|
|
}
|
|
|
|
if __name__ == "__main__":
|
|
print(json.dumps(probe(sys.argv[1] if len(sys.argv) > 1 else "a")))
|