add conversation-token battery and eno1 route preference

bench-conversation.py answers 'what does this endpoint feel like': ten real
jobs streamed once, TTFT split from decode tok/s, salted so no arm inherits a
warm cache, optional concurrent burst for capacity. Needed to compare the two
tensor-parallel runtimes the 2026-09-20 board swap unblocked.

dankserver-route-primary.sh pins eno1 (the fleet address, .235) as the
preferred default route after the board swap brought up eno2 with DHCP and a
lower metric; eno2 stays up as failover. nmcli device reapply, not connection
up, so the link this session rides is not torn down.

scripts/bench-conversation.py

@@ -0,0 +1,273 @@
+#!/usr/bin/env python3
+"""bench-conversation.py — what an endpoint actually feels like, in tokens/s.
+
+The other harnesses here answer engine questions (which batch, which proposer,
+which cache). This one answers the user question: give the same ten jobs to an
+OpenAI-compatible endpoint and report the rate it answers them at. Every job is
+a real conversation turn — system prompt plus one user turn — because that is
+the shape the fleet serves, and it is measured by STREAMING so prefill and
+decode are separated:
+
+  ttft          seconds to the first content delta  -> prefill + queue
+  decode tok/s  completion_tokens / (total - ttft)  -> the rate you read at
+
+Both arms of a comparison must run the SAME prompt bytes, so a per-run salt
+rides in the system prompt: it changes the prompt itself, which is llama.cpp's
+prefix-cache key and vLLM's APC key, so no arm silently inherits a warm cache
+from the one before it. Run one arm, restart the engine, run the next with a
+new salt.
+
+Answers are generated at temperature 0 so the arms can be compared for
+identity as well as speed. Identity is only meaningful across configurations
+of the SAME model — different runtimes serving different quants legitimately
+answer differently, which is why the first line of every answer is printed.
+
+Usage:
+    bench-conversation.py <base-url> [--model NAME] [--salt TAG]
+                          [--max-tokens N] [--burst N] [--out file.json]
+
+    bench-conversation.py http://127.0.0.1:7770 --model frank-medium --salt L1
+    bench-conversation.py http://127.0.0.1:7770 --model frank-medium --salt R2 --burst 4
+
+One engine at a time: on a single GPU pair there is no room for two copies of a
+27B model, and the benchmarking policy requires the production service to be
+stopped while anything measures.
+"""
+
+import argparse
+import json
+import math
+import statistics
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+
+# Ten jobs that cover what this fleet is actually asked to do. Ordered roughly
+# by prefill size so a runtime that is good at short prompts and one that is
+# good at long ones cannot hide behind an average.
+SYSTEM = (
+    "You are a staff engineer on a homelab fleet. Be concise and concrete. "
+    "Salt: {salt}"
+)
+
+BATTERY = [
+    ("greet", 128, "Say hello in one sentence."),
+    ("shell", 192, "One bash command: find every file under /home modified in "
+     "the last 2 days, sorted by size, largest first."),
+    ("math", 256, "A train covers 180 km in 1h40m, then 240 km in 2h20m. "
+     "What is the average speed in km/h? Show the arithmetic."),
+    ("json", 384, "Emit exactly 12 JSON records, one per line, no prose, each "
+     'with keys "host", "service", "port" and "up" for hosts dankclaw, '
+     "dankserver and dankware running llama-server, caddy and grafana."),
+    ("review", 512, "Review this function for bugs and name the failure mode:\n"
+     "def rotate(cfg, n):\n    for k in cfg:\n        cfg[k] = cfg[k][n:] + cfg[k][:n]\n"
+     "    return cfg\n"),
+    ("explain", 448, "Explain in four sentences why PCIe atomic operation routing "
+     "matters for GPU collectives, and what fails when a bridge reports "
+     "AtomicOpsCap: Routing-."),
+    ("translate", 256, "Translate to German, then to Japanese, one line each: "
+     "The load balancer refused the connection because the port was still held."),
+    ("plan", 640, "Give a 6-step runbook for swapping the model behind a live "
+     "LLM endpoint on a host that serves real users, with a rollback at each step."),
+    ("debug", 512, "A systemd user service restarts every 10 seconds and its log "
+     "ends with 'Error starting server failed to bind udp listener on 10.89.1.1:53: "
+     "Cannot assign requested address'. What is the root cause and the repair?"),
+    ("longdoc", 512, None),  # filled from LONG_FILES at runtime
+]
+
+# The long-context job reads real repo files so prefill is large and identical
+# across arms (~30k characters, roughly 8k tokens).
+LONG_FILES = [
+    "/home/dank/serverAppConfigs/AGENTS.md",
+    "/home/dank/serverAppConfigs/serverAppConfigs-docs/LLM/"
+    "frank-medium-llamacpp-dflash2.md",
+]
+LONG_CHARS = 30_000
+
+
+def long_prompt() -> str:
+    parts = []
+    for path in LONG_FILES:
+        try:
+            with open(path, encoding="utf-8", errors="replace") as handle:
+                parts.append(handle.read())
+        except OSError:
+            continue
+    text = "\n\n".join(parts)[:LONG_CHARS]
+    return ("Using only this document, answer: which deployment shapes for "
+            "two GPUs are ruled out by missing PCIe atomics, and what is the "
+            "one software path that still lets both cards serve one model? "
+            "Cite the measured numbers.\n\n---\n" + text)
+
+
+def stream_chat(base: str, model: str, messages, max_tokens: int, timeout: int):
+    """One streamed chat completion -> (ttft, total, completion_tokens, text).
+
+    Returns ttft=None when the engine produced no content delta.
+    """
+    body = {
+        "model": model,
+        "messages": messages,
+        "max_tokens": max_tokens,
+        "temperature": 0,
+        "stream": True,
+        "stream_options": {"include_usage": True},
+    }
+    request = urllib.request.Request(
+        base + "/v1/chat/completions",
+        data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"},
+    )
+    start = time.perf_counter()
+    ttft = None
+    deltas = 0
+    usage_tokens = None
+    text = []
+    with urllib.request.urlopen(request, timeout=timeout) as response:
+        for raw in response:
+            line = raw.decode("utf-8", "replace").strip()
+            if not line.startswith("data:"):
+                continue
+            payload = line[5:].strip()
+            if payload == "[DONE]":
+                break
+            try:
+                chunk = json.loads(payload)
+            except json.JSONDecodeError:
+                continue
+            if chunk.get("usage"):
+                usage_tokens = chunk["usage"].get("completion_tokens")
+            for choice in chunk.get("choices", []):
+                piece = (choice.get("delta") or {})
+                content = piece.get("content") or piece.get("reasoning_content") or ""
+                if content:
+                    if ttft is None:
+                        ttft = time.perf_counter() - start
+                    deltas += 1
+                    text.append(content)
+    total = time.perf_counter() - start
+    return ttft, total, (usage_tokens or deltas), "".join(text)
+
+
+def usage_prompt(base: str, model: str, messages, max_tokens: int):
+    """prompt_tokens for a job, from one non-streamed call with max_tokens=1."""
+    body = {
+        "model": model,
+        "messages": messages,
+        "max_tokens": 1,
+        "temperature": 0,
+        "stream": False,
+    }
+    request = urllib.request.Request(
+        base + "/v1/chat/completions",
+        data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"},
+    )
+    with urllib.request.urlopen(request, timeout=300) as response:
+        return json.load(response).get("usage", {}).get("prompt_tokens")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("base")
+    ap.add_argument("--model", default="frank-medium")
+    ap.add_argument("--salt", default=f"C{int(time.time())}")
+    ap.add_argument("--max-tokens", type=int, default=None)
+    ap.add_argument("--burst", type=int, default=0,
+                    help="also run N concurrent copies of the plan job")
+    ap.add_argument("--timeout", type=int, default=1800)
+    ap.add_argument("--out", default=None)
+    args = ap.parse_args()
+
+    base = args.base.rstrip("/")
+    rows = []
+    print(f"# battery salt={args.salt} model={args.model} base={base}\n")
+    print(f"{'job':<10} {'ptok':>7} {'ctok':>6} {'ttft_s':>8} {'dec_tps':>8} "
+          f"{'total_s':>8}  answer")
+
+    for name, cap, prompt in BATTERY:
+        if prompt is None:
+            prompt = long_prompt()
+        messages = [
+            {"role": "system", "content": SYSTEM.format(salt=args.salt)},
+            {"role": "user", "content": prompt},
+        ]
+        limit = args.max_tokens or cap
+        try:
+            ptok = usage_prompt(base, args.model, messages, limit)
+            ttft, total, ctok, text = stream_chat(
+                base, args.model, messages, limit, args.timeout)
+        except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError) as exc:
+            print(f"{name:<10} FAILED {exc}")
+            rows.append({"job": name, "error": str(exc)})
+            continue
+        decode = None
+        if ttft is not None and ctok and total > ttft:
+            decode = (ctok - 1) / (total - ttft)
+        prefill_rate = (ptok / ttft) if (ptok and ttft) else None
+        first = text.strip().splitlines()[0][:48] if text.strip() else "(empty)"
+        print(f"{name:<10} {ptok or 0:>7} {ctok:>6} "
+              f"{(ttft or 0):>8.2f} {(decode or 0):>8.1f} {total:>8.2f}  {first}")
+        rows.append({"job": name, "prompt_tokens": ptok, "completion_tokens": ctok,
+                     "ttft_s": ttft, "decode_tps": decode, "prefill_tps": prefill_rate,
+                     "total_s": total, "answer_head": first})
+
+    good = [r for r in rows if r.get("decode_tps")]
+    if good:
+        rates = [r["decode_tps"] for r in good]
+        ttfts = [r["ttft_s"] for r in good]
+        pre = [r["prefill_tps"] for r in good if r.get("prefill_tps")]
+        print(f"\ngeomean decode  {statistics.geometric_mean(rates):8.1f} tok/s"
+              f"   (min {min(rates):.1f} / max {max(rates):.1f})")
+        print(f"mean ttft       {statistics.mean(ttfts):8.2f} s")
+        if pre:
+            print(f"geomean prefill {statistics.geometric_mean(pre):8.1f} tok/s")
+
+    if args.burst > 1:
+        prompt = next(p for n, _, p in BATTERY if n == "plan")
+        messages = [
+            {"role": "system", "content": SYSTEM.format(salt=args.salt + "B")},
+            {"role": "user", "content": prompt},
+        ]
+        results = []
+        lock = threading.Lock()
+
+        def worker():
+            try:
+                ttft, total, ctok, _ = stream_chat(
+                    base, args.model, messages, 640, args.timeout)
+                with lock:
+                    results.append((ttft, total, ctok))
+            except Exception as exc:  # noqa: BLE001 - report, do not abort the burst
+                with lock:
+                    results.append((None, 0.0, str(exc)))
+
+        threads = [threading.Thread(target=worker) for _ in range(args.burst)]
+        t0 = time.perf_counter()
+        for thread in threads:
+            thread.start()
+        for thread in threads:
+            thread.join()
+        wall = time.perf_counter() - t0
+        ok = [r for r in results if r[0] is not None]
+        if ok:
+            tokens = sum(r[2] for r in ok)
+            print(f"burst {args.burst}x plan: wall {wall:.1f}s  aggregate "
+                  f"{tokens / wall:.1f} tok/s  "
+                  f"per-stream {statistics.mean([r[2] / r[1] for r in ok]):.1f} tok/s"
+                  f"  mean ttft {statistics.mean([r[0] for r in ok]):.2f}s")
+        else:
+            print(f"burst {args.burst}x plan: all failed: {results[0][2]}")
+
+    if args.out:
+        with open(args.out, "w", encoding="utf-8") as handle:
+            json.dump({"salt": args.salt, "model": args.model, "base": base,
+                       "rows": rows, "burst": args.burst}, handle, indent=1)
+        print(f"wrote {args.out}")
+    return 0 if good else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/dankserver-route-primary.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Pin Dankserver's fleet NIC (eno1, 192.168.0.235) as the primary default route.
+#
+# The 2026-09-20 board swap (TUF GAMING B650-E -> ProArt B850-Creator WiFi NEO)
+# brought up a second port, eno2, which took DHCP and — with NetworkManager's
+# default per-connection metric of 100 — came up with metric 101 while eno1's
+# static gateway got 102. Outbound traffic therefore started leaving by eno2
+# even though every fleet peer, Caddy upstream and monitoring target addresses
+# this host as .235. Keep eno2 (it is a real second link) but make it secondary:
+# route metrics decide the default route, so eno1 wins while eno2 stays a
+# failover.
+#
+# Run on Dankserver.  Idempotent; NetworkManager owns these connections (the
+# netplan files under /etc/netplan are its generated shim, so edit via nmcli).
+#
+#   bash scripts/dankserver-route-primary.sh [--check]
+set -euo pipefail
+
+PRIMARY='Wired connection 1'      # eno1, static 192.168.0.235, fleet address
+SECONDARY='Wired connection 2'    # eno2, DHCP 192.168.0.247
+PRIMARY_IF='eno1'
+SECONDARY_IF='eno2'
+PRIMARY_METRIC=100
+SECONDARY_METRIC=200
+
+show() {
+    printf 'routes:\n'; ip route show default
+    printf 'wanted: %s metric %s, %s metric %s\n' \
+        "$PRIMARY" "$PRIMARY_METRIC" "$SECONDARY" "$SECONDARY_METRIC"
+}
+
+if [[ "${1:-}" == '--check' ]]; then
+    for conn in "$PRIMARY" "$SECONDARY"; do
+        printf '%s: ipv4=%s ipv6=%s\n' "$conn" \
+            "$(nmcli -g ipv4.route-metric connection show "$conn")" \
+            "$(nmcli -g ipv6.route-metric connection show "$conn")"
+    done
+    show
+    exit 0
+fi
+
+nmcli connection modify "$PRIMARY" \
+    ipv4.route-metric "$PRIMARY_METRIC" ipv6.route-metric "$PRIMARY_METRIC"
+nmcli connection modify "$SECONDARY" \
+    ipv4.route-metric "$SECONDARY_METRIC" ipv6.route-metric "$SECONDARY_METRIC"
+
+# `nmcli device reapply <ifname>` pushes the edited profile without tearing the
+# link down.  `nmcli connection up` would re-run DHCP and briefly drop the route
+# this session rides on, and `connection reapply` is not a verb (NM 1.54).
+nmcli device reapply "$PRIMARY_IF"
+nmcli device reapply "$SECONDARY_IF"
+
+show
+if [[ "$(ip route show default | head -1)" != *"dev $PRIMARY_IF"* ]]; then
+    printf 'ERROR: eno1 is still not the preferred default route\n' >&2
+    exit 1
+fi
+printf 'eno1 is the preferred default route; eno2 remains up as failover.\n'