docs(probe): price the corpus chars/token ratio at run time
The stored 3.28 chars/token (measured 2026-10-11) had drifted to 3.20 as the repo's docs grew, so leg B sent ~201,900-token prompts and took a legitimate 400 from the engine's own window check — which reads as a capacity-contract failure on a healthy endpoint. One ~4k-token request now prices the ratio, and the leg sizes to 97 % of the deployed window.
This commit is contained in:
@@ -147,10 +147,21 @@ def main():
|
||||
deep = corpus(160_000)
|
||||
wide = corpus(1_100_000)
|
||||
|
||||
# 3.28 chars/token is the measured ratio of this corpus through the
|
||||
# froggeric template (646,000 chars -> 196,919 tokens at the 2026-10-11
|
||||
# deploy); 98.5 % of the window leaves the response room under the ceiling.
|
||||
chars = int(0.985 * maxlen * 3.28)
|
||||
# The corpus's chars/token ratio is MEASURED here, not stored. A stored 3.28
|
||||
# (646,000 chars -> 196,919 tokens, 2026-10-11) had drifted to 3.20 by
|
||||
# 2026-10-20 (646,299 chars -> 201,876 tokens) as the repo's docs grew, and a
|
||||
# stale constant sends leg B past --max-model-len, where the engine answers a
|
||||
# perfectly legitimate 400 ("the request (201900 tokens) exceeds the available
|
||||
# context size (200000)") and the leg reads as a capacity failure. One small
|
||||
# request prices the ratio; 97 % of the window leaves the response room.
|
||||
sample = prompt(wide, 4000, "CAL", "cal-marker")
|
||||
cal, _, _ = post({"model": MODEL,
|
||||
"messages": [{"role": "user", "content": sample}],
|
||||
"max_tokens": 1, "temperature": 0.0,
|
||||
"chat_template_kwargs": {"enable_thinking": False}})
|
||||
ratio = len(sample) / cal["usage"]["prompt_tokens"]
|
||||
print(f"[cal] corpus ratio {ratio:.2f} chars/token")
|
||||
chars = int(0.97 * maxlen * ratio)
|
||||
# Each marker's FIRST hyphen-separated word is unique, because show()
|
||||
# scores the hit on that word — a shared prefix would score every slot HIT
|
||||
# on any answer that says the word.
|
||||
@@ -176,7 +187,7 @@ def main():
|
||||
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
||||
print(f"[B] total {time.time() - t0:.1f}s")
|
||||
# D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
|
||||
dchars = int(0.25 * maxlen * 3.28)
|
||||
dchars = int(0.25 * maxlen * ratio)
|
||||
dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
|
||||
for i in range(seqs)]
|
||||
print(f"[D] {seqs} concurrent streams x 512 tokens, "
|
||||
|
||||
Reference in New Issue
Block a user