docs(probe): price the corpus chars/token ratio at run time
The stored 3.28 chars/token (measured 2026-10-11) had drifted to 3.20 as the repo's docs grew, so leg B sent ~201,900-token prompts and took a legitimate 400 from the engine's own window check — which reads as a capacity-contract failure on a healthy endpoint. One ~4k-token request now prices the ratio, and the leg sizes to 97 % of the deployed window.
This commit is contained in:
1 parent
cd0be97b39
commit
1aeb7af6df
1 file changed
+16
-5
@@ -147,10 +147,21 @@ def main():
|
|||||||
deep = corpus(160_000)
|
deep = corpus(160_000)
|
||||||
wide = corpus(1_100_000)
|
wide = corpus(1_100_000)
|
||||||
|
|
||||||
# 3.28 chars/token is the measured ratio of this corpus through the
|
# The corpus's chars/token ratio is MEASURED here, not stored. A stored 3.28
|
||||||
# froggeric template (646,000 chars -> 196,919 tokens at the 2026-10-11
|
# (646,000 chars -> 196,919 tokens, 2026-10-11) had drifted to 3.20 by
|
||||||
# deploy); 98.5 % of the window leaves the response room under the ceiling.
|
# 2026-10-20 (646,299 chars -> 201,876 tokens) as the repo's docs grew, and a
|
||||||
chars = int(0.985 * maxlen * 3.28)
|
# stale constant sends leg B past --max-model-len, where the engine answers a
|
||||||
|
# perfectly legitimate 400 ("the request (201900 tokens) exceeds the available
|
||||||
|
# context size (200000)") and the leg reads as a capacity failure. One small
|
||||||
|
# request prices the ratio; 97 % of the window leaves the response room.
|
||||||
|
sample = prompt(wide, 4000, "CAL", "cal-marker")
|
||||||
|
cal, _, _ = post({"model": MODEL,
|
||||||
|
"messages": [{"role": "user", "content": sample}],
|
||||||
|
"max_tokens": 1, "temperature": 0.0,
|
||||||
|
"chat_template_kwargs": {"enable_thinking": False}})
|
||||||
|
ratio = len(sample) / cal["usage"]["prompt_tokens"]
|
||||||
|
print(f"[cal] corpus ratio {ratio:.2f} chars/token")
|
||||||
|
chars = int(0.97 * maxlen * ratio)
|
||||||
# Each marker's FIRST hyphen-separated word is unique, because show()
|
# Each marker's FIRST hyphen-separated word is unique, because show()
|
||||||
# scores the hit on that word — a shared prefix would score every slot HIT
|
# scores the hit on that word — a shared prefix would score every slot HIT
|
||||||
# on any answer that says the word.
|
# on any answer that says the word.
|
||||||
@@ -176,7 +187,7 @@ def main():
|
|||||||
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
||||||
print(f"[B] total {time.time() - t0:.1f}s")
|
print(f"[B] total {time.time() - t0:.1f}s")
|
||||||
# D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
|
# D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
|
||||||
dchars = int(0.25 * maxlen * 3.28)
|
dchars = int(0.25 * maxlen * ratio)
|
||||||
dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
|
dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
|
||||||
for i in range(seqs)]
|
for i in range(seqs)]
|
||||||
print(f"[D] {seqs} concurrent streams x 512 tokens, "
|
print(f"[D] {seqs} concurrent streams x 512 tokens, "
|
||||||
|
|||||||
Reference in new issue
Block a user