live: heartbeat sync 2026-10-30T21:32:40-07:00 (Refs #11)
This commit is contained in:
1 parent
d03fe57bbd
commit
d627368f71
13 files changed
+280
-26
No files matched your search
@@ -0,0 +1,75 @@
|
||||
---
|
||||
name: magic-number-provenance-audit
|
||||
description: "Audit every numeric threshold in a change before shipping it: classify provenance (inherited port / declared authority / invented), delete invented arithmetic, and replace budgets with derivations quoted from the repo's own config. Use when adding or reviewing a cap, window, timeout, ratio, or size ladder, or when asked \"what's with the magic numbers?\"."
|
||||
---
|
||||
|
||||
# Magic-number provenance audit
|
||||
|
||||
A threshold with no source is a bug with a comment on it. The deliverable is either a
|
||||
derivation from an authority the repo already trusts, or deletion of the arithmetic.
|
||||
|
||||
## 1. Census, do not defend
|
||||
|
||||
Grep the changed paths for every numeric literal (exclude the module being rewritten so
|
||||
its own digits don't hide the pattern):
|
||||
|
||||
```
|
||||
grep -rn "1536\|4000\|700\|6000\|n_ctx\|contextWindow\|maxTokens\|tokens" src/*.ts
|
||||
```
|
||||
|
||||
Table per number: value | file:line | **provenance** | verdict. Provenance is exactly one of:
|
||||
|
||||
- **inherited port** — copied from upstream, pinned by a contract test. Keep, but say so in
|
||||
the doc comment.
|
||||
- **declared authority** — a field the runtime/fleet config already publishes.
|
||||
- **derived** — computed from beats it guards (e.g. lease TTL = 3 heartbeats).
|
||||
- **invented** — yours. This is the finding. Say "mine, invented", not "chosen".
|
||||
|
||||
Also flag the *application* of a number, not just its value: a slice like
|
||||
`text.slice(0, MAX_INPUT)` is a **head-only silent drop** (loses the tail of a dump);
|
||||
a floor is a different animal from a cap even at the same digit.
|
||||
|
||||
## 2. Find the authority before writing a digit
|
||||
|
||||
Rank of acceptable sources, best first:
|
||||
|
||||
1. **The runtime's own published field.** `~/.omp/agent/models.yml` model rows carry
|
||||
`contextWindow` (PER-SLOT for llama.cpp: it divides `--ctx-size` across slots) and
|
||||
`maxTokens`. The fleet's own clamp law is quoted in the comments there — reuse its shape:
|
||||
`chars = (contextWindow - maxTokens - 4096) * 4 - lawChars` (`4` = the file's `chars/4`
|
||||
token estimator, `4096` = its clamp margin). Then every figure in the derivation is a
|
||||
quotation, not a guess.
|
||||
2. **A shipping style/behaviour law in the repo.** e.g. "two or three sentences, one breath
|
||||
each" → the ceiling belongs in the prompt as *words the model reads* ("at most three
|
||||
sentences; fewer if little happened"), not as `sqrt(chars / 1200)` in code.
|
||||
3. **A physical constant of the medium** (speech rate, packet size), measured against the
|
||||
actual device. Last resort: it is still a number you must defend.
|
||||
|
||||
Traps when parsing that config: commented decoy lines (`# contextWindow: 200,000` with a
|
||||
comma) must be skipped, so strip only lines whose trimmed form starts with `#`; the row
|
||||
scanner that returns on the `id:` line never sees the fields below it — keep scanning to the
|
||||
next `- id:`/provider key.
|
||||
|
||||
## 3. Encode the fix so the digit cannot come back
|
||||
|
||||
- Make the budget a **required parameter** of the pure module (no default: a module with no
|
||||
default cannot guess); the caller that resolved the endpoint does the derivation.
|
||||
- A derived budget that comes out non-positive is a **refusal** (`null`, zero requests),
|
||||
never a truncation — the caller's cursor/queue must hold and retry.
|
||||
- When an old cap must survive as a fallback for endpoints that declare nothing, rename it
|
||||
to what it is (`INHERITED_FLOOR`) and comment: floor, used only when nothing is declared.
|
||||
|
||||
## 4. Tests that pin the law, not the digit
|
||||
|
||||
- **Size-invariance:** assert the instruction/behaviour is *identical* for a tiny input and a
|
||||
huge one, proving volume no longer drives the removed ladder.
|
||||
- **Honours caller budget:** same span folded under an injected small window.
|
||||
- **Refusal, not truncation:** zero-window → null, zero model calls.
|
||||
- **DELETE** tests that pinned the deleted arithmetic; never re-pin them (house rule).
|
||||
- Pinning shipped prompt *words* is legitimate where the repo already does (prompt-contract
|
||||
test precedent); pinning an env/config value is a config assertion and belongs nowhere.
|
||||
|
||||
## 5. Report
|
||||
|
||||
Table first, verdict per number, and the net list of digits that remain after the change —
|
||||
each with its quoted source. If any digit survives with no source, that is the finding.
|
||||
Reference in new issue
Block a user