chore(agent): commit the machine tier and the doctrine text the checkout was already running
Landing blocker for #61 and #63: the deployment checkout is 12 commits behind and `merge --ff-only` refuses while tracked paths are dirty. Discard is not available here (workspace policy: no `checkout`/`stash` on shared state), so the dirt is committed. - `agent/APPEND_SYSTEM.md`: the point-of-use / one-work-log / PR-link doctrine, already byte-identical to `origin/main` (`cmp` against the trunk blob: no difference). No new prose — the commit exists so the trunk merge can proceed. - `agent/config.yml` and `agent/managed-skills/**`: the machine-live tier the heartbeat owns (AGENTS.md § Live-sync tiers). `ALLOW_SECRETS=1` was used, deliberately and after auditing the staged diff by hand: the only credential-shaped strings in it are two copies of `AKIA0000000000000EXAMPLE` — the inert all-zero AWS example id — and they appear inside the two skill pages that teach that exact placeholder shape. This checkout is still running the pre-#61 hook; the merged gate (value-scoped placeholder filter, `71601f7`) exempts this value, so the refusal dies with this landing. No `--no-verify` was used: the gate ran and its verdict is recorded here. Refs #61
This commit is contained in:
1 parent
004962ffce
commit
8af40c402b
79 files changed
+2771
-332
No files matched your search
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: rng-pool-sweep-blackbox-test
|
||||
description: "Prove membership/absence in a randomly sampled pool (epithets-style) from black-box tests: prime-cycled Math.random sweeps every index; pool-first probe for setting precedence."
|
||||
---
|
||||
|
||||
# Deterministic pool sweeps for random-sampling black-box tests
|
||||
|
||||
When a built bundle samples uniformly at random from a curated pool (epithet-style) and tests must prove a setting admits/excludes whole classes of pool entries, do NOT write probabilistic sampling tests. Two proven designs:
|
||||
|
||||
## 1. Full-pool sweep (class membership, distinct counts)
|
||||
|
||||
In the test driver (the process that imports the built bundle), before firing utterances:
|
||||
|
||||
```ts
|
||||
let seed = 0;
|
||||
Math.random = () => (seed++ % PRIME) / PRIME; // PRIME > pool size, e.g. 1009
|
||||
```
|
||||
|
||||
The extension calls `Math.random()` at speak time, so the patch applies regardless of import order. Picks are `floor(random * poolSize)`; with N=PRIME samples the step is `< 1`, so **every index of every pool ≤ PRIME is visited exactly once**. Observable per-utterance text (e.g. the fixture server logs the synthesized `input=`) then reveals the entire sampling pool:
|
||||
|
||||
- distinct picks == pool size exactly (pin the total only if it is itself law);
|
||||
- a sentinel name from class X appears / must never appear per mode — deterministic, zero flake;
|
||||
- absence proofs are exhaustive, not lucky.
|
||||
|
||||
Drive one narration per iteration and gate the loop on the fixture's call counter advancing (`while (speechCount() < before + expected) await nap(2)`), otherwise typed/coalescing paths collapse iterations and counts drift.
|
||||
|
||||
**Wall-clock budget:** one sweep ≈ 20–40 s per pool; bats serializes and jobs cap at 600 s — run at most two sweeps per backgrounded bats `-f` selection, or launch full suites with `timeout: 0`.
|
||||
|
||||
## 2. Pool-first probe (setting precedence, milliseconds)
|
||||
|
||||
With the same patched RNG, the FIRST pick is `pool[0]`. Run each config mode with exactly one utterance and compare the first spoken name:
|
||||
|
||||
- mode-A setting vs env-mode-A → equal (settings key honored);
|
||||
- env-X overriding file-Y → equals env-X's pick (precedence law);
|
||||
- pick strings are pool literals that live in exactly one class (e.g. `comrade operator`).
|
||||
|
||||
Avoid: regex scans over spoken text (possessives/compounds false-fire), probabilistic "absent in 300 samples" tests (flake floor unless the block is contiguous), and reading source internals (white-box).
|
||||
|
||||
Red-first check: the sweep test for a not-yet-filtered pool must fail by SPEAKING the forbidden sentinel — if a "new mode" test passes pre-implementation, the probe accidentally tautologized (e.g. old pool[0] == new pool[0]); pick probe literals whose values differ between old and new behavior. Beware bats `setup()` rebuilding the bundle per test file: a long first sweep lets mid-session src edits leak into later tests' "red" proof — prove red from a pristine tree before implementing.
|
||||
Reference in new issue
Block a user