77 lines
4.4 KiB
Markdown
77 lines
4.4 KiB
Markdown
---
|
|
name: dankvoice-behavior-scenario-authoring
|
|
description: "Author or extend a dankvoice behavior.bats scenario: driver env knobs (DV_REWRITE/DV_FAIL/DV_CUT_MS/DV_TAG_LOG), the SPOKEN heard-words census, private-TMPDIR leak assertions, and the arm-the-precondition-before-absence trap."
|
|
---
|
|
|
|
# Writing a dankvoice behavior test
|
|
|
|
`tests/behavior.bats` drives the **built bundle** through a generated harness. Each `@test`
|
|
is: set env → run a named *scenario* → assert on printed census lines. Never assert on
|
|
internals; the observable vocabulary is `CALLS`, `PLAYS`, `PLAYORDER`, `SPOKEN`, `NOTIFY`,
|
|
`CALL <url> input=… auth=… at=…`, `PLAYER`, `KILLED`.
|
|
|
|
The driver lives in a heredoc inside `setup()`; find and read it before adding anything:
|
|
|
|
```bash
|
|
sed -n '/cat > "$DV_DRIVER" </,/^TS$/p' tests/behavior.bats
|
|
```
|
|
|
|
A scenario is a `case "<name>":` in that switch that calls the extension's own handlers
|
|
(`say.handler("on", ctx)`, `messageEnd?.(said("…"), ctx)`, `agentEnd?.(answer("…"), ctx)`).
|
|
Prefer delivering narration through the harness (`agent_end`/`message_end`) over a typed
|
|
`/say`: the humanizer, mood, heckle, framing and the frequency hold govern **unsolicited**
|
|
narration only, so a typed line cannot exercise them.
|
|
|
|
## Fixture knobs (set per test, all read by the driver)
|
|
|
|
| Knob | Effect |
|
|
|---|---|
|
|
| `DV_REWRITE=off\|echo\|prose\|reasoning\|stall` | what the chat (humanizer) route answers; unset ⇒ it 404s and the rewrite falls back to mechanical text |
|
|
| `DV_STALL_MS` | delay the chat answer (timeout behaviour) |
|
|
| `DV_FAIL=<N>` | the first N `/audio/speech` requests answer HTTP 500 JSON — an endpoint outage a test can watch the queue weather |
|
|
| `DV_HOLD=<sec>` | how long the player stub holds each clip; sets the playback windows |
|
|
| `DV_CUT_MS=<ms>` | in `stagecut`, when another session cuts the line |
|
|
| `DV_TAG_LOG`, `DV_TIMING_LOG` | per-clip tag census (`PLAYORDER`, `SPOKEN`) and player START/END ms |
|
|
| `DV_CT` | content type of the answer (non-WAV routing) |
|
|
| `DV_KEYS` | key replayer for the settings modal |
|
|
| `TMPDIR="$WORK/scratch-tmp"` | scope synthesis temp dirs so a leak assertion is yours alone |
|
|
|
|
Always pass `DANKVOICE_STAGE_DIR="$WORK/stage"` when the test touches the stage, and use the
|
|
`say_run <scenario>` helper (it wraps `run env …` so `$output` is captured); a bare
|
|
`scenario_name` invocation prints without capturing.
|
|
|
|
## Two traps that have each hidden a real bug
|
|
|
|
**Assert absence only with the producer armed.** `DV_REWRITE` unset ⇒ the chat route 404s,
|
|
so `[[ "$output" != *"/chat/completions"* ]]` and "payload equals the source text" pass even
|
|
when rewriting is happening. Nine tests pinned the opposite of the repo's law this way. Pin
|
|
with `DV_REWRITE=prose` (the fixture answers with its *own* words) and assert the payload you
|
|
expect — then survival is evidence, not absence.
|
|
|
|
**Count what was HEARD, not what was requested.** A segment whose clip was killed mid-play,
|
|
or aborted mid-synthesis, is legitimately synthesized again on resume. Laws about framing
|
|
("one intro per narration") are about hearing, so count `SPOKEN` lines: the driver maps each
|
|
played tag back to the `/audio/speech` request that produced that clip. `CALLS` counts
|
|
requests and will over-report across a line-cut.
|
|
|
|
Temp-dir leak assertions mirror this: `TMPDIR` private, then
|
|
`find "$TMPDIR" -maxdepth 1 -type d -name 'dankvoice-*' | wc -l`.
|
|
|
|
## Pinning timing
|
|
|
|
Derive windows from `DV_HOLD`: with 0.5 s clips, clips occupy ~[10,310], [310,610], [610,910];
|
|
choose `DV_CUT_MS` so the cut lands strictly inside the clip you mean to interrupt, leaving
|
|
margin for the 250 ms stage poll and synthesis time. Default scenario timings are env-overridable
|
|
(`DV_CUT_MS ?? 600`) so a new test never edits an existing one's schedule.
|
|
|
|
## Red-proof and gate (project invariants)
|
|
|
|
- Prove a new assertion can fail: in a **private worktree**, `git show HEAD:src/<file> > src/<file>`,
|
|
`bun run build`, run the filter, restore the file and rebuild. Never while another suite runs
|
|
(`setup()` rebuilds the bundle from `src/`). See `dankvoice-bats-serialization`,
|
|
`dankvoice-bundle-swap-red-proof`.
|
|
- `bun run build` before staging: `tests/packaging.bats` byte-compares committed `dist/` with a
|
|
fresh build, so `dist/index.js` must be in the same commit as the `src/` change.
|
|
- Gate with the whole suite (`bats tests/`), not the filter; ~5 min, so run it unbounded/async and
|
|
never write to `src/` while it runs.
|