Compare commits

...
Author SHA1 Message Date
Frank b651d61056 docs(niri): portable desk runs landscape, HDMI left of the laptop
record the normalized layout that landed on main as 9b88208 and the law
that surprised it: positions are absolute, so rotating a panel widens
its logical box and every x to the right of it moves — recompute the
row, never just the rotated block.
2026-10-30 15:56:39 -07:00
Frank e197c80dcb Merge remote-tracking branch 'origin/main' into docs/niri-portable-desk 2026-10-30 15:56:39 -07:00
Frank 9b8820827c niri: desk-2 panels landscape, HDMI to the left of the laptop
Frank asked for normal rotation on both PM1601 panels and the HDMI-A-1
panel to the left of eDP-1. rotation drops 90 -> normal, so each panel
goes 1200x1920 -> 1920x1200 and the absolute x of everything to its
right moves with it: HDMI-A-1 now x=0, eDP-1 x=1920 (y=60, centred
against the taller panels), DP-2 x=3840. modes unchanged.
2026-10-30 15:56:35 -07:00
Frank b8584cd6a6 docs(niri): own the portable desk and the no-match defect
displays.md still described desk-2-portable as an EDID-pending skeleton
and left the PM1601 identity unnamed. record the second desk's rig,
its position math, and the law that a rig with no eligible profile runs
the previous desk's values rather than "unchanged" — the failure that
left the portable panels on 3440x1440@180 fallbacks for weeks.
2026-10-30 15:54:43 -07:00
Frank 5bfdc98e61 niri: make desk-2-portable a real profile for the second desk
The portable rig (laptop + two 16" PM1601 panels in portrait) had no
profile of its own, so display.kdl stayed on desk-1-home: the Sceptre
identity block matched nothing, its connector-keyed fallbacks carried
3440x1440@180.000 that no panel here reports, and niri silently drove
every panel at its preferred mode with positions written for the home
desk. key the two panels by captured EDID identity, pin the modes they
actually report, and centre the laptop against the 1920-tall panel band.

desk.sh match now selects this profile, so blankpad-desk-autoswitch no
longer dies at its initial match with "no desk profile matches".
2026-10-30 15:54:34 -07:00
Frank 9f142b7eaf live: heartbeat sync 2026-10-30T15:20:04-07:00 (Refs #11) 2026-10-30 15:20:04 -07:00
Frank ad5903c294 live: heartbeat sync 2026-10-30T14:17:40-07:00 (Refs #11) 2026-10-30 14:17:40 -07:00
Frank ef7478f9c7 live: heartbeat sync 2026-10-30T13:16:49-07:00 (Refs #11) 2026-10-30 13:16:49 -07:00
Frank 05f4fa412e live: heartbeat sync 2026-10-30T12:13:49-07:00 (Refs #11) 2026-10-30 12:13:49 -07:00
Frank 7508541310 live: heartbeat sync 2026-10-30T11:11:49-07:00 (Refs #11) 2026-10-30 11:11:49 -07:00
Frank d5b6b0e930 live: heartbeat sync 2026-10-30T10:10:49-07:00 (Refs #11) 2026-10-30 10:10:49 -07:00
Frank 28fd76fa21 live: heartbeat sync 2026-10-30T09:07:40-07:00 (Refs #11) 2026-10-30 09:07:40 -07:00
Frank b6e3bf2769 live: heartbeat sync 2026-10-30T08:05:49-07:00 (Refs #11) 2026-10-30 08:05:49 -07:00
Frank 05f818ac03 live: heartbeat sync 2026-10-30T07:04:49-07:00 (Refs #11) 2026-10-30 07:04:49 -07:00
Frank 223768cc91 live: heartbeat sync 2026-10-30T06:02:49-07:00 (Refs #11) 2026-10-30 06:02:49 -07:00
Frank 57fc14057e live: heartbeat sync 2026-10-30T04:59:49-07:00 (Refs #11) 2026-10-30 04:59:49 -07:00
Frank b9dcb88f9b live: heartbeat sync 2026-10-30T03:56:49-07:00 (Refs #11) 2026-10-30 03:56:49 -07:00
Frank fb9b8b7d22 live: heartbeat sync 2026-10-30T02:53:49-07:00 (Refs #11) 2026-10-30 02:53:49 -07:00
Frank 53dd12e73d live: heartbeat sync 2026-10-30T01:50:49-07:00 (Refs #11) 2026-10-30 01:50:49 -07:00
Frank 52e8cca8f3 live: heartbeat sync 2026-10-30T00:48:49-07:00 (Refs #11) 2026-10-30 00:48:49 -07:00
Frank 7b7566928f live: heartbeat sync 2026-10-29T23:45:49-07:00 (Refs #11) 2026-10-29 23:45:49 -07:00
Frank ee7489b546 live: heartbeat sync 2026-10-29T22:44:49-07:00 (Refs #11) 2026-10-29 22:44:49 -07:00
Frank 4b032e011b live: heartbeat sync 2026-10-29T21:41:49-07:00 (Refs #11) 2026-10-29 21:41:49 -07:00
Frank dc56eb5b02 live: heartbeat sync 2026-10-29T20:38:49-07:00 (Refs #11) 2026-10-29 20:38:49 -07:00
Frank a5e00a09eb live: heartbeat sync 2026-10-29T19:37:40-07:00 (Refs #11) 2026-10-29 19:37:40 -07:00
Frank 04ada2c913 live: heartbeat sync 2026-10-29T18:36:49-07:00 (Refs #11) 2026-10-29 18:36:49 -07:00
Frank 4df5a28ba6 live: heartbeat sync 2026-10-29T17:35:49-07:00 (Refs #11) 2026-10-29 17:35:49 -07:00
Frank 03bf48956f live: heartbeat sync 2026-10-29T16:32:39-07:00 (Refs #11) 2026-10-29 16:32:39 -07:00
Frank 4d76c6038e live: heartbeat sync 2026-10-29T15:29:37-07:00 (Refs #11) 2026-10-29 15:29:37 -07:00
dank 3eb1ada55b Merge pull request 'feat: add the DeepSeek platform provider and deepseek-flash' (#84) from feat/83-deepseek-provider into main 2026-10-29 14:52:48 -07:00
Frank 206c9acb46 feat: add the DeepSeek platform provider and deepseek-flash
Refs #83

The first vendor-cloud chat row in agent/models.yml. The provider label is the
bare word `deepseek` because pi-catalog/src/hosts.ts keys the direct-DeepSeek
behaviour set (max_tokens field, thinking extraBody, thinking-disable mode) off
that exact name and off api.deepseek.com; the house `<api> (<host>)` pattern
would not match and the row would lose the quirks the endpoint answers to.

The key is command-backed (`!cat ~/.config/secrets/deepseek_api_key`, 600), so
no sk-… value enters the tracked tree, and the row declares the model card's own
pair: contextWindow 1048576 / maxTokens 262144, input [text, image], with the
October 2026 peak rate card in cost. Docs move to the new
omp-docs/core/deepseek-provider.md, indexed and logged in the same commit.
2026-10-29 14:47:34 -07:00
Frank 53025a6be5 live: heartbeat sync 2026-10-29T14:27:39-07:00 (Refs #11) 2026-10-29 14:27:39 -07:00
Frank dc88d6f961 live: heartbeat sync 2026-10-29T13:25:49-07:00 (Refs #11) 2026-10-29 13:25:49 -07:00
Frank f2d8aa4f9e live: heartbeat sync 2026-10-29T12:22:39-07:00 (Refs #11) 2026-10-29 12:22:39 -07:00
Frank 7506970f9d live: heartbeat sync 2026-10-29T11:19:49-07:00 (Refs #11) 2026-10-29 11:19:49 -07:00
Frank c736456413 live: heartbeat sync 2026-10-29T10:17:49-07:00 (Refs #11) 2026-10-29 10:17:49 -07:00
Frank 441519c1f0 live: heartbeat sync 2026-10-29T09:15:49-07:00 (Refs #11) 2026-10-29 09:15:49 -07:00
Frank 90cd6bb766 live: heartbeat sync 2026-10-29T08:12:39-07:00 (Refs #11) 2026-10-29 08:12:39 -07:00
Frank 757fa0106b live: heartbeat sync 2026-10-29T07:10:49-07:00 (Refs #11) 2026-10-29 07:10:49 -07:00
Frank a8e74e7266 live: heartbeat sync 2026-10-29T06:09:49-07:00 (Refs #11) 2026-10-29 06:09:49 -07:00
Frank f859230ff6 live: heartbeat sync 2026-10-29T05:06:49-07:00 (Refs #11) 2026-10-29 05:06:49 -07:00
Frank 6569290398 live: heartbeat sync 2026-10-29T04:04:49-07:00 (Refs #11) 2026-10-29 04:04:49 -07:00
Frank d83531e3fe live: heartbeat sync 2026-10-29T03:03:49-07:00 (Refs #11) 2026-10-29 03:03:49 -07:00
Frank 41f577c883 live: heartbeat sync 2026-10-29T02:02:39-07:00 (Refs #11) 2026-10-29 02:02:39 -07:00
Frank 32bea490c4 live: heartbeat sync 2026-10-29T01:00:49-07:00 (Refs #11) 2026-10-29 01:00:49 -07:00
Frank b698b0e35e live: heartbeat sync 2026-10-28T23:58:49-07:00 (Refs #11) 2026-10-28 23:58:49 -07:00
Frank 0d41c3d61d live: heartbeat sync 2026-10-28T22:56:49-07:00 (Refs #11) 2026-10-28 22:56:49 -07:00
Frank ea2cfad847 live: heartbeat sync 2026-10-28T21:53:49-07:00 (Refs #11) 2026-10-28 21:53:49 -07:00
Frank 6f0aa7d72e live: heartbeat sync 2026-10-28T20:51:49-07:00 (Refs #11) 2026-10-28 20:51:49 -07:00
Frank 7d394e74b4 live: heartbeat sync 2026-10-28T19:48:49-07:00 (Refs #11) 2026-10-28 19:48:49 -07:00
Frank 10cd18ec57 live: heartbeat sync 2026-10-28T18:46:49-07:00 (Refs #11) 2026-10-28 18:46:49 -07:00
Frank e4721a6713 live: heartbeat sync 2026-10-28T17:45:49-07:00 (Refs #11) 2026-10-28 17:45:49 -07:00
Frank 4648ba7f3f live: heartbeat sync 2026-10-28T16:42:49-07:00 (Refs #11) 2026-10-28 16:42:49 -07:00
Frank 2d71c603cb live: heartbeat sync 2026-10-28T15:42:39-07:00 (Refs #11) 2026-10-28 15:42:39 -07:00
Frank 388058d134 live: heartbeat sync 2026-10-28T14:40:49-07:00 (Refs #11) 2026-10-28 14:40:49 -07:00
Frank 979382e5cc live: heartbeat sync 2026-10-28T13:39:49-07:00 (Refs #11) 2026-10-28 13:39:49 -07:00
Frank 47d4e0e8f9 live: heartbeat sync 2026-10-28T12:38:49-07:00 (Refs #11) 2026-10-28 12:38:49 -07:00
Frank a071d71e99 live: heartbeat sync 2026-10-28T11:37:39-07:00 (Refs #11) 2026-10-28 11:37:39 -07:00
Frank 38236f35f7 live: heartbeat sync 2026-10-28T10:34:49-07:00 (Refs #11) 2026-10-28 10:34:49 -07:00
Frank d73deb61e2 live: heartbeat sync 2026-10-28T09:33:49-07:00 (Refs #11) 2026-10-28 09:33:49 -07:00
Frank 68d6889100 live: heartbeat sync 2026-10-28T08:31:49-07:00 (Refs #11) 2026-10-28 08:31:49 -07:00
Frank 5a3f1a77ac live: heartbeat sync 2026-10-28T07:29:49-07:00 (Refs #11) 2026-10-28 07:29:49 -07:00
Frank 0b27a68c33 live: heartbeat sync 2026-10-28T06:26:49-07:00 (Refs #11) 2026-10-28 06:26:49 -07:00
Frank 0b72e9a427 live: heartbeat sync 2026-10-28T05:23:49-07:00 (Refs #11) 2026-10-28 05:23:49 -07:00
Frank b04a634ec2 live: heartbeat sync 2026-10-28T04:21:49-07:00 (Refs #11) 2026-10-28 04:21:49 -07:00
Frank 240324a18c live: heartbeat sync 2026-10-28T03:18:49-07:00 (Refs #11) 2026-10-28 03:18:49 -07:00
Frank 9d172eff9c live: heartbeat sync 2026-10-28T02:15:49-07:00 (Refs #11) 2026-10-28 02:15:49 -07:00
Frank ddebfb63bb live: heartbeat sync 2026-10-28T01:13:49-07:00 (Refs #11) 2026-10-28 01:13:49 -07:00
Frank b7fd64b241 live: heartbeat sync 2026-10-28T00:11:49-07:00 (Refs #11) 2026-10-28 00:11:49 -07:00
Frank 414b4cd578 live: heartbeat sync 2026-10-27T23:08:49-07:00 (Refs #11) 2026-10-27 23:08:49 -07:00
Frank b289aa59d4 live: heartbeat sync 2026-10-27T22:05:49-07:00 (Refs #11) 2026-10-27 22:05:49 -07:00
Frank 175bbdf451 live: heartbeat sync 2026-10-27T21:02:39-07:00 (Refs #11) 2026-10-27 21:02:39 -07:00
Frank 531ead5775 live: heartbeat sync 2026-10-27T20:01:49-07:00 (Refs #11) 2026-10-27 20:01:49 -07:00
dank abee84ddad Merge pull request 'docs(reports): point the reports index and work log at the vuln issues' (#78) from docs/vuln-issue-refs into main 2026-10-27 19:57:11 -07:00
Frank 9df5c50fda live: heartbeat sync 2026-10-27T18:58:49-07:00 (Refs #11) 2026-10-27 18:58:49 -07:00
dank 702847cefd Merge pull request 'docs(core): snapcompact accuracy audit — measured shape table + pricing law' (#82) from docs/81-snapcompact-accuracy into main 2026-10-27 18:39:36 -07:00
Frank ed3a43bff1 docs(core): snapcompact audit — measured shape/recall table, frank-archon image-count budget, 85% trigger law
Live probe against frank-archon (212,916-char archive, 8 planted facts):
auto shape 8on22-bw reads back 8/8 recall at 47% of raw-text billed
tokens; both denser grids drop characters. Lands the docs for the paired
live-config changes (thresholdPercent 95->85, imageCountBudget 16) and
records the catalog-only frame-pricing law (radiance ~2,400 tok/frame vs
the harness's blind 5,024 floor).

Refs #81
2026-10-27 18:37:31 -07:00
Frank cd3fe7fd03 live: heartbeat sync 2026-10-27T17:57:53-07:00 (Refs #11) 2026-10-27 17:57:53 -07:00
Frank 218357cb66 live: heartbeat sync 2026-10-27T16:55:49-07:00 (Refs #11) 2026-10-27 16:55:49 -07:00
Frank 31d6924a27 live: heartbeat sync 2026-10-27T15:54:49-07:00 (Refs #11) 2026-10-27 15:54:49 -07:00
Frank feda9fe668 live: heartbeat sync 2026-10-27T14:52:38-07:00 (Refs #11) 2026-10-27 14:52:38 -07:00
Frank eae9dea2b0 live: heartbeat sync 2026-10-25T19:49:50-07:00 (Refs #11) 2026-10-25 19:49:50 -07:00
Frank bc597edec1 live: heartbeat sync 2026-10-23T16:20:45-07:00 (Refs #11) 2026-10-23 16:20:45 -07:00
Frank 5f560f33c1 live: heartbeat sync 2026-10-23T15:17:45-07:00 (Refs #11) 2026-10-23 15:17:45 -07:00
Frank 461f0b8a6e live: heartbeat sync 2026-10-23T14:16:45-07:00 (Refs #11) 2026-10-23 14:16:45 -07:00
Frank 02b65989ca live: heartbeat sync 2026-10-23T13:14:45-07:00 (Refs #11) 2026-10-23 13:14:45 -07:00
Frank 42eaf9e1a7 live: heartbeat sync 2026-10-22T22:47:45-07:00 (Refs #11) 2026-10-22 22:47:45 -07:00
Frank f51f4dd1aa live: heartbeat sync 2026-10-22T20:42:45-07:00 (Refs #11) 2026-10-22 20:42:45 -07:00
Frank 0e2cf13e3b live: heartbeat sync 2026-10-22T11:24:45-07:00 (Refs #11) 2026-10-22 11:24:45 -07:00
Frank 1d1059f7b3 live: heartbeat sync 2026-10-22T11:02:06-07:00 (Refs #11) 2026-10-22 11:02:06 -07:00
Frank b7089ef570 live: heartbeat sync 2026-10-22T10:01:40-07:00 (Refs #11) 2026-10-22 10:01:40 -07:00
Frank dc4de763f3 live: heartbeat sync 2026-10-22T09:01:07-07:00 (Refs #11) 2026-10-22 09:01:07 -07:00
Frank 9d284106ac live: heartbeat sync 2026-10-22T07:59:45-07:00 (Refs #11) 2026-10-22 07:59:45 -07:00
Frank ac2c5c519a live: heartbeat sync 2026-10-22T06:58:47-07:00 (Refs #11) 2026-10-22 06:58:47 -07:00
Frank f1402ca89c live: heartbeat sync 2026-10-22T05:56:18-07:00 (Refs #11) 2026-10-22 05:56:18 -07:00
Frank 4d515f7e81 live: heartbeat sync 2026-10-22T04:54:18-07:00 (Refs #11) 2026-10-22 04:54:18 -07:00
Frank f7281eee42 live: heartbeat sync 2026-10-22T03:51:42-07:00 (Refs #11) 2026-10-22 03:51:42 -07:00
Frank ecb81dd8b8 live: heartbeat sync 2026-10-22T02:50:34-07:00 (Refs #11) 2026-10-22 02:50:34 -07:00
Frank 1b1a296ec5 live: heartbeat sync 2026-10-22T01:47:38-07:00 (Refs #11) 2026-10-22 01:47:38 -07:00
Frank 9a3d364832 live: heartbeat sync 2026-10-22T00:46:57-07:00 (Refs #11) 2026-10-22 00:46:57 -07:00
Frank e529dc5386 live: heartbeat sync 2026-10-21T23:44:55-07:00 (Refs #11) 2026-10-21 23:44:55 -07:00
Frank 9faf7e0652 live: heartbeat sync 2026-10-21T22:44:37-07:00 (Refs #11) 2026-10-21 22:44:37 -07:00
Frank 017404aa51 live: heartbeat sync 2026-10-21T17:49:41-07:00 (Refs #11) 2026-10-21 17:49:41 -07:00
dank c0d840a8bb Merge pull request 'feat(models): WAN model access through the OpenWebUI door' (#80) from feat/79-wan-model-access into main 2026-10-21 16:59:51 -07:00
Frank f3360a28f1 feat(models): reach the fleet from off-network through the OpenWebUI door
Every provider in agent/models.yml is a LAN address, so a workstation that
leaves the house has no model at all. Add `openwebui (franksdank.cloud)`:
the already-public OpenWebUI vhost as an OpenAI-compatible provider that
proxies the same engines, with frank-dragoon and frank-zealot rows and the
WAN row as its own compaction target. LAN rows, modelRoles and the speech
runner stay untouched — the door is purely additive.

The key is command-backed (`apiKey: "!cat ~/.config/secrets/openwebui_api_key"`,
mode 600) instead of inline: a literal sk-... in a tracked file is a key in
every agent context on every box that pulls this repo, and the pre-commit
shape gate would block it anyway. Rotate = rewrite the file + restart omp,
because a `!cmd` value is cached for the process lifetime.

Compat blocks copy their LAN twins because the proxy was probed, not assumed
(2026-10-21): streaming SSE carries reasoning_content and tool_calls deltas
plus final usage, image_url content parts reach both vision towers, and
chat_template_kwargs is forwarded — proven on frank-zealot, where
enable_thinking false yields zero reasoning and true a real block.
frank-archon gets no WAN row: OpenWebUI answers 400 "Model not found"
because its picker never published it, so that path is a fleet change.

Verified end-to-end from a scratch HOME whose LAN baseUrls point at a dead
port, so any answer can only have come through the door (text and a real
tool round-trip); `omp models` still lists all four chat providers and the
tts runner, so the file validates. bats 137/137, kb-lint clean.
Docs: omp-docs/core/wan-model-access.md.
2026-10-21 16:48:29 -07:00
Frank dcdf84719c docs(reports): point the reports index and work log at the vuln issues
The audit's vulnerability clusters are now filed on Gitea as
dank/command-center#67..#77, one issue per host-and-package cluster. Link
them from omp-docs/reports/index.md and record the PR/issue references on
the matching log.md entry, so a reader of the audit can find the open work
without leaving the bundle.
2026-10-21 15:05:41 -07:00
dank 8c3dc6bfc9 Merge pull request 'docs(reports): land the fleet security audit with its per-host vuln passes' (#66) from docs/fleet-security-audit-2026-10-21 into main 2026-10-21 14:56:06 -07:00
Frank 27da5cd9ec docs(reports): land the fleet security audit with its per-host vuln passes
Fold the Dankclaw and Dankserver vulnerability passes into
fleet-security-audit-2026-10-21.html: the stub "Advisory cross-reference (in
progress)" section becomes the full matrix (host advisories, GHSA coverage,
image-vulnerability matrix, no-advisory census, observed-without-credentials
rows for the two livingroom hosts) and the header links the two working files,
which move into omp-docs/reports/ with an index of their own.

Correct three figures the audit had wrong, all found by re-probing: the
mutable-tag census (17 -> 19 Dankclaw manifests / 17 unique references, +4 on
Dankserver), the Dankserver OpenSSH advisory count (0 -> 5 unpatched CVEs; the
first OSV query omitted the version epoch, and OSV answers "0 vulns" rather
than "unknown" for a malformed purl), and the Plex version ("12.1.0" is
Jellyfin's - the live container reports 1.43.5.11029-6bf8af3f0-ls329). Also
repair a malformed <code> tag in F1's evidence cell.

kb-lint and kb-coverage both report 0 issues; index.md gains the Reports
section and log.md the matching entry.
2026-10-21 14:04:33 -07:00
Frank ec9bf77b10 live: heartbeat sync 2026-10-21T11:49:23-07:00 (Refs #11) 2026-10-21 11:49:23 -07:00
Frank 6a12ffaef4 live: heartbeat sync 2026-10-21T10:47:56-07:00 (Refs #11) 2026-10-21 10:47:56 -07:00
Frank d33ee26e67 live: heartbeat sync 2026-10-21T09:46:51-07:00 (Refs #11) 2026-10-21 09:46:51 -07:00
Frank b8f66e22b2 live: heartbeat sync 2026-10-21T08:45:51-07:00 (Refs #11) 2026-10-21 08:45:51 -07:00
Frank 07768ce8b5 live: heartbeat sync 2026-10-21T07:43:51-07:00 (Refs #11) 2026-10-21 07:43:51 -07:00
Frank d845160e62 live: heartbeat sync 2026-10-21T06:41:51-07:00 (Refs #11) 2026-10-21 06:41:51 -07:00
Frank 3ab48bf1f2 live: heartbeat sync 2026-10-21T05:39:51-07:00 (Refs #11) 2026-10-21 05:39:51 -07:00
Frank 51f9846ac6 live: heartbeat sync 2026-10-21T04:37:51-07:00 (Refs #11) 2026-10-21 04:37:51 -07:00
Frank 4085ee403f live: heartbeat sync 2026-10-21T03:35:51-07:00 (Refs #11) 2026-10-21 03:35:51 -07:00
dank 1dc01bd006 Merge pull request 'docs(#64): one owner per fact, truth re-read from the boxes, readable pages' (#65) from docs/64-bundle-audit-dedupe into main 2026-10-21 02:43:28 -07:00
Frank 336201e814 live: heartbeat sync 2026-10-21T02:32:51-07:00 (Refs #11) 2026-10-21 02:32:51 -07:00
116 changed files with 4232 additions and 459 deletions

No files matched your search

+1 -1
View File
@@ -85,7 +85,7 @@ compaction:
- handoff
- remote
idleThresholdTokens: 100000
thresholdPercent: 95
thresholdPercent: 85
idleEnabled: true
thresholdTokens: -1
dropUseless: false
@@ -0,0 +1,42 @@
---
name: agenttrash-purger-invocation
description: "Invoke the .AGENTTRASH purger (scripts/purge-agenttrash.sh) — plus its two refusal traps: ENOSPC masquerading as a stale lock, and an unreadable container-storage directory poisoning the whole root scan."
---
# Purging .AGENTTRASH — invocation, and the two traps that make it refuse
`scripts/purge-agenttrash.sh` (serverAppConfigs, merged in PR #79 `fad87aa`) is the tool form of the fleet's deletion policy: files land in `.AGENTTRASH`, and an approved purge runs this instead of being improvised per session.
## Invoke
```sh
sh scripts/purge-agenttrash.sh # plan only — prints, deletes nothing
sh scripts/purge-agenttrash.sh --execute # perform the plan
sh scripts/purge-agenttrash.sh --execute /some/root # scope to one root (default: $HOME)
```
- `--quiet` silences per-line output; errors and the summary still print. `--log PATH` overrides the default `~/.AGENTTRASH-cleanup.log` (one dated line per run — written even for a dry run).
- Every printable line is fixed at plan time in `$HOME/.agenttrash-purge.lock/plan`, so dry-run and `--execute` output cannot drift. Exit `0` ok · `1` any refusal or failure · `2` usage.
- Deleting a big staged item by hand (e.g. `find <dir> -depth -delete`) is the fallback **only** when the tool cannot run at all — see trap 1 — and the operator's purge approval covers exactly that.
## Trap 1: a full filesystem makes it lie about a lock (measured 2026-10-30)
With **0 bytes free**, three invocations refused with
```
purge-agenttrash: refused /home/<user>: exists and is not ours to reclaim
```
and no lock directory existed anywhere. The script writes its lock *and* its plan file before scanning; both hit `ENOSPC`, and the failure surfaced as the lock-reclaim message. There is nothing to reclaim — **free space first** (delete the staged item, move it to another volume, `journalctl --vacuum-size=`), then run plan + `--execute`. Never hunt a phantom lock because the message says so; check `df` first.
## Trap 2: an unreadable directory spoils the whole root
`collect_root` turns any nonzero `find` status into a root refusal, so the plan is built from a partial scan and the root reports `refused=1` even though `find` printed every `.AGENTTRASH` it found.
- On the dev box the cause is uutils find (`find --version` → `find 0.8.0`), which exits 1 on `~/.local/share/containers/storage/**`. Proof: `find /home/blank -xdev -name .AGENTTRASH >/dev/null 2>&1; echo $?` → `1`, while a narrower root → `0`.
- On **dankserver** GNU find does the same, because rootless podman storage holds foreign-UID directories (`overlay/*/diff/var/lib/mysql`, `valkey`, `calibre/plugins`, `apt/.../partial`): `find /home/dank -xdev -name .AGENTTRASH 2>&1 >/dev/null | head` shows the denials.
- Fix to land in the script (not to improvise around): prune `~/.local/share/containers` from the scan — a `.AGENTTRASH` never lives inside container storage — or accept partial output and reserve refusal for a hard find failure. `sh -n` / shellcheck is the gate; `bash -n` is not.
## Verified good path (dankserver, 2026-10-30)
After freeing space by hand: `/tmp`-scoped plan → `--execute` deleted the staged item and reported `removed=1 deleted=1`; a second `--execute /mnt/entropy` cleaned that volume's root (`refused=0`). The cleanup log carries one dated line per run; read it instead of re-deriving what happened.
@@ -1,61 +1,41 @@
---
name: audit-subagent-deliverables
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), adversarial-review-report triage, resource-safety pass (fd/memory), and cutover with running-process restarts."
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), merge shared-function collisions, fix scope creep, verify env var propagation."
---
# Audit subagent deliverables
Integrating code written by subagents (fan-out implementation). The code is usually correct; the failures are structural — collisions between agents, scope creep, and assumptions that never reach the runtime.
A subagent that wrote code is a source of plausible output, not a verified change. Integrate it the way you'd integrate an untrusted PR — with extra suspicion, because the author is an agent that shares your blind spots and its prose is more confident than its evidence.
## Workflow
Order of operations: read everything → gates → tests (as evidence, not proof) → resource-safety pass → fix in-place → cutover.
1. **Read every deliverable line by line** before integrating — not the summary.
2. **Run the mechanical gates immediately**, in parallel: `bash -n` on every script; `bats tests/ 2>&1 | grep "not ok"` for the failure list, never `tail` (tail shows a passing tail and hides the red in the middle).
3. **Merge shared-function collisions.** When two agents add different versions of the same function, pick the superset and re-point the other's callers at it. (Real case: one agent's `fleet_base()` carried `-b`, the other's `resolve_fleet_base()` carried `--base-url`; resolve to the one flag set the tests use.)
4. **Delete scope creep.** Code added for a goal nobody asked for — a new env var, a new CLI flag, a new output file — goes, with its docs, unless the operator asked.
5. **Verify env var propagation.** A var the orchestrator computes must be exported before the `bash -c` / subshell that reads it, or the child silently takes a default. Check the export site, not the doc.
## 1. Read every produced file, line by line
Full file, not a diff skim. You are the reviewer who must catch the line the author never thought about.
## Audit checks
## 2. Mechanical gates before logic review
```bash
bash -n <script> # syntax (scripts)
bunx tsc --noEmit # types (TS)
bun test # suite
```
Run the suite for information, not as the stopping point — it only proves the author's own scenarios.
| Check | How |
|---|---|
| Flag names real | `bash script.sh --help` |
| Paths exist | `test -e` each literal in the diff |
| Env reaches the runtime | grep the export/inline-assignment site, then the read |
| Tests assert behaviour | not `[[ -n "$X" ]]`, not `test -f <new file>` |
| Tests not weakened | `git diff HEAD~ -- tests/` — a deleted assertion is a red flag |
## 3. Hunt the three classic agent defects
- **Quoting bugs:** `set -e` + `"${@/$p/$r}"` = substitution regex containing user input (bash can't escape `/` in a `${var/re/pat}` pattern: `/` reads as the pattern/replacement separator). Fix with a separator absent from inputs (e.g. `%`) and `printf '%q'` for shell-interpolated values.
- **`cd` assumptions:** parallel `runShell` calls share one CWD (pool mode). Any command sequence that assumes CWD (e.g. `git -C .` after a sibling's `cd`) must use explicit absolute paths.
- **Unbounded resources:** every fd opened must be closed on ALL paths (success, throw, validation fail) — `using` for fs.sinks (closes on throw; sinks are `AsyncDisposable`, not `fs.FileHandle`, no `.close()`); bounded `Map` for caches (`MAX` + FIFO eviction); explicit caps (lines read, file sizes).
## Red flags
## 4. Check the test file is real
Tests that assert internal state (`expect(wcwd).toBe(...)`) over output contracts; tests of mock echo; tautologies. Delete them per house rules, keep behavior contracts.
- Test asserting a new file exists rather than behaviour.
- Doc claiming a flag the script never defines.
- Function added with no caller.
- Doc example that contradicts the code it documents.
- Same helper defined twice with different flag spelling.
- New env var read but never exported.
## 5. Cutover audit (repo-wide)
`grep -r` the symbol namespace the subagent touched — `wcwd`-style globals are the classic blast radius. Grep every `cd`/`process.chdir` site and verify each paired command sequence uses explicit paths. Stale comments describing the old design must go — comments that lie are time bombs for the next reader.
## Re-verify after the operator's own edits
## 6. Cutover restarts running processes
A code fix on disk is NOT live in any process that loaded the file at spawn (bun/node load-at-spawn). After shipping fixes: restart every long-running supervisor/daemon/UI-server that carries the affected code path; prefer takeover protocols (stale-pid detection) that make restarts cheap. Symptom of forgetting: "fixed" code producing old-code symptoms in production (observed: 812 poisoned ledger rows).
Run the gates again — their edit may undo an integration (a re-typed helper, a reverted flag).
## 7. Triaging an adversarial REVIEW report (not code)
When a subagent produced findings, treat each finding as a hypothesis with a base rate ~40% (measured over 3 rounds, 2026-10):
1. **Walk every finding end-to-end against the CURRENT tree** before accepting — cite file:line yourself. Reviewers frequently read a mid-edit snapshot; many findings are stale by construction.
2. **Classification that kills most findings:**
- *Stale read* — code changed during review; verify against live lines.
- *Inverted logic* — "failed rows are skipped" when the filter makes failed rows RETRY. Re-read the filter.
- *Unreachable* — trigger requires breaking a stated design law (single-worker, append-only). Not a defect.
- *By-design* — the reviewer labels a spec law a bug (serial execution, non-stream client). Check the spec, not the reviewer's confidence.
- *Constant-trust* — "XSS via manifest field" where the field is git-pinned config; "path traversal via REMOTE_TMP" where it's a hardcoded const. Ask: what untrusted input reaches this sink?
- *Self-refuting* — the reviewer's own probe output disproves its claim (raw==dedup probe in a "dedup" finding). Quote it back.
3. **Acceptance bar:** concrete trigger with real inputs + full path walked. Real accepts cluster in: resource caps (pids/timeout/awaited-cleanup), honest labels (default signatures), escaping discipline, self-healing probes.
4. **One-line verdicts + evidence per finding**, shipped in the commit message; calibration section ("genuinely solid") accepted as free verification of past fixes.
## Report
## Verification (the non-negotiable)
Build/run the real artifact with its real inputs. Observe behavior at the seams:
- concurrent calls → no CWD races;
- resource ceilings → caps enforced;
- cleanup → no fd/pid/dir leaks;
- security → injection probes inert (quotes, slashes, metachars);
- normal case → output identical to pre-audit.
## Evidence (2026-10-07 + 2026-10-10/11 sessions)
- Shell-engine subagent: 426 LOC, `bun test` 39/39 green. Line-audit found: 5 leaked fds per call (`fd` sink never closed), CWD-race in `git -C .`, `set -e` + `${@/\/g\//}` slash-escaping bug, unbounded 10k-entry cache, 45s timeout on git-status.
- Review-triage census over 3 blind adversarial rounds (same repo, ~41 findings): accepted 9, rejected ~32 — rejects dominated by stale reads (protocol rewritten mid-review), inverted filter logic, unreachable-under-law triggers, base64-alphabet "quote injection" (charset has no quotes), SIGPIPE non-leaks, "no memory cap" quoting the `--memory 4g` line above the claim. Real accepts: fork-bomb pids cap, stalled-ssh upload cap, stale-flag self-heal probe, honest census label, freshness gate on pid reuse.
- Poison-law incident: sandbox fix shipped while supervisor ran old build → 812 rows graded by the dead protocol; takeover made recovery cheap. The restart step (#6) exists because of this.
Per deliverable: correct / changed / removed, with file:line, plus the gate output that proves it (`bats` failure list before and after).
@@ -0,0 +1,39 @@
---
name: bats-line-scoped-assert-traps
description: "When writing or reviewing bats tests over multi-line command output, catch glued-glob adjacency that hops lines, env-exec of shell functions (127), and file-wise runs that hide full-suite reds."
---
Three traps that produced false-green bats runs; check each when writing or reviewing tests that assert over multi-line `$output`.
## 1. `[[ "$output" == *"a b"* ]]` can hop lines
`$output` is multiline; a glob `*url auth=Bearer key*` matches across newline boundaries, and inserting a new column into a per-line log format silently breaks *old* adjacency pins elsewhere in the file.
Pin per-line facts with a line-scoped grep instead:
```bash
grep -Eq '/audio/speech .*auth=Bearer dk-speech' <<<"$output"
```
`grep` is line-scoped by construction: the assertion can only match within one CALL/log line. When a fixture adds a column to its log lines, grep the test suite for glued globs over those lines first (`grep -n '\[\[ "\$output" == \*' tests/`).
## 2. `run env PATH=... my_shell_function args` is always 127
`env` execs an external binary; bats test-file functions are invisible to it, and `BW01` (127) does not by itself fail the test — a following `[ "$status" -eq 0 ]` does, or worse, nothing checks it.
Inherit and extend PATH around the function call instead:
```bash
export PATH="$WORK/bothbin:$PATH"
DV_VAR=x my_function scenario
```
If a restricted PATH is truly required, write an executable wrapper script into that bin (`exec bun "$DRIVER" "$@"` with an absolute interpreter path).
## 3. File-wise green ≠ suite green
A single file (`bats tests/behavior.bats`) can pass while `bats tests/` is red (order-dependent state, shared fixtures, or simply a stale dist pin file in another file). Always run the whole gate — `bats tests/` plus unit runner (`bun test`) — on the exact final tree before opening a PR, and grep the log, never tail it:
```bash
bats tests/ > /tmp/gate.log 2>&1; echo "EXIT $?"; grep "^not ok" /tmp/gate.log; grep -c "^ok" /tmp/gate.log
```
## Verify a red is red for the stated reason
When a freshly-added test fails immediately and inexplicably fast (seconds), suspect harness-level breakage (127, missing exec) rather than product behavior — inspect with a temporary `printf '%s\n' "$output" > /tmp/dbg.txt` dump inside the test, delete it after, and reproduce the actual line format before fixing assertions.
@@ -0,0 +1,48 @@
---
name: browser-screenshot-disk-and-cookies
description: "Save omp managed-browser screenshots to real files (tab.screenshot path is silently ignored), prove/clear cookie state (clearCookies is domain-scoped), and get embedded-image commits past the secret gate."
---
# omp managed browser from Eval: screenshots, cookies, page reads
Friction-earned 2026-10-21 (cookie-state proof on a local stack), same day (static HTML guide verification — four wasted API calls from signature guesses), re-earned same day on the franksign screenshot-walkthrough campaign (19 embedded PNGs, two cookie traps, one secret-gate false positive).
## Screenshots to real files
- `tab.screenshot({ path })` returns the image to the model but the `path` option is **silently ignored** on the managed-browser path — no file appears on disk.
- Working recipe: re-capture raw and decode base64 to disk:
```js
await tab.run(async ({ page }, p) => {
const r = await page.screenshot({ encoding: "base64", type: "png" });
await Bun.write(p, Buffer.from(typeof r === "string" ? r : r.data, "base64"));
}, { args: ["/tmp/x.png"] });
```
This puppeteer variant returns the base64 **string directly**, not `{data}` — handle both or you write `undefined`.
- `computer.window("Firefox").screenshot()` fails (`captureWindow: unavailable` under niri) — don't go that way.
- Downscale before embedding: `magick "$f" -resize 1000x "$f"` keeps dark-theme UI PNGs at 20–80 kB each.
## Cookie state (corrected 2026-10-21 — the old claim was wrong)
- Cookies DO cross `browser.open` calls: the headless browser is one shared process with one jar (a stale persona session from an earlier campaign hijacked an "anonymous" walkthrough tab).
- `tab.clearCookies()` is **domain-scoped to the current page origin** — running it on `about:blank` clears nothing and silently leaves the previous session live (a "signer" tab showed the admin's dashboard). Correct sequence: `goto(app-origin)` → `clearCookies()` → `reload()`, then verify with `page.cookies()`, not with page text.
- Cookie proof (ground truth):
```js
const cs = await page.cookies()
return cs.map(c => `${c.name}=${c.value.slice(0,12)}… dom=${c.domain} exp=${c.expires}`).join("\n")
```
## Committing screenshots: the secret-gate false positive
- Data-URI PNGs in a diff WILL trip shape-based secret gates (`ASIA[A-Z0-9]{16}` etc. appear as random substrings in base64 streams, worse with case-insensitive grep).
- Prove noise, don't assume: run the gate's own pattern with `grep -oEi` over `git diff --cached` and eyeball the matches (image-byte substrings, not credentials; confirm screenshots show passwords masked).
- Then use the gate's sanctioned override with the evidence in the commit body: `ALLOW_SECRETS=1 git commit -m "... ALLOW_SECRETS: the N hits are ASIA-shape coincidences inside PNG base64 ..."`. Never `--no-verify`.
## API signatures (guesses cost calls)
- `browser.open({ url, name })` — options object; `browser.open(url)` → TypeError.
- `tab.wait(ms)` does not exist → `await Bun.sleep(ms)`.
- `tab.run(fnOrCode)` executes in an isolated **Node/Puppeteer realm** (`{ tab, page, browser }`, plus `args` as trailing params), NOT the page — `document is not defined` means you wanted `tab.evaluate(exprString)` (page-global expression; no top-level `return`; wrap an IIFE when you need one).
- `tab.text("svg")` etc. are direct page helpers — prefer them over evaluate for plain reads.
- For app UIs with labelled forms, `role/textbox[name="Exact label *"]` beats `label=...`: visible labels often include a required-marker the source selector lacks.
## Eval cell output
- `log(...)` inside a JS eval cell can be swallowed (printed nothing across several runs); **`return` from the cell always shows**. Put every value you need to see into the return string.
## Static page verification loop
Serve with `python3 -m http.server <port> -d <dir>` spawned via `Bun.spawn`, open, `tab.evaluate` anchor cross-check (`hrefs` vs `ids`), `img` census (`complete && naturalWidth > 0`), screenshots, `tab.close()`, `proc.kill(9)`. If a rendered label looks wrong in a low-res screenshot, read the real DOM text before "fixing" markup — `&lt;id&gt;` renders as `<id>` and misreads as `</id>` at thumbnail size.
@@ -0,0 +1,24 @@
---
name: correct-stale-todo-before-next-turn
description: "Recover from marking a todo done on an inferred outcome instead of the verified one: rm the wrong entry, re-append with the true state, and state the correction before continuing."
---
## When this applies
You reported a task as finished (or told the user an outcome) from the *expected* result of a script or plan, while the evidence you had already fetched in the same turn says otherwise. Common shape: a chain script documents `rm -f FILE` "on success", you read the docs, announce the file gone — and the same output also contains `ls -l FILE` (file present) plus the gate line `... kept (recordings incomplete)`.
Never let the script's intent outrank the captured output. An error you noticed yourself is fixed mechanically, not narrated later: the tool record and the transcript are both verifiable, so a wrong `done` is corrected on the next turn, in the same message as the next real action.
## Procedure
1. **Re-read your own claim against the evidence** you already have (the exact `ls -l`, `df`, log tail line, rc file). Quote the contradicting line to yourself before deciding.
2. `todo op:rm task:<the wrong entry, exact string>` — do not re-append over a `done` item, and do not edit it in place with a new phase.
3. `todo op:append` the same work with its **true** state in the text, so the list itself carries the reason: `Purge X (still on disk: the chain's gate kept it because <evidence>; <script> removes it when <condition>)`.
4. State the correction in one line to the user, naming the evidence that was already in your own message. Do not restate the whole prior turn and do not re-explain the plan.
5. Only then continue with the next real action — pair the correction with a tool call, never with a bare message.
## Traps this prevents
- A `done` item silently becomes the record of an action nobody took; the next agent trusts the list instead of the host.
- "Documented behaviour" substituted for "observed behaviour": the script's comment says what happens on success, not whether success happened. Read the gate line, the rc file, or the `ls` before claiming the effect.
- Stacking the next mistake on top: a purge or follow-up step launched because the predecessor was believed complete.
@@ -0,0 +1,64 @@
---
name: deploy-target-dirty-blob-audit
description: "When scripts/deploy.sh on serverAppConfigs aborts with \"Your local changes to the following files would be overwritten by merge\", prove whether the target's dirty file is already-merged content before restoring it, then deploy from the main checkout."
---
# Deploy target dirty-file audit
`deploy.sh <host>` pushes the local branch, then ssh-pulls on the target. The pull is
`git pull --ff-only origin main` on the TARGET checkout, so any uncommitted edit there
aborts the whole deploy with:
```
error: Your local changes to the following files would be overwritten by merge:
scripts/run-qwen38-bf16.sh
Aborting
```
Nothing is dispatched, so the service is untouched (no wasted outage) — but the deploy
cannot proceed until that path is reconciled. This happens when an agent edited the file
directly on the host (the repo's "never edit on the remote" law) and committed the same
bytes later from a worktree, leaving the host's HEAD behind its own working tree.
## 1. Classify before touching anything
```bash
ssh <host> 'cd <checkout> && git status --porcelain' # every dirty path
ssh <host> 'cd <checkout> && git hash-object <file>
&& git rev-parse <target_sha>:<file>' # compare blobs
```
- Hash equal to the blob the pull wants (`origin/main:<file>`, or a commit main contains)
→ the edit is already merged; restoring is lossless, no trash copy needed.
- Hash matches neither HEAD nor the incoming blob → real unmerged content: `cp` it to
`.AGENTTRASH/YYYYMMDD-topic/` first, then decide.
- `D` entries under `.AGENTTRASH/` (deletions, not modifications) do not block the pull
unless those paths changed between HEAD and main. Leave them — only Frank purges trash.
## 2. Restore, then deploy from the MAIN checkout
```bash
ssh <host> 'cd <checkout> && git checkout -- <file>'
cd ~/Projects/serverAppConfig && ./scripts/deploy.sh <host> --service <svc>
```
`deploy.sh` refuses with `FAIL: deploy.sh must run from the main branch.` when run from a
feature worktree — use the host's `~/Projects/serverAppConfig` main checkout, and pull it
`--ff-only` first so `[1/2] Pushing main to origin` reports `Everything up-to-date`.
A missing server-side path (`/home/dank/models`) plus an `rm -f` in the chain means the
`.rad` was already retired; confirm before relaunching anything that needs it.
## 3. Long deploys never fit an async bash slot
An LLM-service deploy waits up to 20 min for model load plus the Playwright gate, but an
async `bash` call is killed at 600 s. Launch it as a named service instead, then watch the
pid with a deadline-free watcher:
```bash
bash command="cd ~/Projects/serverAppConfig && ./scripts/deploy.sh <host> --service <svc>" \
name=deploy-<host> ready.log="Deployment plan"
# then: while kill -0 <pid> 2>/dev/null; do sleep 20; done (timeout 0, async)
```
`ready.log="Deployment plan"` matches the plan banner, so readiness reports the deploy
started rather than finished; the real result arrives when the watcher completes.
@@ -1,49 +1,44 @@
---
name: docker-pin-latest-census
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag census plus digest-equals-latest verification, no docker CLI needed."
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag enumeration, the rolling-latest trap, digest pinning, and rollback proof."
---
# Is my pinned image actually the latest?
Use when asked whether a Docker Hub / lshr.io image pin is the latest version before bumping it, or when a manifest claims an image is stale. Registry API is the answer; the `latest` tag is not.
Use when bumping a pinned container image tag (e.g. `lscr.io/linuxserver/*` quadlet `Image=` lines) and "latest" must be PROVEN, not assumed. Works from any box with curl — no podman/docker daemon required. `lscr.io` fronts Docker Hub; its tags are NOT on quay.io (querying quay returns nothing and reads like "no newer tags exist" — false).
## 1. Full tag census via the registry API
## The API
```bash
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:<owner>/<repo>:pull" \
| python3 -c "import json,sys;print(json.load(sys.stdin)['token'])")
curl -s "https://registry-1.docker.io/v2/<owner>/<repo>/tags/list?n=2000" \
-H "Authorization: Bearer $tok" | python3 - <<'PY'
import json,sys,re
tags=json.load(sys.stdin).get('tags') or []
# linuxserver form: <version>-<sha>-ls<N>; group by version, keep newest ls per version
v={}
for t in tags:
m=re.match(r'^(\d+(?:\.\d+)+)-([0-9a-f]+)-ls(\d+)$',t)
if m: v.setdefault(tuple(map(int,m.group(1).split('.'))),[]).append((int(m.group(3)),t))
for k in sorted(v)[-5:]:
print('.'.join(map(str,k)),'-> newest:',max(v[k])[1]) # max on (ls#, tag) tuple
PY
# hub: token then paginate (the list is 100 per page)
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" | jq -r .token)
for off in 0 100 200 300 400 500; do
curl -s -H "Authorization: Bearer $tok" \
"https://hub.docker.com/v2/repositories/${repo}/tags?page_size=100&ordering=last_updated&page=${page}" \
| jq -r '.results[] | "\(.name)\t\(.last_updated)"'
done | sort
```
- Sort versions as **tuples of ints**, never strings (`1.4.10 < 1.4.9` lexicographically).
- Keep the max `ls` build per version, then the newest version — both can lag independently.
- Trap already eaten: joining the tuple key to a string mid-loop then re-looking-it-up → `KeyError`. Keep keys as tuples; format only when printing.
- Some channels have NO `latest` tag in `/tags/list` (or the endpoint truncates) — absence there is not evidence.
## 2. Digest-proof the candidate against `latest`
```bash
for t in latest <candidate-tag>; do
curl -sI "https://registry-1.docker.io/v2/<owner>/<repo>/manifests/$t" \
-H "Authorization: Bearer $tok" \
-H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.docker.distribution.manifest.v2+json" \
| grep -i '^docker-content-digest'
done
# lscr.io (linuxserver): tags are semver, `latest` is a rolling alias
curl -s https://images-api.github.com/v2/linuxserver/${repo}/tags/list # wrong host for lscr
curl -s "https://hub.docker.com/v2/repositories/linuxserver/${repo}/tags?page_size=100" | jq -r '.results[].name'
```
Equal digests ⇒ the pinned tag IS what `latest` serves right now — the strongest "ensure latest" evidence, and recordable in the endpoint page's pin note (the dank-fleet law: *pinned; verified == the registry's `latest` manifest digest at pin time*). A multi-arch index digest matches only if both requests send the same Accept set — send all three types above.
Docker Hub's own tag list (`hub.docker.com/v2/repositories/<repo>/tags`, ordered by `last_updated`) is the practical source. The registry API (`/v2/<repo>/tags/list`) needs the bearer token flow above; both return tags, neither sorts by semver.
## 3. Bump through git, not by hand
## The `latest` trap
Edit the quadlet/unit `Image=` line + the vault page's pin row (one owner each), run the repo suite, PR, deploy `--service <svc>` only. Rollback stays "revert one line".
- `latest` is a **rolling alias**, not a version. "Pin is older than `latest`" says nothing about whether a *versioned* tag exists.
- `latest` can lag or be rebuilt without a version bump; comparing a digest pin against it proves nothing.
- For linuxserver images: `stable` is the rolling branch and version tags (`2026-09-30`) are what you pin; multi-arch manifest lists mean the same tag has per-arch digests — the pin must be the manifest-list digest (`RepoDigests[0]`) you actually pull, or a re-pull on another arch silently changes the image.
## Verification recipe (before bumping a pin)
1. Read the pinned tag and digest from the manifest/quadlet.
2. Enumerate tags from the registry API (paginate; `last_updated` is not semver order).
3. Compare **versions**, not recency: does a version greater than the pinned one exist?
4. If bumping: pull the new tag and record `docker/podman inspect --format '{{index .RepoDigests 0}}'` as the new digest — pin the digest, not the tag.
5. Prove the service: it starts, version endpoint answers the new version, no crash-loop, and the rollback is the previous digest.
## Report
"pinned: X (digest …), latest available: Y, upstream changelog: …" — or "pin is current: no version tag newer than X exists". Never claim staleness from the `latest` tag alone.
@@ -1,35 +1,60 @@
---
name: docs-dedupe-verify-campaign
description: "Run an exhaustive docs dedupe + fact-verification campaign in dank/command-center: scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
description: "Run an exhaustive docs dedupe + fact-verification campaign (command-center or any OKF vault, e.g. franksign): scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
---
# Exhaustive docs dedupe + verification campaign (dank/command-center)
# Exhaustive docs dedupe + verification campaign (dank/command-center; transplants to any OKF vault — proven on franksign 2026-10-21, #64 2026-10-21)
## Frame
- Read `omp-docs/references/okf-spec.md` + root `omp-docs/index.md` first; inventory with `find omp-docs -name '*.md' | wc -l`.
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir).
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees.
- Read `omp-docs/references/okf-spec.md` (or the vault's vendored OKF contract, e.g. `franksign-docs/read-only/okf-v0.2-spec.md`) + root `index.md` first; inventory with `find <vault> -name '*.md' | wc -l`.
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir). On franksign the whole mechanical gate is one seam: `bun tests/lib/okf-lint.mjs` (frontmatter, links, index coverage, version-once) — run it before and after.
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees. In a session worktree, `git checkout -b` is intercepted anyway: commit on the session branch, then named-ref push `HEAD:refs/heads/docs/<n>-slug` (verify `git log origin/main..HEAD` shows ONLY your commit first).
- Open the Gitea issue first (multi-moving-parts law); branch name `docs/<issue#>-<slug>`.
## Scout fleet (overlap detection)
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task (skills trees, llama trees, pi vs packages, niri, sleep/power, core, gitea, gitea+cross-cutting sweep). Cap 2 concurrent; queue rest.
- Ledger format per cluster → `local://dedupe/<cluster>.md`: inventory table (file | lines | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE), overlaps, facts-to-keep, inbound links, staleness.
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. NEVER apply a scout verdict without re-checking the file, the tests, and the changelog yourself.
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task. **Fan out 2-3 clusters at a time, never seven**: `task.maxConcurrency` is 2 on this fleet, so the excess only queues, and `proc://` reports queued tasks as `running` — identical uptimes with zero deliveries means thrash, not progress (#64: seven scouts, nothing in 17 min; killing the five redundant ones let the two survivors land within minutes).
- Do NOT scout a cluster you have already audited inline. Kill the redundant jobs early with `write proc://<id>/kill` — the endpoint is the scarce resource, and your own session shares it.
- Ledger format per cluster: inventory rows (file | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE | evidence line/command). **Demand the ledger as the yield TEXT, never a `local://` file reference**: #64 produced one 0-byte ledger plus three files whose names contained the scout's own serialized parameter block (`gitea-caddy-inventory.md` + newline + a parameter tag), recoverable only by `ls`-ing `~/.omp/agent/sessions/<session>/local/<dir>/`.
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. Franksign 2026-10-21: 2 of 3 cluster scouts fabricated — "7 missing component stories" refuted by the pairing-law seam that runs green on every push, invented entry/page counts, DELETE verdicts on live-box census numbers the purpose test KEEPS; a third hit the 30-min runtime cap producing zero output. #64 same day: three pages "missing an H1" that have one on their first heading line; a 215-line `omp-docs/AGENTS.md` that does not exist; a "wikilinks + 700-char wall" defect in `core/settings.md` that a commit from that same morning had already removed; `core/subagent-limits.md` called superseded while `agent/config.yml` carries the cited `maxRuntimeMs` / `softRequestBudget` / `maxRecursionDepth` values verbatim.
- **Four pre-checks before a scout row enters the diff**: (1) the file exists; (2) the named heading/marker is actually absent (`grep -c '^# '`, `grep -c '^# Citations'`); (3) `git log --oneline -3 -- <file>` — a same-day commit may already have fixed it; (4) if the claim is "config supersedes the doc", grep the config for the cited keys.
- Cluster sizing: for a vault under ~20 pages, scout fleets are net-negative — audit inline yourself (reading 5 pages beats babysitting a scout). Reserve scouts for multi-hundred-page trees.
## Mechanical sweeps that pay (run on every vault)
- Empty-text links: `grep -rnE '(^|[ ,])\]\(' <vault>` — a generated "See Also" pass left five `- ](index.md)` links (#64); rendered, an empty bullet reads as style, never as a broken reference.
- Citation convention census BEFORE adding anything: count `grep -rl '^resource:'` against `grep -rl '^# Citations'`. command-center's convention is `resource:` frontmatter (12 vs 3) — "add Citations to six pages" is a convention change, not a fix; only the pages missing `resource:` were real hits.
- Long-line walls: `awk 'length>300 {print FILENAME":"NR"("length")"}'` — reflow prose to ~100 columns, wording intact; leave tables (they cannot wrap) and the text of verbatim blockquotes alone.
- Dated log headings: three `## YYYY-MM-DD` sections for one day is three logs wearing one date. Collapse to one `##` plus one `###` per campaign, and write the shape into the spec page so nobody has to infer it.
## Fact verification
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32).
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32; #64 then found `8C/16T` where SMT is off, a "masked" unit that is simply absent, Pi-hole missing from a DNS list, and a weekly-drifting engine count that became a size class with `/config` named as the only authority).
- Local box facts too: `pacman -Qs <pkg>` / `command -v` before trusting a page's "installed tool" claim (#64: `wlr-randr` documented as available, never installed).
- Unit health is not endpoint health: an `:7770` that answers `/v1/models` while its systemd unit sits inactive `Result=exit-code` means a container escaped its unit. Report the drift to the operator; never record transient state as docs, and never touch a production unit without a per-run OK.
- Code-bound facts (franksign-style): every doc claim about routes/columns/shipped status gets grepped against `apps/*/src`, `packages/db/migrations/` — docs narrating features that never shipped ("page ships in the slice" while only the API landed) get truthed with a status pointer to the owning issue.
- External claims: web_search/web read yourself (no subagents per Frank's rule). Kill fabrications outright (invented distro `FreeToken`, nonexistent `ornith-llama-cpp-setup` tree, unattributable "44.5% peer-reviewed CUA") — delete/drop with an explicit note, never keep "plausible" numbers.
- Policy drift (live ≠ tuned docs): flag in the doc + PR body as an operator decision; NEVER silently re-apply policy.
- Deletion proposals from scouts die by default: archived trees carrying an honest archive banner (`pi/**`, banner 2026-10-18) stay, a directory `index.md` cannot be deleted (index-coverage gate), and incident logs are laws, not bookkeeping. Deletion is a pruning session with Frank present.
## OKF §9 / frontmatter audit (cheap wins, check every vault)
- Log entries filed under the wrong ISO-date heading (crossed midnight): cut the entry (`CUT N.=M @reg`), insert the new date heading above the old one (newest-first headings), paste the entry under it.
- Missing trust family (`generated` on machine-written pages) — lint may not catch it; grep every frontmatter block.
- Laws living only in `log.md`: promote each to its owner page (mechanism law → architecture page owning the thing), leave the log entry as the incident record; the HANDOFF/README boundary then points at the vault instead of restating.
- A law restated in two tiers (vault spec page + always-loaded `AGENTS.md`) is not a duplicate to cut: name the co-ownership in the spec page ("change one, change the other") instead of silently dropping one copy.
## Log compaction (log.md)
- Entry law: dated entry = what changed + durable lesson + concept-page pointer; narratives recoverable via `git log --follow`. Record the law at the top of the log. Tag historical-era entries.
- One `## YYYY-MM-DD` per day, `###` per campaign. Put the fan-out and ledger-verification lesson in the entry — that is what stops the next campaign from trusting a scout.
## Gates + landing
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands.
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands. On franksign: kill the :3000 demo stack first (a live API hangs the gate ~10 min; law lives in `franksign-docs/architecture/testing.md`), then `bun tests/lib/okf-lint.mjs && tests/run.sh docs`.
- Before opening the PR: `git diff --stat origin/main..HEAD | grep -i 'tmp\|scratch\|\.bak'` — scaffolding merged under a `.tmp-` name in the 2026-10-19 campaign and needed a follow-up PR to promote.
- PR via API (skill `gitea-api-pr-operations` for token-extraction + merge payload); `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue.
- After Frank's merge OK: propagate deployment `git -C ~/.omp fetch && git merge --ff-only origin/main`; reap campaign worktree (`git worktree remove --force`) + delete branch local/remote; close issue with landing comment.
- PR via API (skills `gitea-api-pr-operations`, `gitea-merge-form-encoded`): token through a 600-perm `-K` curl config, never `-H` (`/proc/<pid>/cmdline` is world-readable); create/PATCH = JSON WITH Content-Type, merge = form body with NO Content-Type, `do=merge&merge_style=no-ff`, success = HTTP 200 empty body verified by landed state, not by re-reading the PR. `shred -u` the cfg and payload afterwards. `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue — ownership-move table, re-read facts with the command that read them, and rejected findings with their evidence.
- After Frank's merge OK: propagate the deployment with `git -C ~/.omp fetch && git merge --ff-only origin/main`, but first prove the FF cannot touch live machine-tier state: `comm -12 <(git -C ~/.omp status --porcelain | grep -v '^??' | awk '{print $2}' | sort) <(git diff --name-only HEAD..origin/main | sort)` must print nothing, and re-check the dirty count after the merge.
- Worktree reaping is the live-sync heartbeat's job (`wt/*` older than 7 days) unless Frank says otherwise — no `rm`, no deleting a session branch you did not create.
## Line-edit op syntax trap (bit three times)
## Line-edit op syntax trap (bit five times)
- Valid ops: `PUT N.=M:` (body `+text` lines), `CUT N.=M` / `CUT N.=M @reg`, `PUT <N @reg`, `PUT >N:` / `PUT <N:`. There is NO `PUT N+=:` form — it lands in the file as a literal garbage line and shifts the insert to the wrong position. For gap-insert use a lone `PUT >N:` or `PUT <N:` per call; pre-compute line numbers against the file map (ops reference original lines within one call, but a failed op poisons positions).
- **PUT with an empty body DELETES the range** (franksign 2026-10-21: a typo-fix `PUT 7-7:` with empty body deleted the whole entry). Bodyless deletes: use `CUT N.=M`.
- A `CUT`/`PUT` range outside the file errors cleanly (`out of range (file has N lines)`) — that is the cheapest way to learn a body was shorter than assumed; re-read the tail rather than guessing offsets.
- Bare body rows get auto-prefixed with `+` and a warning — harmless, but never rely on it: prefix every body row yourself.
- Editing a PUT body that ends adjacent to the next original line can silently consume the closing frontmatter `---` — after any frontmatter-range PUT, re-read lines 1–10 of the result.
- Recovery recipe when a literal op line lands: read exact bounds, `CUT N.=N` (junk), `CUT A.=B @block` (misplaced section), then `PUT <M @block` at the true anchor, re-read tail, add spacing blank if a `---` now abuts a list line.
@@ -1,62 +1,29 @@
---
name: fleet-service-runtime-census
description: "Answer \"what is service running right now?\" for a dank-fleet service: live endpoint probe plus host units, image ids, live-vs-repo env, and drop-ins."
description: "Answer \"what is service running right now?\" for a dank-fleet service: unit state, container image/digest, endpoint contract, loaded env — machine over repo, in that order."
---
# Fleet service runtime census
Answer "what is service running right now?" for a dank-fleet service without trusting the repo. Repo state is intent; the machine is truth, and they differ by however long since the last deploy.
Answer the question "what is `<service>` running currently?" with evidence, not from docs. Docs say what *should* run; the host says what *does*. Run all five probes, then report one table.
## Order of operations (cheap → expensive)
## 1. Probe the published port (works from any LAN box, no ssh)
1. **Unit state on the host** (rootless quadlet ⇒ `--user`; some units are root):
`ssh -o BatchMode=yes host 'systemctl --user show S -p ActiveState -p SubState -p ExecMainStartTimestamp -p NRestarts'`
Boot-loop tell: `NRestarts` climbing with a short `ExecMainStartTimestamp`.
2. **The container's own view**: `podman ps --filter name=S --format '{{.Image}} {{.Status}} {{.Names}}'`, then `podman inspect` for `Image` (tag **and** `RepoDigests`) and `Env`.
3. **The endpoint contract, from the service itself** — this is the only answer that matters to a consumer:
`curl -s host:port/v1/models` (served model id), `curl -s host:port/props` (`n_ctx`, `n_ctx_train`, caps), `curl -s -o /dev/null -w '%{http_code}' host:port/health`.
4. **Effective flags**: the live process is the truth. `systemctl --user cat S` shows `EnvironmentFile=`; read that file. A repo `scripts/S.env` can be ahead of, or behind, what the running unit loaded — and a unit restart is what re-reads it.
5. **Logs last**: `journalctl --user -u S -n 80 --no-pager` for the boot ledger / refusal messages, after you know whether it is even up.
```bash
curl -s -m 10 http://<host-ip>:<port>/health; echo
curl -s -m 10 http://<host-ip>:<port>/v1/audio/voices # per-endpoint extras: /v1/models, /metrics
```
## Facts that bite
The extra route matters more than `/health` for identity claims:
- llama.cpp and radiance **echo the requested model name** — a 200 proves nothing; read `/v1/models`.
- halogen 404s `/completion`, `/ping`, `/props`, `/tokenize`.
- The frank-tts façade answers `/health` **without dialing the engine**, so "façade healthy, engine down" is a real state; the voice list in that same response is the honest census.
- **A service is up ≠ configured how you think.** llama.cpp's `/v1/models` reports `max_model_len`, not the engine's per-slot ceiling; halogen reports `n_ctx` per slot. Read `/props` before quoting any window number.
- **`systemctl is-enabled` answers enablement, not activity.** A timerless service reports `linked`/`static`; `deploy.sh` that prints `ENABLE` for it is lying. Read `systemctl is-enabled --quiet` exit status, not the text.
- **`NRestarts: 0` with `ActiveState=activating` is a crash-loop only if the timestamps move**; check `ExecMainStartTimestamp` twice, 5 s apart.
- **Read-only is always allowed** from the dev box (`ssh -o BatchMode=yes`); state changes on remote boxes are not, without a per-run OK. Production services (`frank-archon`, `frank-dragoon`, `frank-zealot`) are never stopped/restarted to "see what happens".
- **Digest-pinned images**: "when was this image updated?" is answered by `podman inspect` `RepoDigests` + the registry API, not by the unit's start time (a restart re-uses the local digest).
## 2. Units + containers on the host (ssh as the deploy user, no sudo)
## Report
```bash
ssh <host> 'systemctl --user status <svc>.service <engine>.service --no-pager \
| grep -E "^(●| Active| Loaded)"; \
podman images --format "{{.Repository}}:{{.Tag}} {{.ID}} {{.Created}}"; \
podman ps -a --format "{{.Names}}\t{{.Status}}\t{{.Image}}"'
```
- `active (running) since …` + `Up N hours (healthy)` is the answer; `activating (auto-restart)` with a ~50 ms exit is a **preflight refusal**, not an engine fault.
- **NEVER prefix these podman calls with `sudo`.** Containers live in the deploy user's rootless storage; `sudo podman inspect frank-tts-engine` returns `no such object` while the plain form succeeds. Treating that as "container missing" is a false finding.
## 3. Drop-ins — the override that no health gate sees
```bash
ssh <host> 'systemctl --user show <svc>.service -p DropInPaths; \
systemctl --user show <engine>.service -p DropInPaths'
```
`DropInPaths=` must be empty. A leftover bench drop-in (`Environment=FRANK_<SVC>_ENV_FILE=/tmp/<arm>.env`) makes the next deploy boot a different model at a different shape while `/health` still answers 200 and the e2e gate passes.
## 4. Live env vs repo env — the only truth test
```bash
# repo side (local checkout)
grep -vE '^\s*#|^\s*$' scripts/<svc>.env
# host side
ssh <host> 'sudo cat /home/dank/serverAppConfigs/scripts/<svc>.env' | grep -vE '^\s*#|^\s*$'
```
Diff them. Per repo law the env file beside the serve script **is** the authority; a doc page that disagrees with it is the bug, and host-side drift means an uncommitted edit. Also check `git log --oneline -3 -- scripts/<svc>.env` to confirm the host isn't ahead of git.
## 5. Report
Table of: unit state + since, image repo:tag@short-ID + age, backend/alias actually served (from step 1, not from the name), drop-ins, and whether live env == repo env. State explicitly when a fact came from docs rather than from a probe.
## Traps that produce false findings
- Digest-pinned images: the local image ID's *creation date* answers "is this stale" better than the tag; `latest` + `Pull=never` is a locally built image, not a registry pull.
- `ssh` user context changes what podman sees (root vs rootless) — see step 2.
- Two ports on one host are two providers in `models.yml`; a port mismatch is a rename or drift, not a config typo.
One line per layer with the evidence: unit (`active (running)`, since, restarts), container (image + digest), endpoint (served id, `n_ctx`, health code), loaded env (file path + the knobs that matter), and explicitly which of those came from the machine versus the repo — that gap is the answer to most "why is it behaving like that" questions.
@@ -1,35 +1,37 @@
---
name: frank-job-hunt-maintenance
description: "Maintain frank-job-hunt agents: live-census gate hardening loop, registry locator verification, and the workstation/Dankserver/Dankclaw deploy-sync ritual."
description: "Maintaining frank-job-hunt: the census gate's fail-closed rules, ATS locator token derivation, and the two-host deploy-sync verification."
---
# frank-job-hunt maintenance
Use when maintaining `frank-job-hunt`: the live-census gate, the registry locator, or Dankserver/Dankclaw deploy-sync. The repo is built and deployed; agents live on Dankserver, ntfy on Dankclaw.
Repo `~/Projects/frank-job-hunt` (Gitea `dank/frank-job-hunt` on Dankclaw 192.168.0.143:3000). Production: Dankserver user timers `frank-job-sweep.timer` 07:30, `frank-job-digest.timer` 08:15 (`Linger=yes`). ntfy relay: Dankclaw quadlet `ntfy.service`, route `ntfy.franksdank.cloud` (no anubis), LAN record in `hosts/custom.list` (Pi-hole v6 file at `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`; v6 API paths are hostile — edit the file, then `podman exec pihole pihole restartdns`). Secrets: `~/.config/frank-job-hunt.env` (600, workstation + Dankserver).
## The two rules that caused every bug here
## Law (validated 2026-10-11)
- The fleet judge (`frank-zealot`, dankclaw:7771) is a **ranker, never a gate** — it inflates anything reaching it (non-technical roles scored 88-91). All discrimination lives in deterministic gates (`agents/lib/gates.ts`); zealot only ranks survivors.
- Thinking-model fault: verdict lands in `reasoning_content` with `content:""` → judge sends `chat_template_kwargs:{enable_thinking:false}` (0.7s clean JSON).
- **Family beats topic**: role family (recruiter/sales/finance/PM/designer/people-ops/revops/…) gates `gate:scope` unconditionally — no infra-token override ("DevOps Recruiter" is recruiting). Compound technical titles keep the `INFRA_TITLE` override via `NEG_TITLE` branch.
- **gate:region**: remote is US-only (visa reality). "Remote - Bangalore" = onsite abroad → gate; `US` regex pass wins over `NONUS` match ("U.S. & Canada" passes).
1. **A URL is data.** The census gate and the registry locator treat portal URLs as opaque strings; nothing may template a URL from a company name. `jobs.dankclaw.example` is a fixture host (RFC 2606) — never derive a real domain from a company name, and never click a link the pipeline constructed.
2. **A stale row is worse than a missing row.** Every gate must fail closed: an unverifiable posting is reported as unverifiable, not kept as live. Deleting is allowed; asserting is not.
## Gate hardening loop (live-census, TDD)
1. `bun agents/sweep.ts --dry` (full lane ≈ 30–90 s; 852 postings ≈ 62 pass / 790 filed).
2. Classify leaks in the top-10: family gap / region gap / scoring tail. Accept long tail (Director/Founder edge cases) — digest + human curation is the design.
3. Token fix in `gates.ts` regexes (`NEG_FAMILY` = no override; `METRO`/`US`/`NONUS` for arrangement). NEVER dedupe-check by memory — read file first; the file has bitten line-offset edits repeatedly.
4. Pin every leak with a named regression test in `agents/test/gates.test.ts` before/with the fix.
5. `bun test agents/test/ && bun x tsc --noEmit` — hooks re-run these at commit.
6. Ship: `git commit` (pre-commit = tests + tsc + docs-link + secret scan) → `git push` → `ssh dankserver 'cd ~/Projects/frank-job-hunt && git pull'` (timers pull from master; Dankserver drift = tomorrow's run is stale).
## Census gate
## Registry verification (monthly, `agents/verify-registry.ts`)
Scouts' `ats`/`ats_identifier` are guesses (14/84 were wrong). Ritual:
1. `bun agents/verify-registry.ts` (dry, ~70 s serial) → REPAIR census.
2. `--write` applies; repairs with live jobs preferred over empty boards (0-job boards 404 later).
3. Unresolved = honest manual lane (vanity Workdays, self-built ATS incl. majors, neogov) — NOT a code bug; per-entry error census in sweep output is the same signal.
4. Update `targets/README.md` census lines + `docs/log.md`; commit/push/pull (same ship step).
`scripts/census-gate.sh` decides keep / stale / unverifiable per row.
## Pitfalls
- `--ats=<x>` smoke filters the census line — "N API-sweepable" reflects the filter, not the registry; don't diagnose a phantom bug.
- Workday vanity careers URLs (`careers.uw.edu`) are NOT CXS-API-resolvable; tenant-only identifiers are (site path required).
- Root `README.md` vs `targets/README.md` name collision in edit anchors: use full path headers.
- Zealot endpoint probe: `curl http://192.168.0.143:7771/v1/models`.
- Live proof = HTTP GET on the posting URL against the ATS API or the page itself; `404`/`410` ⇒ stale, non-200 with no body ⇒ unverifiable. Never infer from age alone.
- The gate runs against the *live* registry, so a row that passes here and then fails in production means the registry drifted — re-run before blaming the gate.
- Test seams: fixture ATS server + `…_BASE_URL` env. Tests assert the row's fate (kept / dropped / flagged), never internal state.
## Registry locator
Resolves which ATS a company uses (Greenhouse `boards-api.greenhouse.io/v1/boards/<token>/jobs`, Lever `api.lever.co/v0/postings/<token>`, Ashby `api.ashbyhq.com/posting-api/job-board/<token>`, Workable, SmartRecruiters).
- Token ≠ company name: it comes from the discovered posting URL, so a locator change must keep the derivation (URL → token), not substitute a slug.
- Unknown ATS ⇒ unverifiable, not "no jobs". "Zero jobs" is a strong claim; only a successful API answer with an empty list may say it.
- Rate limits: 429 ⇒ back off and mark unverifiable for this pass, never stale.
## Deploy-sync (Dankserver agents, Dankclaw ntfy)
- Two hosts, two roles: inference/agents on Dankserver, notification sink on Dankclaw. A deploy that only touches one leaves the pair inconsistent — the agent can post to a sink that no longer accepts its token, silently.
- Deploy is `scripts/deploy.sh` from the correct checkout (dev side, clean tree); it pushes the branch first, so a stale local `main` makes the push fail — `git merge --ff-only origin/main` before deploying after any externally merged PR.
- Verify after deploy on the *served* surface: the agent answers a probe, and a test ntfy message arrives on the phone. "The unit is active" is not the proof.
## Tests
Black-box, fixtures over network: fake ATS servers on localhost, fake ntfy sink, temp registry file. Assert output rows and exit status. Never assert that an env var is set or that a helper was called.
@@ -19,6 +19,8 @@ for p in (home/".config/secrets").iterdir():
if m: vals.add(m.group(1))
known[p.name] = {v for v in vals if len(v) >= 12}
```
Also add every secret the repo itself embeds (e.g. credentials living in a *test helper*)
by concatenating its quoted fragments, then search for the joined value.
Report ONLY `value[:4] + len + sha256[:10]`. Never print a value, and never quote a
secret-shaped literal in a doc/commit message — the pre-commit gate will (correctly) block it.
@@ -41,6 +43,19 @@ git status --porcelain -uall --ignored=matching # !! lines: on-disk ignored fi
git config --list --show-origin # local/global/system credentials
git ls-files | grep -E '(auth\.json|trust\.json|\.env|\.pem|\.key|identity)' # quarantine
```
**Invocation forms that actually work** (each of these cost a wasted round-trip):
- `git grep --all -F '<fragment>'` — the options MUST precede the pattern; `git grep -l -F '<frag>' --all`
treats `--all` as a pathspec and silently returns 0 lines. Grepping a *fragment* (not the full
value) is what catches split-concatenation disguises like `"Assf.""1234"`.
- `git ls-tree -r --format=... <rev>` works for ONE rev; pass several revs and it emits nothing,
no error. For sha→path across all refs use `git rev-list --objects --all`, which yields
`"<sha> <path>"` lines directly.
- `git cat-file --batch` fed with `rev-list --objects` lines echoes `sha path` headers, so naive
header parsing misaligns and content is lost. Feed bare SHAs in chunks, or use
`git cat-file --batch-all-objects --batch` and parse `"<sha> blob <size>\n<content>\n"` records.
- Check every worktree/checkout: `git -C <other-checkout> rev-parse HEAD` often differs from the
audited worktree, so the leak may exist on `main` while the audited tree looks clean.
LFS: scan `.git/lfs/objects/**` as BYTES (`decode("latin-1")`) — a screenshot is a classic
secret carrier and the tracked tree holds only pointers.
@@ -53,6 +68,9 @@ Flag every one whose value is in `known` as a LIVE leak; list the rest for triag
- prose: `Set password: user.set_password(...)`, `authenticate(user, password)`
- 40-hex hits that are git object ids / packfile names (`pack-<40hex>.pack`)
- memory/KB records that *describe* a secret operation
- length-threshold gates that miss short passwords: a check like
`len(val) >= 24 and re.fullmatch(...)` clears a real 12-char password, so a passing
secret-scan test proves nothing about short credentials
A hit is a leak only when the value matches a live secret, decodes to a real credential, or
sits in a field a program reads.
@@ -61,8 +79,12 @@ Check file modes of the secret dir (`0600`, dir `0700`) — world-readable secre
most common real finding. Then grep the live values across shell rc files, `~/.gitconfig`,
`~/.npmrc`, `~/.netrc`, `~/.curlrc`, `~/.docker/config.json`, `~/.cache/huggingface/token`,
`~/.aws/credentials`, `~/.ssh/config` and report filenames only.
Also check files the deployer *generates* on hosts: a script that writes a credential file
without `chmod 600` (while sibling writes in the same script do chmod) is the same class.
## 6. Report
Per layer: scope numbers + verdict. Every hit: path:line, redacted value, verdict
(leak / benign + why). Name the gate's coverage gap if the project's own hook misses shapes
you matched. Fix modes/chmod on the spot; changes to a security hook are the operator's call.
A secret in git history is only removable by rotation or history rewrite — say so, don't
imply an edit fixes it.
@@ -1,11 +1,11 @@
---
name: gitea-pr-api-cycle
description: "\"Open/update/comment/merge PRs and issues on the Dankclaw Gitea API (192.168.0.143:3000) with the token-safe sequence (POST /pulls rejects Basic with 'token is required'), numeric label ids, the form-encoded merge field, the async 405 'try again later' merge trap, the empty-token-returns-404 trap, and post-merge live-checkout fast-forward plus the plugin-package-name upgrade gotcha.\""
description: "\"Open/update/merge PRs and issues on the Dankclaw Gitea API (192.168.0.143:3000) with the token-safe sequence (POST /pulls rejects Basic with 'token is required'), numeric label ids, the form-encoded merge field, the async 405 'try again later' merge trap, the empty-token-returns-404 trap, merged-branch deletion via the branches endpoint, and post-merge live-checkout fast-forward plus the plugin-package-name upgrade gotcha.\""
---
# Gitea PR/issue cycle — Dankclaw API
Friction-earned 2026-10-18 (PRs #15–#20, issue #18), re-earned 2026-10-19 on `dank/dankvoice` PRs #4–#6 (the merge payload claim was wrong; an empty token masqueraded as a missing repo), and re-earned 2026-10-20 on `dank/command-center` PRs #55…#57 (the 405 recalculation trap). Re-earned again 2026-10-20 on `dank/dankvoice` #25 (merge gate loop worked; suite verdict delivered as an issue comment). Re-earned 2026-10-21 on `dank/command-center` issue #58 / PR #59 (issue labels are numeric ids; the `<type>/<issue#>-<slug>` branch name has to be right *before* the first push). Same day on `dank/serverAppConfigs` #64: the Basic-vs-token split below cost two wasted POSTs.
Friction-earned 2026-10-18 (PRs #15–#20, issue #18), re-earned 2026-10-19 on `dank/dankvoice` PRs #4–#6 (the merge payload claim was wrong; an empty token masqueraded as a missing repo), and re-earned 2026-10-20 on `dank/command-center` PRs #55…#57 (the 405 recalculation trap). Re-earned again 2026-10-20 on `dank/dankvoice` #25 (merge gate loop worked; suite verdict delivered as an issue comment). Re-earned 2026-10-21 on `dank/command-center` issue #58 / PR #59 (issue labels are numeric ids; the `<type>/<issue#>-<slug>` branch name has to be right *before* the first push). Same day on `dank/serverAppConfigs` #64: the Basic-vs-token split below cost two wasted POSTs. Re-earned 2026-10-21 on `dank/franksign` #30 (branch-delete endpoint guess cost one 404; docs bullet went stale under the merge).
## Where
- API base: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>/{pulls,issues}` — repo varies (`command-center`, `dankvoice`, `serverAppConfigs`); numbering is per-repo, so a footer citing `#36` means nothing without the repo.
@@ -31,6 +31,7 @@ Friction-earned 2026-10-18 (PRs #15–#20, issue #18), re-earned 2026-10-19 on `
- **Issue labels are numeric ids, not names.** `POST /issues` with `"labels":["ready-for-human"]` answers `422 {"message":"[]: json: cannot unmarshal JSON string into Go int64 within \"/labels/0\""}`. Resolve first: `GET /labels` → pick `id` by `name` → post `"labels":[20]`. (PRs take no `labels` field on create; label afterwards via `/issues/N/labels` with `{"labels":[id]}`.)
- Comments (the channel for late-arriving evidence, e.g. a test-suite verdict posted after PR creation): `POST /issues/N/comments` with JSON `{body}` — works for PRs too, returns **201**.
- **Open the issue before the first commit**, so `Refs #N` fits the message, and get the branch name right before the first push. `git branch -m <new>` + `git push -u origin <new>` + `git push origin --delete <old>` is a clean rename with no PR yet; do it rather than ship a branch that violates `<type>/<issue#>-<slug>` (a `docs/…` branch carrying a `fix:` commit is the common slip, and `git grep` on the old name still shows it).
- A PR whose branch carries already-merged commits plus one new one is fine: open against `main`, the diff is just the new commit — the cheap way to land a follow-up fix from a worktree whose branch history is the merged stack (no checkout/branch gymnastics needed).
## Merge (the 422 trap) — corrected
`POST /pulls/N/merge` needs **form-encoded** fields; the merge form field name is lowercase **`do`**:
@@ -69,13 +70,15 @@ All three landed within ~5 s of retrying. Treat `mergeable` as a readiness gate,
When two open branches edit the same section of the same file, merge the sibling branch into the later one (`git merge docs/other-branch`) instead of rebasing — Frank rejects the history rewrite. Consequences to state in the PR body: merging the lower PR first reduces the upper diff to just its own work; merging the upper first makes the lower an already-contained no-op. Either order is clean; `git branch -d` on the lower branch refuses until its commits reach `main`, because `-d` checks against the current HEAD.
## Post-merge landing
- Delete the merged branch via the **branches endpoint**: `DELETE /repos/dank/<repo>/branches/<urlencoded-branch>` (`chore%2Fname`) → **204**. The `refs/heads/…` path answers 404 — that is the wrong endpoint, not missing auth (auth failures also 404 here, so check which you changed last). If you then run `git push origin --delete <branch>` it errors `failed to push some refs` — harmless: the API already deleted it; don't "fix" the error.
- Command-center: the checkout IS the deployment — `git -C ~/.omp fetch -q origin && git -C ~/.omp merge --ff-only origin/main`. This refuses if **any** incoming path is dirty in the worktree — see `land-merged-pr-with-blocked-tree-ops` for the shared-workspace case where `checkout` and `stash` are both policy-blocked.
- serverAppConfigs: `deploy.sh` pushes the dev checkout's branch first — if the checkout's `main` is behind a merge you just made (via the API), the deploy dies with `error: failed to push some refs`. `git merge --ff-only origin/main` in the deploy checkout BEFORE running `deploy.sh` after any externally-merged PR.
- **A PR that changes a path you edited live adds that path to the discard list.** Tonight: the live dotfile fix was dirty *and* incoming, so landing needed `git -C ~/.omp checkout -- <that file>` as well as the usual file. Discard is content-free only when you proved byte identity first (`cmp` the branch file against the live one before merging) — say so in the PR body, and note that if `live-sync.sh` commits the dirty file first the merge stays clean because both sides hold identical bytes.
- Package repos (`~/Projects/<pkg>`): same `--ff-only`, then reinstall (`omp plugin upgrade <name>`) and `cmp` the installed `dist/index.js` against the repo `dist/index.js` to prove the live artifact is the merged one. Check `ls -ld node_modules` after landing anything that touched it.
- **`omp plugin upgrade` takes the npm package name, not the short name.** `omp plugin upgrade dankvoice` fails with `Failed to upgrade dankvoice: dankvoice is not installed` while the plugin is plainly installed; the package is `omp-dankvoice`. Find the truth with `ls ~/.omp/plugins/node_modules` or `find ~/.omp -path '*<name>/dist/index.js' -not -path '*/Projects/*'` — the installed tree is unscoped (`~/.omp/plugins/node_modules/omp-dankvoice/dist/index.js`), not `@dank/dankvoice`.
- **`cmp` exiting 1 BEFORE the upgrade is the expected state** (installed bundle is the pre-merge build, diff reported at some char/line). Do not treat it as a failed deploy and do not "fix" the build; run the upgrade, then `cmp` again for byte identity, and grep the new bundle for a string only the merged code contains (e.g. `grep -c "Heckle instead of summarizing"`) as a second, semantic confirmation.
- Prune: `git worktree remove --force <path>` (force: untracked deps link), `git branch -D`, `git push origin --delete <branch>` (allowed: local Gitea). Before removing a worktree, check no live process has a cwd inside it (`ls -l /proc/*/cwd | grep -c "wt/<name>"`) — same guard `live-sync.sh` uses — and copy any file you moved into a gitignored `.AGENTTRASH/` there to its permanent home first, or the removal destroys the only copy.
- Prune: `git worktree remove --force <path>` (force: untracked deps link), `git branch -D`, branch delete on the remote (branches endpoint above). Before removing a worktree, check no live process has a cwd inside it (`ls -l /proc/*/cwd | grep -c "wt/<name>"`) — same guard `live-sync.sh` uses — and copy any file you moved into a gitignored `.AGENTTRASH/` there to its permanent home first, or the removal destroys the only copy.
- **Status bullets written pre-merge go stale at merge time.** A HANDOFF/readme line saying "unmerged (open PR)" is wrong the moment you merge — audit those lines as part of the merge turn and land a one-bullet docs PR (see the Create section's merged-stack branch trick for doing that without a fresh worktree).
## Attribution footer (AGENTS.md)
Every issue/PR body ends with `---`, then `cwd: <session cwd> · session: <session id>`. Read the real values, never invent an id.
@@ -0,0 +1,27 @@
---
name: gitea-silent-ignore-patch-trap
description: "Gitea API answers success codes while silently ignoring mutation bodies: labels in PATCH /issues/N, and any JSON body sent without Content-Type: application/json. Verify mutations with a follow-up GET, never the status code."
---
# Gitea silent-ignore family (issue mutations)
Two verified failure modes on the Dankclaw Gitea API (1.27.3, verified live 2026-10-21 on `dank/franksign`); both **return a success code and change nothing**:
1. **`PATCH /repos/{o}/{r}/issues/{n}` with `{"labels":[...]}`** — 200, labels untouched, even *with* a proper JSON Content-Type. Labels move only via the dedicated endpoints:
- add: `POST /issues/{n}/labels` with `{"labels":[<id>]}`
- replace: `PUT /issues/{n}/labels` with `{"labels":[ids]}` (works without Content-Type)
- remove: `DELETE /issues/{n}/labels/{id}`
2. **Any JSON body without `header = "Content-Type: application/json"` in the curl config** — e.g. closing with `PATCH {"state":"closed"}` returns **201 and no-ops**; identical request with the header closes. curl's default `-d` type (`application/x-www-form-urlencoded`) makes this silent.
## Rules
- Mutating JSON calls: always send `Content-Type: application/json` alongside `Authorization: token` in the same `-K` config file.
- **Never trust the status code for mutations — follow up with `GET` and read the field** (`labels`, `state`). Both traps look like success otherwise.
- Token stays out of `-H` args (world-readable `/proc/<pid>/cmdline`): extract from `~/.config/secrets/gitea` into a 600-perm curl config, use `-K`.
## Quick verification idiom
```bash
curl -sK "$cfg" "http://192.168.0.143:3000/api/v1/repos/dank/franksign/issues/22" \
| jq '{state, labels: [.labels[].name]}'
```
Franksign's vault page `franksign-docs/agents/issue-tracker.md` owns the repo-local copy of this law; this skill exists so other repos (serverAppConfigs, command-center) don't re-burn the cycle.
@@ -1,43 +1,49 @@
---
name: gitea-token-from-http-credential-store
description: "Extract the Dankclaw Gitea token from ~/.config/secrets/gitea (it is the URL password, not key=value lines) for API calls, and diagnose the silent empty-token trap that makes public-repo reads pass while writes fail."
description: Extract the Dankclaw Gitea token from ~/.config/secrets/gitea (a URL-line credential store using http:// and %3a-encoded port) without the empty-token 401; covers the https-only extraction trap and the $$..$$ shell-expansion trap.
---
The Gitea credential store line on Frank's boxes is **http** (not https) with a percent-encoded port, and the **token is the URL password** — there are NO `username=`/`password=` lines:
# Gitea token out of the HTTP credential store
`~/.config/secrets/gitea` is a git **credential-store file**, not a bare token file. On the Dank
boxes it reads:
```
http://dank:<40-char-token>@192.168.0.143%3a3000
http://dank:b7caa2f689cdc00841854b85cd0173386f707d86@192.168.0.143%3a3000
```
## Extraction that actually works
Three traps, all of which produce a silent **401 Unauthorized** rather than a parse error:
```bash
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
curl -s -K <600-perm cfg file> http://192.168.0.143:3000/api/v1/<path>
```
1. **Scheme is `http://`, not `https://`.** An extraction pattern written `s#^https://…#` matches
nothing and yields an empty token. GNU BRE has no `?` quantifier — use the
`http[s]*://` form or Python `https?://`:
## `git credential-store --file … fill` answers EMPTY — do not debug it
```bash
TOK=$(sed -n 's#^http[s]*://[^:]*:\([^@]*\)@.*#\1#p' ~/.config/secrets/gitea)
test -n "$TOK" || echo "extraction failed" # never ship an unasserted capture
```
Tried 2026-10-21 while opening `dank/command-center` PR #61: the obvious extraction
```py
tok = re.search(r"https?://[^:]*:([^@]+)@", pathlib.Path.home().joinpath(".config/secrets/gitea").read_text()).group(1)
```
```bash
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
```
Always assert non-empty (`test -n`) or print `${#TOK}` once: a correct token here is 40 chars,
and the length check catches a wrong pattern without echoing the secret.
prints **nothing**, with no error. It is not a permissions problem and not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure if the `&&` hides it.
2. **The token body contains `$$…$$`.** An omp output token inside a *file* is inert, but pasted
literally into a shell script line it expands (`$$` = shell PID). Keep it in a variable or read
it from the file — never inline it in command text.
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
3. **Port is percent-encoded (`%3a3000`).** Don't rebuild the API URL from that line; the working
endpoints are `http://192.168.0.143:3000/api/v1/...` and `https://gitea.franksdank.cloud/api/v1/...`
— both serve the same API, so a host/port mismatch looks like an auth failure.
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
Auth header for every call: `Authorization: token $TOK` (the `token` keyword is required; a bare
value 401s too).
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
- The temp `-K` config holds a live token: `rm -f` it in the same session that made the calls, and never `cat` it.
**Never print the token.** Extract into a variable, then use `$TOK`; read the file with `read
path=...:raw` only when you must inspect the format.
## Other notes
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
- Never print the token: mask when inspecting the file —
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
Related: `gitea-pr-api-cycle` (POST/PATCH body shapes), `gitea-merge-form-encoded` (merge needs
`do=merge&merge_style=no-ff`, answers 200 with an empty body — verify by fetching the PR, not by
reading the response), `gitea-credential-store-url-format`.
@@ -1,41 +1,48 @@
---
name: gitignore-trailing-slash-symlink-contamination
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get committed; detect with git ls-tree mode 120000, fix with git rm --cached."
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get tracked and pushed as 1-line blobs; same-name rules silently match worktree dirs, and .deploy-stale pollutes. Three fixes with tests."
---
A `.gitignore` entry written with a **trailing slash matches directories only**. A symlink of the same name is not a directory, so it slips past the rule, and `git add -A` in a `git worktree` checkout silently **tracks** it. Cost measured 2026-10-19: a machine-specific symlink (`node_modules -> /home/blank/Projects/dankvoice/node_modules`) landed in `main` with a merged PR.
Gitignore semantics trap with three real-world consequences (serverAppConfigs, 2026-10-19):
Why worktrees are the trigger: a worktree has no `node_modules` (or other installed-deps dir) of its own, so the cheap way to make build/test work there is a symlink into the primary checkout. With `node_modules/` ignored, nothing complains.
## The rule
## Detect
`.gitignore` patterns ending in `/` match directories only. Worktree-relative: `**/node_modules/` is `**/node_modules/` → a symlink named `node_modules` does NOT match, so it is "untracked" no more — it gets added, committed, and pushed as a 1-line blob containing its target path.
## Consequence 1: worktree symlink contamination
- `git add -A` then commit then push in a worktree whose root holds `node_modules -> ...` (the worktree convention symlink) commits it.
- The repo then holds `package/frontend/node_modules` as a text file containing the absolute path on the author's box.
- The receiving worktree gets that 1-line file instead of the symlink → "module not found" and "why is this a 30-byte file?".
- Detection: `git ls-tree -r origin/main | awk '$1=="120000"'` (mode 120000 = symlink) or `git ls-files -s | grep 120000`.
- Fix: ignore rules WITHOUT trailing slash (`**/node_modules`, `**/.venv`) so symlinks match too.
## Consequence 2: same-name worktree rules silently matching worktree dirs
A root rule like `wt-*` (meant to ignore local worktree checkouts) matches every path whose last segment starts with `wt-` — so legitimate content under such a directory goes untracked. `git status --ignored` is the only visible trace; detection: `git check-ignore -v <path>` prints the owning rule + line number. (Repo-specific: `dank/command-center`'s `.gitignore` carries a "per-worktree memory buckets" block ignoring `agent/memories/sharpshooter/wt-*/` and `agent/memories/summaries/wt-*/` — intentional there.)
## Consequence 3: `.deploy-stale`
`deploy.sh` renames a stale tree to `<path>.deploy-stale` before recreating it. Without an ignore rule those directories land in `git status` as untracked and get swept into `git add -A` commits; they accumulate on every server.
## Fixes with tests
```gitignore
**/node_modules
**/.venv
.deploy-stale/
```
Black-box test against the real rules, not a fixture: `git check-ignore -v <symlink-target>` must name the rule. `git check-ignore` takes hypothetical paths — nothing needs to exist on disk:
```bash
git ls-tree HEAD -- node_modules # mode 120000 == tracked symlink
git ls-files -s | awk '$1==120000' # every tracked symlink in the index
git check-ignore -v natives/x.wasm dotfiles/x/.config/noctalia/Noctalia.json
```
## Fix (branch, never on main directly)
Two ignore surfaces must be kept in sync (deploy exclusions vs ignore rules): if a deployer excludes `*/Noctalia*` but the ignore list has only the lowercase class, generated files become tracked-but-never-deployed — test both halves with `git check-ignore`.
```bash
git rm -q --cached node_modules # untrack; leaves the file on disk
```
## Recovery (if contamination already pushed)
plus relax the ignore rule to the slash-free form, with the reason inline so nobody re-adds the slash:
```
# Trailing slash would only match directories: a node_modules symlink (the
# shape `git worktree` checkouts end up with) slips past it and gets tracked.
node_modules
```
Then `git check-ignore -v node_modules` must print the rule, and the suite runs after (the symlink still resolves, it is just ignored).
## Prevent
In a fresh worktree, create the deps symlink **after** confirming the ignore rule has no trailing slash:
```bash
git check-ignore -v node_modules || sed -i 's|^node_modules/$|node_modules|' .gitignore
```
Same class for any ignored-by-convention dir that a worktree might provide as a link: `node_modules`, `vendor`, `.venv`, `target`, `dist` when built artifacts are ignored.
1. `git rm --cached <path>` on the symlink blob, commit, push.
2. Fix the ignore rule first (else the next `git add -A` re-adds it).
3. Untracked ≠ deleted from history: the blob stays in every old commit. Only a rewrite removes it; that is the operator's call, not a cleanup run alone.
@@ -1,44 +1,47 @@
---
name: grafana-provisioned-dashboard-verify
description: "Author and verify repo-provisioned Grafana dashboards on Dankclaw live: staging to the host provider path, the Grafana-13 unified-storage false-negative on grafana.db, the authenticated /api/search check, per-panel PromQL/LogQL validation before shipping, and the Loki $__auto shell-expansion trap in test harnesses."
description: "Author and verify repo-provisioned dashboards on Dankclaw live: Grafana 13 has no unauthenticated API and no local admin, so prove through the e2e SSO browser path, and close the copier gap that hides never-provisioned boards."
---
Verify or ship Grafana dashboards that are provisioned from the repo (`shared/grafana/provisioning/dashboards/*.json`, auto-loaded by the `fleet` file provider). Use when adding/staging a dashboard, or when asked to confirm provisioned dashboards actually load.
## When to use
You changed `shared/grafana/provisioning/dashboards/*.json` (or panel PromQL) and must prove the LIVE boards actually receive data — or a panel/whole board reads empty and you must classify: dead expression vs never-provisioned vs legitimately-empty.
## Paths (Dankclaw)
- Repo source: `shared/grafana/provisioning/dashboards/` (git is the owner).
- Host provider dir: `/opt/grafana/provisioning/dashboards` (root-owned), bind-mounted `:ro` into the container at `/etc/grafana/provisioning/dashboards` (from `dankclaw/observability/quadlet/grafana.container`: `Volume=/opt/grafana/provisioning:/etc/grafana/provisioning:ro`).
- Provider reload interval = 30s, so a staged file appears within ~35s.
## The two gaps this skill exists for
1. **The copier gap.** A repo dashboard JSON is NOT a live dashboard until the deployer carries it. Until 2026-10-21 `copy_grafana_provisioning` in `scripts/deploy-remote.sh` named `llm-fleet.json` alone: the `containers` board was merged, seam-tested, and documented as "working" yet had NEVER been provisioned; the other boards existed on disk (`/opt/grafana/provisioning/dashboards/`) only via one-time hand-copies, and one had silently gone stale. Fixed: copier globs `dashboards/*.json`; `GRAFANA_PROV_ROOT` is the test seam; a bats case executes the copier for real into a sandbox.
2. **The API-proof illusion.** The vault used to say "verify through the APIs". On Grafana 13.2.2 that is dead: `/api/health`, `/api/frontend`, `/api/org`, `/login` all 404 **unauthenticated**, and there is NO local admin password (root `.env` sets no `GF_SECURITY_ADMIN_*`; the only account is the Authentik RP login). Do not hunt for credentials; use the browser path below.
## Stage on the host (canonical install is `deploy.sh dankclaw --service grafana` after merge)
## Procedure
### 1. Mechanical expression census (over SSH — Prometheus binds loopback)
Extract every panel `expr` from the repo JSONs with python, substitute template vars (`$host`,`$container`,`$job` → `.*`, `$__auto`/`$interval` → `5m`), and run each as an instant query against `http://127.0.0.1:9090/api/v1/query` via `ssh dankclaw`. `curl` to `127.0.0.1:9090` from the dev box is blocked; keep queries on-host. Watch zsh globbing on `?`/`[` — quote the remote command.
Empty result ≠ defect automatically:
- `ALERTS` panel empty = **healthy** (ALERTS carries only firing/pending series).
- Zero by design: no cgroup limits exist on the fleet → a memory-vs-limit panel is dead by premise (rewrite as % of host total, don't delete).
- Real join defects: cadvisor `host` label vs node-exporter `job` label sets are incompatible — fix with `group_left label_replace(node_memory_MemTotal_bytes{job=~"node-.*"}, "host", "$1", "job", "node-(.*)")`, and `count by (job) (expr) / on(job) count by (job) (...)` needs the `on(job)` matcher when label sets differ. Parens around `on(host)` after `group_left` are **illegal**; `group_left label_replace(...)` without them parses fine.
### 2. Prove the provisioned files (filesystem = installation proof only)
```
scp -q <files>.json dankclaw:/tmp/
ssh dankclaw 'for f in <names>; do sudo install -m644 -o root -g root /tmp/$f.json /opt/grafana/provisioning/dashboards/$f.json; done'
ssh dankclaw 'for f in /opt/grafana/provisioning/dashboards/*.json; do \
cmp -s "$f" "/home/dank/serverAppConfigs/shared/grafana/provisioning/dashboards/$(basename "$f")" \
&& echo "$(basename $f) same" || echo "$(basename $f) DIFFERS"; done'
```
Missing file or DIFFERS ⇒ the copier (or an unmerged branch) is the bug, not the JSON.
## Verification — what DOES and DOESN'T work
### 3. Browser proof (the real surface)
Throwaway `.mjs` placed **inside `tests/e2e/`** (`@playwright/test` resolves from `node_modules` there; a script in `/tmp` fails with `ERR_MODULE_NOT_FOUND`). Node on this box is Zig-rewritten v26: `readFileSync(p,"text")` throws `ERR_INVALID_ARG_VALUE` — use `readFileSync(p).toString()`.
Sequence, exactly as `specs/20-grafana-sso.spec.ts` drives it:
1. `newContext({storageState: ".auth/qa-e2e-admin.json", userAgent: REAL_UA})` — state is bound to the homepage origin; on the grafana vhost it is only the Authentik session cookie, so you MUST:
2. `goto https://grafana.franksdank.cloud/` and poll until body lacks `"Making sure you're not a bot"` (Anubis PoW warm-up).
3. `goto .../login/generic_oauth` and poll until `url` host is the vhost and path ≠ `/login` (RP round-trip). Landing on `/login` = auth flow broken, not the dashboard.
4. `goto /d/<uid>?orgId=1&from=now-1h&to=now`, wait ~10 s, then read panels: titles via `.panel-title,[class*='panel-title']`; a panel is `NO-DATA` iff its container text contains `"No data"`, `RENDERED` iff a descendant matches `svg:not(:empty),canvas,img`. Panels lazy-render; scroll (`mouse.wheel`) before judging.
Classify: `PANEL-NOT-FOUND` for a whole board ⇒ board not provisioned (step 2). `NO-DATA` with a proven-live expression ⇒ variable/label mismatch — re-run the expression with template vars expanded.
Throwaway scripts → `~/.AGENTTRASH/<date>-topic/` after use; never `rm`.
**DO NOT trust the sqlite `dashboard` table.** Grafana 13 uses **unified storage**: file-provisioned dashboards do NOT live in the legacy `dashboard` table. Reading `grafana.db` (`sudo python3 - <<PY ... select ... from dashboard`) returns **total dashboards: 0** even when everything is provisioned correctly — a false negative. There is also no `sqlite3` binary in the grafana image or on the host. Grafana logs **no** line on a successful provider load (only errors), so a clean `podman logs grafana` proves only "no error", not "loaded".
## Sandboxed deployment-function tests (bats)
To behaviorally test a deploy-remote.sh copier: extract with `sed -n '/^func()/,/^}/p'`, `eval` it, stub `run_privileged() { "$@"; }`, point the function's `*_ROOT` seam env var at a sandbox, execute, assert files. Traps:
- `VAR=x func` prefix assignments do NOT bind for shell **functions** — `export VAR=...` first.
- `mktemp -d` + `trap 'rm -rf' RETURN` races bats' trap machinery: the sandbox vanishes mid-function and bats blames the `trap` line. Use `$BATS_TEST_TMPDIR` (auto-cleaned).
- Prove red on the OLD body by patching a `mktemp -d` copy of `tests/ shared/ scripts/` and running bats there — no checkout mutation.
- Full suite must run from the repo root (`./tests/run.sh`), not from a subdirectory.
**The authoritative check is `/api/search` through an authenticated browser session.** No admin password is configured and anonymous is off — do NOT reset the Grafana admin to do a read check. Drive the RP SSO login with the Playwright `qa-e2e` storage state, then GET the API:
```js
const ctx = await b.newContext({ storageState: '.auth/qa-e2e.json', userAgent: "<chrome UA>" });
const p = await ctx.newPage();
await p.goto('https://grafana.franksdank.cloud/', { waitUntil: 'domcontentloaded' });
for (let i=0;i<30;i++){ const t=await p.locator('body').innerText(); if(!t.includes('not a bot')) break; await p.waitForTimeout(3000);} // clear Anubis PoW
await p.goto('https://grafana.franksdank.cloud/login/generic_oauth', { waitUntil: 'domcontentloaded' });
await p.waitForTimeout(6000);
const r = await ctx.request.get('https://grafana.franksdank.cloud/api/search?type=dash-db&limit=50'); // 200 + [{uid,title}...] = provisioned
```
Run it from the checkout that has `tests/e2e/node_modules` + `.auth` + `.env.local` (a fresh worktree lacks them; symlink `node_modules` and copy `.auth` + `.env.local`). Throwaway `_*.cjs`, then move to `.AGENTTRASH/`.
## Validate every panel query BEFORE shipping (catch empty / mis-scaled panels)
Query Prometheus (`http://127.0.0.1:9090/api/v1/query`) and Loki (`http://127.0.0.1:3100/loki/api/v1/query`) on Dankclaw loopback and confirm non-empty. Real traps found here:
- **Unit mislabel**: `sum by(job,mode)(rate(node_cpu_seconds_total{mode!="idle"}[5m]))` returns **cores per mode** (up to #cores) — unit `short`, NOT `percentunit`. For a per-host busy fraction use `1 - avg by(job)(rate(node_cpu_seconds_total{mode="idle"}[5m]))`.
- **Missing metrics**: `scrape_duration_seconds_bucket` and `prometheus_*`-prefixed self metrics may be absent — confirm from `/api/v1/label/__name__/values?match[]=<metric>` before building a panel; `prometheus_target_interval_length_seconds` (a summary with a `quantile` label) exists.
- **Alertmanager** runs on :9093 but is **NOT a Prometheus scrape target** → its metrics are unavailable; don't build panels on it (they read empty).
- **Loki streams**: journal arrives as `{job="systemd-journal"}`; `container_name` is a **parsed JSON field, not a stream label** → group at query time: `sum by (container_name)(count_over_time({job="systemd-journal"} | json [$__auto]))`.
- **Shell trap in test harnesses**: `$__auto` inside a double-quoted ssh/curl string is eaten by the local shell → empty query → Loki returns empty (looks like a broken panel, is a harness bug). For live validation substitute a concrete `[5m]` window, and drive Loki queries with an on-host Python `urllib` client to avoid quoting hell entirely.
## Datasource uids
`prometheus` (uid `prometheus`), `loki` (uid `loki`); reference as `"datasource": {"type":"prometheus","uid":"prometheus"}` in panel JSON.
## Land flow (repo law)
Branch off fresh `origin/main`; merge needs Frank's explicit current OK; Gitea merge endpoint wants **JSON** `{"Do":"merge"}` (form-encoded `Do=merge` 422s with `[Do]: Required`); then `deploy.sh dankclaw --service grafana` and repeat steps 2–3 as the post-deploy gate.
@@ -0,0 +1,29 @@
---
name: halogen-npu-lane-boot-loop-triage
description: "Triage a frank-dragoon (halogen-flash-server) unit crash-loop caused by the NPU lane: signature classification, restart-debris trap, XRT mount-path census, and the transient env-file proof boot."
---
Triage when `frank-dragoon` (or any halogen-flash-server unit with `HALOGEN_NPU_MODELS` armed) sits in `activating (auto-restart)` with `ExecStart … code=exited, status=1/FAILURE` seconds after start, while `/health` stays dark.
## 1. Classify the exit BEFORE touching the config
- `NRestarts` climbing + exit in <5 s = refusal, not an engine fault. Read `journalctl -u frank-dragoon --since "-30m"` AND the engine log `/home/dank/logs/frank_dragoon.log` (engine stdout never goes to the journal).
- Lane signature: log ends with `halogen npu: … <name>: N files checked against this image's record` then `halogen-npu: No such library '<path>'` + `the NPU engine exited before it was ready (rc=1)` → **the NPU lane exit is fatal to the whole container** (supervisor kills the Flash engine with it). Chat-down is the lane's fault even though chat code never changed.
- KV/pre-flight lines present but no `locked in RAM` line = died before weight pinning.
## 2. The restart-debris trap
`Restart=always` re-runs the failing boot every ~10 s; each attempt leaves GPU/NPU handles being released, and the next boot logs `GTT in use before this start: N GiB … held by 1 process(es)`. That NOTE is debris from your own crash-loop — do not chase it as a foreign co-tenant (real co-tenants: ComfyUI `python` on `/dev/kfd`, find with `sudo fuser -v /dev/dri/renderD128 /dev/kfd`). `sudo systemctl stop <unit>` FIRST, then any diagnosis.
## 3. XRT path census (the `No such library '/opt/xilinx/lib/<multiarch>/libxrt_*.so.2'` case)
- The image ships NO XRT at all (`find / -xdev -name "libxrt*"` inside the image = empty); the NPU child `dlopen`s the host's libs through the serve script's mounts, which land at TWO destinations only: `/opt/xilinx/xrt/lib/<L>` and `$XRT_DIR/<L>`. A child searching `/opt/xilinx/lib/<multiarch>/` names a THIRD prefix nobody mounts.
- The serve loop resolves `REAL="$(readlink -f …)"` and mounts resolved files — if a host package upgrade retargets the symlinks (check `grep libxrt /var/log/dpkg.log` + `ldconfig -p | grep xrt` mtimes vs the last good boot), the file set the child expects can change under byte-identical mounts.
- Rootless gotcha: `podman mount` fails outside `podman unshare` and leaves `$M` EMPTY — `grep -r pat "$M/opt"` then greps your CWD = HOST tree (false hits like `/opt/fastflowlm`). Always `podman unshare bash -c 'M=$(podman mount $C); …'` or `--entrypoint ls|cat` probes instead, and distrust hits that name host-only paths.
## 4. Proof boot without touching production
The serve script honors `FRANK_DRAGOON_ENV_FILE` (sanctioned seam). Throwaway script on the host:
1. scratch env: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + the knob under test.
2. `sed` the serve script to inject the candidate extra mounts into the `NPU_ARGS+=(--device …` line — mount RESOLVED real files (`.so.2.25.00`), never symlinks (vendor: dangling links inside the container).
3. `export FRANK_DRAGOON_ENV_FILE=/tmp/… FRANK_DRAGOON_LOG=/tmp/…; FRANK_DRAGOON_CONTAINER=<scratch-name> timeout 1200 /tmp/serve-test.sh` — the container still publishes 7770, so `systemctl stop` the unit first (port binds at CREATE, exit 126 on race).
4. Retire same session: `podman rm -f <scratch-name>`, trash `/tmp` scripts; the unit stays stopped until the real fix deploys.
## 5. Restore chat while the lane is unfixed
`HALOGEN_NPU_MODELS=` (blank) is the documented one-line lane-off switch — lane absent = endpoint exactly as before the lane existed. Land through git + `deploy.sh dankclaw --service frank-dragoon`; never hand-edit the remote env.
@@ -0,0 +1,66 @@
---
name: halogen-npu-mount-prefix-proof
description: "Prove and fix a halogen frank-dragoon NPU-lane mount failure: crash-loop where a dead lane kills the whole container, dlopen-prefix census, and the sanctioned FRANK_DRAGOON_ENV_FILE proof-boot."
---
# Halogen NPU lane: dead lane kills the container — prove the mount prefix, don't guess
Applies when a `frank-dragoon` unit crash-loops in seconds (`exit 1` in ~200 ms,
`NRestarts` climbing) and the engine log's last lines are:
```
halogen npu: <model>: 15 files checked against this image's record
halogen-npu: No such library '<some/prefix>/libxrt_core.so.2'
halogen npu: the NPU engine exited before it was ready (rc=1)
```
## Laws this incident wrote (2026-10-21, frank-dragoon, 88-restart loop)
- **A dead NPU lane is fatal to the whole container**: the flash-server supervisor
exits with the lane. Chat dies beside a healthy GPU preflight — do not read the
GTT/weights lines as "engine fine, side feature broken".
- The NPU child `dlopen`s distro XRT from **three** container prefixes per library:
`/opt/xilinx/xrt/lib/`, the library's own install path, **and
`/opt/xilinx/lib/<multiarch>/`**. The missing one is exactly what the error names.
- Host state can flip this under a byte-identical container: the lane ran for days
with two prefixes, then a host `libxrt2`/`libxrt-npu2` reconfigure (check
`grep libxrt /var/log/dpkg.log`) made the child demand the third. Image digest,
serve script, payload all unchanged ⇒ suspect HOST package state, not the deploy.
- Fastest discriminator vs engine faults: exit in <1 s with the engine's preflight
lines present = serve/lane gate failure, not model load. Model-load failures take
minutes and print pin/KV lines.
## Proof procedure (sanctioned seam, no scratch ports)
1. Read the error's path verbatim — it names the missing prefix.
2. `sudo systemctl stop frank-dragoon` (unit must be down: the proof binds :7770
at container CREATE).
3. Scratch env file on the host using the bench-sanctioned seam:
```
/tmp/dragoon-nputest.env:
source /home/dank/serverAppConfigs/scripts/frank-dragoon.env
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
```
4. Scratch serve copy that appends the candidate mount to the `NPU_ARGS+=(--device`
line (`sed`), REAL files only (`readlink -f`; vendor forbids dangling symlinks
under /opt/xilinx), and run with:
`FRANK_DRAGOON_ENV_FILE=/tmp/... FRANK_DRAGOON_LOG=/tmp/... timeout 1200 /tmp/serve-nputest.sh`
5. Success = boot ledger (`… in all`) **plus** `npu_api: <model> … buckets u0…u9`
plus `/health` `npu.status ok`. Then codify the mount in the serve script, pin
it in `tests/frank_dragoon_unit.bats` (the fixture XRT dir's basename IS the
multiarch name — name it `x86_64-linux-gnu`), update the endpoint page's NPU
prerequisite to the three-prefix law, and cut over via PR +
`deploy.sh dankclaw --service frank-dragoon`.
6. Retire scratch: the `timeout` kills the proof container; `podman rm -f`,
move /tmp scripts to `~/.AGENTTRASH/<date>-<topic>/`. A proof instance
serving production while you wait is drift — deploy the merged fix promptly.
## Traps
- Greping image contents rootless: `podman mount` fails (needs `podman unshare`)
and leaves `$M` EMPTY, so `grep -r $M/opt` silently greps the HOST cwd — that
is how host `/opt/fastflowlm` masqueraded as image content. Always check `$M`
non-empty, or use `podman unshare bash -c 'M=$(podman mount C); …'`.
- The entrypoint swallows `podman run IMG sh -c …` as a mode name; use
`--entrypoint sh/cat/ls`.
- The proof container inherits production name/port unless the override is set
AFTER sourcing the env (the env file re-exports `FRANK_DRAGOON_CONTAINER`).
- `Restart=always` re-runs the lane into its own GPU/NPU debris; stop the unit
before any manual boot, don't fight the loop.
@@ -0,0 +1,44 @@
---
name: halogen-npu-xrt-third-prefix
description: "Triage a halogen frank-dragoon (flash-server) unit crash-loop caused by the NPU lane: signature \"No such library /opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2\", dead-lane-kills-container law, third-prefix mount fix, and the FRANK_DRAGOON_ENV_FILE scratch-boot proof recipe."
---
# Halogen NPU lane: dead lane kills the container; XRT needs THREE mount prefixes
Friction-earned 2026-10-21 on `frank-dragoon` (Dankclaw): unit boot-looped 88× (`activating`, exit 1 in ~200 ms) with the GPU/serve_api lines looking healthy; chat dead beside a fine-looking ledger.
## Signatures (in order of appearance)
```
halogen-npu: No such library '/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2'
halogen npu: the NPU engine exited before it was ready (rc=1; ...)
```
then `systemctl show <svc> -p ExecMainStatus` = 1, `NRestarts` climbing, and `/home/dank/logs/frank_dragoon.log` truncated at ~10 lines each boot (serve script truncates the log per start).
**Laws:** a dead NPU lane is NOT survivable — the engine supervisor takes the WHOLE container down on it. And `Restart=always` makes each restart race its own GPU/NPU debris: the FIRST failure's message is the only real evidence; later loop entries are noise. Stop the unit immediately (`sudo systemctl stop frank-dragoon`) before probing.
## Root cause class
The NPU child `dlopen`s XRT from `/opt/xilinx/lib/<multiarch>/` IN ADDITION to `/opt/xilinx/xrt/lib` and the install path. Host package churn (dpkg.log reconfigure of `libxrt2` the previous day) made the child demand the third prefix; our mounts filled only two. A lane that worked for days with byte-identical mounts can die purely from host package changes — diff the mounts last, check `grep libxrt /var/log/dpkg.log` first.
Fix: mount each REAL file (never a symlink) at all three prefixes; derive the multiarch name from `FRANK_DRAGOON_XRT_DIR`'s basename (`MULTIARCH="${XRT_DIR##*/}"`).
## Proof recipe (production-shaped, no scratch port)
One card holds one server — never bench on a side port; use the sanctioned env-file seam:
1. Scratch env on the host: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + knob overrides (e.g. `HALOGEN_NPU_MODELS=...`).
2. Copy the serve script with the candidate change applied via `sed` into `/tmp` (or edit + `FRANK_DRAGOON_ENV_FILE`); export `FRANK_DRAGOON_ENV_FILE=<scratch>` and `FRANK_DRAGOON_LOG=/tmp/<name>.log`; run under `timeout 1200`. The seam exists precisely for this.
3. Judge from `/tmp/<name>.log`: `npu_api: <model> ... buckets u0…u9` = lane alive; `No such library` = still broken. Then `curl :7770/health` → `npu.status`.
4. Retire the same session: `podman stop -t 60` (weights unpin slowly), `podman rm -f`, kill wrapper procs BY PID (`pgrep -f` the scratch serve script — `pkill -f` a pattern containing your own ssh line kills the session), move scratch files to `~/.AGENTTRASH/<date>-<topic>/`, then prove `pgrep -f serve-<scratch>` is empty. Leftover scratch state silently hijacks the next deploy (same class as leftover bench drop-ins).
## Traps hit en route
- Rootless `podman mount` fails ("cannot run in rootless mode") and leaves `$M` EMPTY — a later `grep -r "$M/..."` then reads the HOST cwd and "finds" the answer in Loki WAL files. Never grep an unverified mount var; `podman unshare` or `--entrypoint` probe instead.
- Image ships NO `/opt/xilinx` at all: all XRT comes from host mounts; `docs/NPU.md` is not inside the image.
- `ssh host 'tail <log>'` runs on the DEV box if you forgot `ssh` — a missing-file error is not "the engine wrote nothing".
- Health `context` field echoes the RUNNING engine's ceiling — use it to prove which shape is live, unlike `/v1/models` name echo.
## Verification that the fix landed
`systemctl show frank-dragoon -p DropInPaths` empty; boot ledger shows `reserving the KV pool … (about 11.0 GiB)`, `… in all`, an `npu_api:` line, no clamp line; deployer prints `NPU lane verified — <model> embedded`.
@@ -0,0 +1,26 @@
---
name: handdrawn-svg-diagram-verification
description: "Author legible hand-drawn SVG diagrams for zero-JS static pages (theme vars, shared markers, text-width budgets) and verify each by managed-browser screenshot loop that catches label collisions; includes the browser/eval API traps hit doing it."
---
Use when adding diagrams to a static, zero-JS HTML artifact (guide/report) where mermaid/JS is not allowed, or when verifying hand-drawn SVGs render legibly.
## Authoring rules (what makes SVGs legible)
- Theme colors ONLY via the page's CSS vars: define classes once — `svg rect.box {fill:var(--surface);stroke:var(--border)}`, `svg .mut {fill:var(--muted)}`, arrow paths stroke var(--muted)/var(--accent); status text via inline `style="fill: var(--ok|var(--err)|var(--warn))"` equivalents. Never hardcoded hex.
- Markers once, shared: hidden `<svg width="0" height="0" style="position:absolute">` right after `<body>` holding `<marker id="arrM">` (fill var(--muted)) + one per stroke color used; reference via CSS `marker-end: url(#arrM)`. `orient="auto-start-reverse"`.
- Text budget: chars × 0.535 ≈ px width at font-size 10.5 (0.56 at 11, ~0.72 bold 13-14). Keep every label ≤ box width − 16px; split long labels into 2–3 lines rather than shrink font.
- Structure > decoration: dashed `rect.frame` lanes with a 11px bold lane label ("REQUEST TIME / AUTHORING TIME"); one crossing arrow between lanes max, labeled; lifelines = `class="dash arrow"` + `style="marker-end:none"`.
- Collision suspects, pre-check: labels near box corners/edges, crossing curves over lane labels (flatten curve control points to arc OVER label text, or move label beside the curve's x-range, not under its start), two labels in the same y-band.
## Verification loop (catches what eyes miss in source)
1. Serve: `Bun.spawn(["python3","-m","http.server","8899","-d",dir])`, `await Bun.sleep(500)`.
2. `const tab = await browser.open({ url: "http://127.0.0.1:8899/#anchor" })` — options object, NOT url string.
3. Position per diagram: `goto(url#anchor)` then `tab.evaluate("window.scrollBy(0, N)")`; shot each with `tab.screenshot({path})`. Attached images are the ground truth — read them for label overlaps, border grazes, clipped text.
4. Fix coordinates with `edit`, re-shoot ONLY the touched regions. Expect 2 rounds: first shots expose collisions you designed in.
## Eval/browser traps (all personally hit)
- `tab.wait` does not exist → `await Bun.sleep(ms)`.
- `log()` output from JS eval cells is SWALLOWED — return the value from the cell instead (`return JSON.stringify(x)`).
- `tab.evaluate(...)` can return undefined/nothing on managed hidden tabs without erroring — for content assertions prefer `tab.text("svg")` or parse the file with node; screenshots remain trustworthy.
- `browser.open` errors list the correct signature — read it and fix in one retry.
- Screenshot misreads at small font: a label that *looks* like mangled markup (`</id>`) in a downscaled shot is usually fine in source — settle with `tab.text(selector)` returning `[...]`, don't "fix" correct source.
@@ -1,60 +1,62 @@
---
name: harden-precommit-secret-gate
description: "Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer."
description: "Fix or build a pre-commit secret-scan hook that actually blocks: the -z path hole, line-wide placeholder exemptions, inert doc placeholders, glued report lines, self-test fixture paths — each with a red-first bats case."
---
Fix or build a pre-commit secret-scan hook that actually blocks leaks: scope hole, fail-closed, quarantine list, and the self-trip that kills an automated committer. Use when a secret gate is wrong (misses a leak) or too right (rejects an inert example and bricks the heartbeat).
A pre-commit secret gate that fails open is worse than no gate: it teaches the operator that scanning happened. Four defect classes, each with a black-box test that goes red against the broken version (serverAppConfigs PR #70, command-center #59 — 4 holes closed same night, all red first).
## The two failure directions
## 1. Non-ASCII paths fail open
A gate can be **too weak** (misses a real key) or **too strong** (rejects its own documentation and takes down an automated committer). Both are outages; the second is silent.
## Audit recipe for an existing gate
1. Read the hook and list what it actually inspects. The classic scope hole: `git diff --cached -- '*.env' '*.pem'` — an extension whitelist. A credential in `agent/config.yml`, a quadlet unit, or a SKILL.md is then invisible to the gate. The rule that survives: scan **the entire staged diff**, every path, and let only binary content out.
2. Test fail-closed behaviour: can the hook be made to print nothing and still exit 0? Common escapes — running outside a git repo, unreadable staged diff, a hook that pipes through `grep` and inherits `grep`'s exit code from the wrong stage, or `set -e` swallowing the verdict. Verdict must be: unknown ⇒ **block**.
3. Quarantine list, never tracked: `.env`, `auth.json`, `trust.json`, `*.pem`, `*.key`, `identity*` — and the `.gitignore` policy must quarantine env files **at any depth under any name** (`**/.env*`, not `.env`), or `config/prod.env` slips past.
4. Escape hatch must be explicit, per-commit, and loud: `ALLOW_SECRETS=1 git commit …`, and the hook says so when it honours it.
5. Gate the gate: a test that commits a fixture repo through the hook and asserts it blocks. `tests/secret_gate.bats` here covers: shape in an unexpected extension, bcrypt hash, JWT, placeholder/indirection allowed, prose allowed, binary ignored, unreadable diff refused, deletion of a credential never blocked, `ALLOW_SECRETS` override.
## Self-trip: the gate rejects its own example (live 2026-10-21)
Symptom: a systemd timer that commits machine-written files fails **every hour**, and the journal shows the gate's own refusal:
```
SECRET GATE: staged diff contains 1 secret-shaped string(s):
agent/managed-skills/git-secret-gate-hook/SKILL.md
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
```
The offender is the AWS **documentation** example key — entirely inert — and `AKIA[0-9A-Z]{16}` cannot tell it from a leak. So the gate's own advice about writing an inert fake is uncommittable by the gate. Auto-learned skills describe credential shapes for a living, so this recurs by construction, and an automated committer cannot use `ALLOW_SECRETS=1`: that would blind the gate for the *other* 64 staged files.
Diagnose, in this order:
`git diff --cached --name-only` quotes non-ASCII paths (`"docs/\303\251.md"`), so `git diff` on that name returns **empty**, and a loop that `continue`s on an empty diff skips the file silently. Fail closed with NUL delimiting:
```bash
journalctl --user -u live-sync.service -b --no-pager | tail -25 # which file, which line
git -C ~/.omp status --short | head # how much is queued behind it
# then scan the WHOLE staged queue, not just the reported file:
git diff --cached -U0 | awk '/^diff --git/{f=$3;sub("^a/","",f)} /^\+/ && !/^\+\+\+/ {
l=substr($0,2)
if (l ~ /sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|glpat-[A-Za-z0-9_-]{16,}|AKIA[0-9A-Z]{16}|xox[baprs]-[A-Za-z0-9-]{10,}|eyJ[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{10,}\.|-----BEGIN [A-Z ]*PRIVATE KEY-----/)
print f": "substr(l,1,90) }'
git diff --cached -z --name-only | while IFS= read -r -d '' f; do ...
```
The second count is the decision input: one hit ⇒ fix the line; many ⇒ the gate needs an exemption channel before anything lands.
Test: a tracked file whose name contains a non-ASCII character, holding a fake key on an added line, must be blocked.
Fixes, in order of preference:
## 2. Placeholder filters must be value-scoped
1. **Describe the shape instead of printing a matching literal** — `AKIA` followed by zeroes and `EXAMPLE`, spelled out in prose, cannot match. One line, the gate stays strict, the next timer run drains the queue.
2. **Per-line allow marker** honoured by the hook (e.g. a trailing `secret-gate: allow` on the flagged line) — durable, keeps documentation honest, still fail-closed everywhere else. Needs a test both ways: a marked line passes, an unmarked one still blocks. Note the risk honestly: an attacker-adjacent string can self-authorise, so the marker belongs in review-able prose tiers, not in a diff that also touches secrets config.
3. Never widen the regex to skip markdown, never run the automation with the override env var set.
`if echo "$line" | grep -q "$placeholders"` exempts the whole line, so a comment word vouches for the credential beside it:
Inert-fake rule for anything that must *show* a credential shape: RFC 2606 domains, all-zero ids, the word `EXAMPLE` — and if the shape itself trips the gate, describe it in prose rather than print it.
```python
token = "sk-…" # example config for the docs
```
## Prose must not be collaterally blocked
Filter only the captured value (the `sk-…` group), never the line. Two directions to test: `key = "example"` stays allowed; `token = "sk-<real>" # example` is blocked.
Markdown explaining credential handling is the bulk of what a knowledge repo commits. `xox` inside a sentence about Slack token prefixes, or a `sk-` prefix discussed abstractly, must pass; the shapes are anchored long-suffix patterns for exactly that reason. When a gate fires on prose, that is a bug in the gate — but if the prose contains a *literal* matching token, the fix is the prose (option 1 above), not the pattern.
## 3. Inert placeholders in documentation
## Verifying a gate change
A skill/README that *documents* a secret gate quotes the vendor's placeholder ids (`AKIA0000000000000EXAMPLE`, all-zero JWTs). Exempt them by value shape (all-zero / `EXAMPLE` suffix / RFC 2606 domain), or the machine tier stops committing — and the refusal usually lands on a heartbeat commit, where it looks like infrastructure failure, not policy. Also note: an unlanded fix means the checkout runs the **old** hook, so a landing can be blocked by the very bug the PR corrects; audit the staged diff and use the documented override (`ALLOW_SECRETS=1`, never `--no-verify`), recording the audit in the commit message.
Write the fixture repo in a temp dir, commit through the hook, assert block/allow for each case. Do not test the regex by grepping the hook's source for the pattern — a copy of the pattern in a comment is not a gate that runs.
## 4. Report-line assembly
`report+=(" $file: $line")` inside a loop that keeps only the last hit per file glues the next filename onto the previous line when a second hit exists — the report attributes the finding to the wrong file. Test a two-file staged set where both match, and assert each report line names exactly one existing path.
## 5. Self-test fixtures in the hook itself
The hook's own fixtures contain fake keys by necessity, so the gate would refuse any commit touching the hook or its tests (or a docs page quoting a key as documentation). Pin the bypass to specific paths and to added lines only, and print what was exempted:
```bash
case "$f" in .githooks/pre-commit|tests/secret-gate.bats)
echo "secret-gate: self-test fixtures exempt: $f" >&2; continue;; esac
```
## The scan rule
Scan **added content only** — `git diff --cached -U0 | grep '^+[^+]'` — or every commit touching a file with an old key in context gets blocked, and the operator's response to a gate that blocks everything is to override it always.
## Test harness (fixture repo, never the real checkout)
`git init` a temp repo, `git config core.hooksPath .githooks`, copy the hook in, `git add` a payload, then run it with the env overrides the hook honours (point its cache/notify paths at `$TMP` so tests never touch the user's config). Pin, in order:
| Case | Expect |
|---|---|
| fake key on an added line | exit 1, message names the file |
| `key = "example"` | pass |
| real key + `# example` comment | exit 1 |
| non-ASCII filename with a key | exit 1 |
| two files with hits | report names both |
| self-test fixtures | pass, with the exempt line printed |
`bats -r` after each edit. `--no-verify` is forbidden on Frank's boxes: the hook IS the control.
@@ -1,48 +1,49 @@
---
name: html-report-ground-truth-audit
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator."
---
Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.**
## Law
Every numeric claim must be re-measured against the repo, not the narrative that produced it.
## Procedure
### 1. Enumerate every checkable claim
Grep the report for numerics, file paths, symbol names, and code blocks:
- line counts (`wc -l` the real file)
- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
- commit hashes (`git log --oneline`)
- "verbatim" code blocks (see 3)
- diagram node lists and captions (see 4)
1. **Dump claims with file:line anchors.**
`sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt`
Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts.
### 2. Ground numeric claims with one command each
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `&quot;`/`&amp;` entities leaking into code, and `<br>` line breaks that split a line mid-pipe.
### 3. Verbatim code blocks: prove them
For a block labeled verbatim, the exact string must exist:
- live code: `grep -n '<distinctive line>' <file>`
- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it.
### 4. Diagram claims: run the engine
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25).
### 5. Check for swallowed sections
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it.
### 6. Structural balance
Python tag-balance pass over the file:
- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
- running-div-depth scan printing per-section balance to localize stray closers
Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
## Filler test (delete on failure)
### 7. Render proof
`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
- A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git.
- A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.)
- Keep only what a reader cannot rebuild from `git log` + `git show`.
## Deliver
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
## Serve and verify the artifact
**Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch:
```bash
base=http://host:port/reports/x.html
for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done # service may still be activating
for i in 1 2; do curl -fsS "$base" | md5sum; done # stable across refetches
diff <(curl -fsS "$base") /path/to/source.html && echo "served == source"
```
## Voice drift
If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one.
## Report
Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.
@@ -0,0 +1,34 @@
---
name: infra-repo-security-review
description: "Full-repo security review of an infra/config repo (serverAppConfigs-shaped: shell deployers, quadlets, systemd, Caddy edge, bats/Playwright) — slice fan-out, mechanical censuses, and subagent-claim verification."
---
# Whole-repo security review (infra/config repo)
For "full security review of this repo" on a deployment-config repo (shell scripts + podman quadlets + systemd units + Caddy + tests). NOT for a diff review — that is `code-review`. NOT for a secrets-only question — that is `git-repo-secret-audit`.
## 1. Slice, then fan out in ONE `task` batch
Five `security-reviewer` slices, each with the repo's own laws pre-injected in `context` so no agent re-derives them (AGENTS.md policy text + exclusions: docs prose, `.AGENTTRASH/**`, binaries, `package-lock.json`):
1. **Shell** — all `*.sh` + `.githooks/` + `tests/run.sh`: injection, quoting, `mktemp` vs `/tmp/fixed`, `set -e`, `curl -f`, secrets in argv, `rm`/`git clean`/`systemctl disable`.
2. **Python** — every `*.py`: `subprocess(shell=True)`, `verify=False`, unbounded `read()`, path traversal on request-supplied filenames, fixed `/tmp` writes, **which HTTP servers exist and which are unauthenticated**.
3. **Container + systemd posture** — every `*.container`/`.network`/`.service`/Containerfile.
4. **Edge + web** — Caddyfile vhost inventory, Anubis/Authentik chain, response headers, static-asset XSS sinks, service configs (`limiter.toml`, `prometheus.yml`, `alertmanager.yml`).
5. **Tests** — vacuous assertions + credentials/secrets in fixtures + harness safety.
**Set a yield deadline in the prompt.** Subagents here get killed at `task.maxRuntimeMs` (30 min) and the dead ones produced **zero** output. When one runs long, write early: `write agent://<id>` "yield now, unconfirmed items prefixed UNVERIFIED" — the resumed agent returns its partial report instead of losing it. Two of four needed this.
## 2. Verify every citation before reporting
`verify-subagent-review-findings` applies in full: re-grep each cited `file:line` and compare text. Expect ~40% of lines to be off (e.g. an agent described a 13-line hook; the file is 6) and several files to be hallucinated entirely (`.githooks/compose/*.container` did not exist). Discard mismatches; publish a **Discarded subagent claims** section. Re-grep your own citations too — two of mine were off by two lines and had to be corrected in an addendum.
## 3. Mechanical censuses (cheap, high yield — run these yourself)
- **Port binding**: every `^PublishPort=` line, classify `127.0.0.1:` / `192.168.0.x:` / **no address prefix** (= implicit `0.0.0.0`). The unprefixed set is the auth-bypass surface: a LAN client hits the container port directly and skips the Caddy→Anubis→Authentik chain that the vhost enforces. Report as one consolidated finding listing all unprefixed lines, and cite the repo's own correct examples (`127.0.0.1:9090`, `192.168.0.143:8080`) as the fix pattern.
- **Image pinning**: count `^Image=` lines with vs without `@sha256:` (this repo: 4 of 54 pinned) — floating tags + `Pull=never` = unreproducible edge.
- **Secret modes**: grep every `chmod 600` in the deployer and diff against every file it writes from an `.env`; a `>` redirect without the `chmod` is the finding (the Caddy/Porkbun env file).
- **Systemd hardening**: `User=`/`ProtectSystem`/`PrivateTmp`/`NoNewPrivileges` per unit, and `ExecStartPre=+` lines — a `+` hook on a root unit running a checkout-writable script is a local privesc.
- **Vhost table**: vhost → upstream → guarded? → exemption documented? Cross-check each upstream against the quadlet `PublishPort` list; unguarded vhost with no exemption comment is a policy violation even when the service self-authenticates.
- **Hook wiring**: `git config core.hooksPath` + `ls "$(git rev-parse --git-path hooks)"`. A repo can ship `.githooks/pre-commit` that git never executes — and its own test can assert the *file* is executable and still miss that it isn't installed.
- **Secret sweep**: `git cat-file --batch-all-objects --batch` (all history + dangling), match known values read from `~/.config/secrets`; report `prefix…(len N, sha 10)` only, never the value. Watch for the **split-concat disguise** (`PW="as""set"`): their own regex can't see it, but `grep -F` on a fragment finds it and history keeps it forever → rotation required.
## 4. Report
Severity-ranked table (Critical→Low, one line each: location + flaw), then prose detail for Critical/High only, then the Discarded-claims section, then remaining coverage gaps, then an ordered fix set and an offer to branch. Do not create a report file unless asked — chat is the deliverable.
@@ -0,0 +1,32 @@
---
name: iptables-insert-order-drop-shadows-accepts
description: "Diagnose or prevent a service port that times out beside a LISTENing socket because a deployer's iptables -I sequence landed a blanket DROP above its ACCEPTs; includes the sandboxed fake-iptables red/green bats pattern."
---
# iptables `-I` order trap: blanket DROP shadowing ACCEPTs
## Failure signature
- Client to a port gets `curl: (28) Connection timed out` (SYN swallowed), never refused.
- `ss -ltn` shows the socket LISTENing; the process is fresh and healthy.
- `sudo nft list ruleset | grep <port>` (or `iptables -L INPUT --line-numbers -v`) shows the catch-all `drop` rule ABOVE the specific ACCEPTs, with all counters on the drop and 0 on the accepts.
## Root cause pattern
`iptables -I CHAIN ...` inserts at position 1. A deployer block like:
```
-C/-I ACCEPT -s A ; -C/-I ACCEPT -s B ; -C/-I ACCEPT -s C ; -C/-I DROP
```
ends with DROP at position 1 — above every ACCEPT. Worse: `-C` checks existence, NEVER order, so re-running the block "verifies clean" while the chain stays poisoned.
## Repair (converges on poisoned hosts)
1. Delete the blanket DROP wherever it sits: `iptables -D INPUT ... -j DROP 2>/dev/null || true`
2. Keep/insert the ACCEPTs at the head (`-C` before `-I`).
3. APPEND the DROP (`-A`), so it is last.
Delete-before-append keeps exactly one DROP per run (idempotent).
## Behavioral test pattern (bats)
Text greps cannot see order. Fake the kernel: write a small `iptables` shim to `$BATS_TEST_TMPDIR/bin` backed by an ordered file `$FW_STATE` (`-C`=grep -Fxq, `-I`=prepend, `-A`=append, `-D`=delete or exit 1), prepend it to `PATH`, `eval "$(sed -n '/^<install_fn>()/,/^}/p' script.sh)"`, stub `run_privileged(){ "$@"; }`, `systemctl(){ :; }`, `curl(){ return 0; }`, and **seed the state file with the poisoned order** before running. Assert: exactly one DROP, its line number equals `wc -l`, and every ACCEPT line number is smaller. Prove red by running the same scenario against `git show origin/main:<script>`.
## Misc traps hit while doing this
- zsh: `UID=...` is read-only/arith (renames blow up); a glob that matches nothing aborts the ENTIRE `&&`-less command line, silently skipping earlier `rm`s on that line.
- `echo ===` in a remote zsh command parses as `=cmd` expansion — quote sentinel strings.
- OpenWebUI `create_token` needs cwd `/app/backend` AND `WEBUI_SECRET_KEY` from PID 1's `/proc/1/environ` (`podman exec` sessions don't inherit the app process env).
@@ -1,50 +1,82 @@
---
name: land-merged-pr-with-blocked-tree-ops
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks checkout/stash — prove the delta content-free, hand the operator the two-liner, verify via live config meanwhile."
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks git checkout/restore/stash: commit-the-dirt pattern, the secret-gate false positive that blocks the landing commit, conflict resolution proven settings-vs-comments, and heartbeat (live-sync) handoff."
---
Landing a merged PR when the deployment checkout is dirty **and** workspace policy blocks `git checkout`/`git stash`. Use when `git merge --ff-only origin/main` aborts with "Please commit your changes or stash them before you merge" in a SHARED workspace where other agents have work in flight. Earned 2026-10-20 on `dank/command-center` (the checkout IS the deployment).
Landing a merged trunk into a checkout that IS the deployment (dank/command-center `~/.omp`), when tracked files are dirty and policy forbids `checkout`/`restore`/`stash` on shared state. Re-earned 2026-10-21 landing PRs #61 + #63 into `~/.omp` (27 commits behind, two conflicts, one gate refusal).
# Land a merged PR with blocked tree ops
## 1. Classify the dirt before touching it
## Why it blocks
`--ff-only` refuses when **any** incoming path is dirty in the worktree — it checks dirtiness against the *old* HEAD, not whether your content matches the target. Proven: overwriting the file so it is **byte-identical to the target commit** (`cmp` clean against `git show origin/main:<path>`) still aborts with the same message. So "make the file match main" is not a way in, and neither is `git merge` from another worktree (the branch is checked out in the dirty one).
The blocking set is what you must know, exactly:
`git status --porcelain` then compare against what the merge brings in — only overlap blocks the merge:
```bash
comm -12 <(git diff --name-only HEAD..origin/main | LC_ALL=C sort -u) \
<(git status --porcelain | awk '{print $NF}' | LC_ALL=C sort -u)
{ git diff --name-only; git diff --cached --name-only; } | sort -u > /tmp/dirty.txt
git diff --name-only HEAD origin/main | sort -u > /tmp/incoming.txt
comm -12 /tmp/dirty.txt /tmp/incoming.txt # these paths must be clean
```
`LC_ALL=C sort` matters — git's own output order trips `comm: file N is not in sorted order` and you get garbage overlap lists. In the case this was earned on, 12 incoming paths and 57 dirty ones intersected in exactly **one** file.
Note the first column of porcelain: `M ` = **staged**, ` M` = unstaged. A staged file shows nothing in `git diff` — look at `--cached` too, or you will "verify" a file is unchanged while it is staged. `comm` needs both inputs sorted; unsorted input prints "file is not in sorted order" and a partial answer.
## Procedure
## 2. Prove a dirty file content-free before resolving
1. **Fetch, don't merge yet.** List incoming commits (`git log --oneline HEAD..origin/main`) so you can show what is waiting.
2. **Compute the blocking set** with the `comm` above (both inputs `LC_ALL=C sort -u`).
3. **Prove the local delta is content-free**, in the direction that matters — lines the working tree has that `main` does not:
```bash
diff <(git show origin/main:<path>) <path> | grep -c '^>' # local-unique lines
```
Zero means the working tree is a strict subset of what was merged: discarding loses nothing.
4. **Preserve before you touch:** copy the current file to `~/.AGENTTRASH/` (snapshot), and copy anything you had moved into a gitignored `.AGENTTRASH/` inside a build worktree to its permanent home in the deployment — otherwise pruning the worktree destroys the only copy of a quarantined page.
5. **Do NOT work around the policy.** `git checkout -- <path>`, `git stash`, and `git stash push -- <path>` are all blocked in a shared workspace and the block is correct: it protects other agents' in-flight work. Do not reach for `git reset --soft` + `git restore --staged` to slide the pointer past it either — that leaves the index holding staged *reverts* of every incoming path, one careless `git commit -a` away from undoing the merge.
6. **Hand the operator the two-liner** and keep working on what does not depend on it:
```bash
git -C ~/.omp checkout -- <path> && git -C ~/.omp merge --ff-only origin/main
```
Say why it is safe (step 3's number, plus where the snapshots live) and what it unblocks.
`cmp` it against the trunk blob, never against memory:
## Why waiting is not free
```bash
git show origin/main:$f > /tmp/incoming
cmp -s "$f" /tmp/incoming && echo "IDENTICAL to trunk" # commit/discard is content-free
```
`scripts/live-sync.sh` merges with a plain `git merge --no-edit origin/main` (its line ~88), so the hourly heartbeat hits the identical wall and raises its ntfy alert `merge conflict with origin/main — local commit kept; needs you at the keyboard`. Report that consequence: the dirt is not merely untidy, it alerts.
If identical, the dirt is a sync artifact, not user work. Say so in the commit message with the `cmp` result — that is what makes a direct commit on `main` defensible for a prose-tier file.
Also note for later: `git branch -d <merged-branch>` **refuses until the commits reach `main`** — `-d` checks against the current HEAD, not `origin/main` — so local branch deletion goes on the post-landing list, not the pruning list.
## 3. Commit rather than discard (policy-blocked verbs)
## What you CAN do while it waits
With `checkout`/`restore`/`stash` unavailable, the only non-destructive unblock is a commit. Two separate commits, and order matters:
- **Verify the merged behaviour through the live config, not the branch pointer.** Merged prompt/config changes take effect from the deployment's files: spawn a fresh session/subagent and have it read and quote the changed file back. That proves the change is live even while `main` is behind.
- Prune your own build worktrees, but check nothing is running inside them first: `ls -l /proc/*/cwd 2>/dev/null | grep -c "wt/<name>"` (the same guard `live-sync.sh` uses).
- Delete the merged remote branches (`git push origin --delete <branch>`, allowed on the local Gitea).
1. Prose-tier dirt proven byte-identical to trunk — commit alone, message states `cmp` proved no prose change.
2. Machine tiers (`agent/memories/**`, `agent/managed-skills/**`, `agent/config.yml`, `agent/models.yml`, `dotfiles/`) — let the heartbeat do it: `LC_ALL=C scripts/live-sync.sh` stages allowlist only, commits, merges trunk, pushes, and **exits 0 even when it aborts the merge on conflict** (the alert goes to stderr + ntfy). `exit 0` does not mean landed — check `behind=` afterwards.
Do NOT `git add -A`: the index may already hold dozens of staged machine-tier files, and `git add <one file>` does not clear them — they ride along into your commit.
## 4. The gate refusal that blocks the landing commit
The pre-commit secret gate scans the **staged** diff, so a single-file commit still trips on previously staged content. It fires on inert placeholders inside the skill docs that document the placeholder (e.g. `AKIA0000000000000EXAMPLE` — the AWS all-zero example id in `git-secret-gate-hook` / `harden-precommit-secret-gate`). If the checkout is behind, it is running the **pre-fix** hook while the fix sits unlanded in trunk: the landing is blocked by the very bug the PR corrects.
Audit first, then override deliberately — never `--no-verify`:
```bash
git diff --cached | grep -nE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|glpat-[A-Za-z0-9]{16,}|BEGIN [A-Z ]*PRIVATE KEY|eyJ[A-Za-z0-9_-]{20,}'
git diff --cached | grep -oE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}' | sort | uniq -c # what exactly, how many
ALLOW_SECRETS=1 git commit -F msg # message records the audit + that the gate ran
```
Record in the message: the hits are inert placeholders, which files they sit in, that this checkout runs the pre-fix hook, and that the refusal dies with this landing.
## 5. Merge conflicts: resolve by proving settings identical
```bash
git merge --no-commit --no-ff origin/main
git diff --name-only --diff-filter=U
```
For each conflicted file, get all three blobs (`HEAD`, merge base, `origin/main`) and ask what the local side uniquely holds:
- **Prose / doc pages**: diff ours vs trunk; if ours only lacks the incoming trim, take trunk.
- **Real settings files (kdl/yaml/env)**: strip comments and diff — `diff <(grep -v '^\s*//' ours) <(grep -v '^\s*//' theirs)`. Empty output ⇒ only comments differ ⇒ taking trunk changes no deployed behaviour, and you can say that in the merge message. (2026-10-21: `desk-1-home.kdl` — modes byte-identical, trunk's comment corrected the false "no 240 Hz mode exists".)
Resolve without the blocked verbs by writing the trunk blob: `git show origin/main:$f > $f && git add $f`. Then validate the live artifact (`niri validate`, the suite) before committing the merge, and push.
## 6. Verify the push actually landed
`git push` output can be swallowed when wrapped in command substitution inside a batch — verify from the remote, not from the push:
```bash
git fetch -q origin && printf 'HEAD=%s origin/main=%s behind=%s\n' \
"$(git rev-parse --short HEAD)" "$(git rev-parse --short origin/main)" "$(git rev-list --count HEAD..origin/main)"
```
## 7. Close and prune
- Close the issues with a comment giving merge SHA + PR link + title (numbers are per-repo; a bare `#N` is not a reference).
- Prune guards, all three before `git worktree remove --force`: `ls -l /proc/*/cwd | grep -c '<wt-name>'` must be 0; `git -C <wt> status --porcelain | wc -l` must be 0; `git rev-list --count main..<branch>` must be 0. Check for `.AGENTTRASH/**` inside the worktree first — removal destroys the only copy.
- `git branch -d` (not `-D`) once the branch is reached by main; `git push origin --delete <branch>` is allowed on the local Gitea.
- Delete the temp `-K` token config file in the same session that created it.
@@ -0,0 +1,24 @@
---
name: line-edit-anchor-discipline
description: "Avoid the recurring edit-tool line-op traps: empty PUT deletes lines, PUT ranges swallowing multi-line block delimiters, and anchors landing mid-expression. Use whenever issuing CUT/PUT line edits on tagged files."
---
# Line-edit anchor discipline (CUT/PUT ops on tagged files)
Five self-inflicted syntax wounds in one session (franksign, 2026-10-21) all trace to three rules.
## Rules
1. **An empty PUT body DELETES the targeted lines** (tool warns: "Interpreted an empty PUT body as deletion"). Twice it ate a whole log entry / a closing `}`. For no-ops use nothing; for deletes use explicit `CUT`.
2. **Never PUT a range whose endpoints cut through a multi-line block.** If a `| {`-style union variant, a frontmatter `---` fence, or a `test(...)` closer straddles the range boundary, the replacement swallows the opener/closer and strands the body. Pre-condition: line N-1 and M+1 must be block-complete. Fix-after-the-fact pattern: read the broken region raw, then `PUT k.=k:` to restore the single swallowed line (e.g. the missing `---`, the orphaned ` | {`, `}` → `});`).
3. **Search-hit reads do not authorize PUT.** A grep/folded-summary read marks lines as "never displayed"; the edit tool refuses and echoes actual content. That refusal is a free verification — re-check the echoed lines match intent, then straight-retry with the same tag.
## Costly mistakes seen
- Empty `PUT 7-7:` deleted a just-written log entry; restoring required re-issuing the full entry text.
- `PUT 6.=7` replacing `timestamp`+`---` dropped the frontmatter close fence → lint-breaking file silently.
- `PUT 10.=11` on a union type swallowed ` | {` opening a 12-line variant → cascade of LSP "unexpected label" warnings.
- `CUT` of a stranded opener left its dangling `.json())` behind; balance errors surface only at EOF (`expected ')' but file ends`) — locate by reading regions raw, not by widening the payload.
## Discipline
- After any edit that reports a syntax error or a warning: read the touched region with `:raw` before the next op; fix the single wrong line, do not resend the block.
- "no change: byte-identical" means your anchor math is off — re-read, don't append more content.
- Prefer replacing whole coherent blocks (start line = block start, end line = block end) over edits that end mid-statement.
@@ -0,0 +1,53 @@
---
name: live-checkout-landing-chain
description: "Prove whether a merged change is actually deployed in ~/.omp: walk origin/main vs the checked-out main commit vs the worktree, then fast-forward in place — git refuses fetch into a checked-out branch, and a running harness caches its config."
---
## When to use
The user says a merged/committed change is "not there" (`/switch` can't see a model, a doc is missing, a config edit is invisible) while the repo work itself is green. In a repo where **the checkout is the deployment** (`dank/command-center` at `~/.omp`), "merged" and "deployed" are different commits — measure both before believing either claim.
## The ladder — read all four levels, in order, before answering
```bash
cd ~/.omp && git rev-parse HEAD origin/main # 1+2: deployment vs forge
git ls-tree --name-only HEAD # is the file even in HEAD?
git worktree list # 3: which worktree am I in vs main?
```
1. `origin/main` — the forge truth (`gh`/`tea` absent; Gitea API or a fetch).
2. `/home/blank/.omp` HEAD — **what the running agent actually reads.** Merged PR at `3eb1ada` while this sat at `53025a6` is the whole bug.
3. The feature/session worktree — where the edit was authored; its files can be correct while main is stale, which is why the author sees the change and the user does not.
4. The running process — `models.yml` / `config.yml` are read at startup and cached for the process lifetime; an open session needs a restart even after level 2 is correct.
Belt-and-braces when the answer hinges on file contents, not just commit ids: parse the live file (`python3 -c "import yaml; ..."` for `agent/models.yml`, or grep the doc) rather than inferring from `git log`.
## Landing main without touching the tree
`git fetch origin main:main` **fails**: `fatal: refusing to fetch into branch 'refs/heads/main' checked out at '/home/blank/.omp'`. The fetch-into-ref trick only works for branches that are *not* checked out anywhere.
For a checked-out main, prove then fast-forward:
```bash
git merge-base --is-ancestor main origin/main && echo "ff possible"
git merge --ff-only origin/main # no rebase — Frank dislikes history rewrites
```
`--ff-only` is the safe verb on a dirty main checkout: tracked-file commits (the live-sync tier) and runtime dirt (`agent/memories/*/state.json`, `dotfiles/**`) survive untouched, and it cannot rewrite history. Confirm the dirt is still there afterwards — if it vanished, something else clobbered the tree.
## The heartbeat is not instant
`scripts/live-sync.sh` (systemd user timer, hourly) is what normally advances main. Check its cadence rather than assuming it ran:
```bash
systemctl --user status live-sync.timer --no-pager # "Trigger: …" = next beat
systemctl --user status live-sync.service --no-pager # "since …" = last run
```
A merge made minutes after the last beat waits until the next one; fast-forwarding by hand is legitimate (idempotent with the script) and is the answer when the user wants it now. Never `--no-verify`, never force-push.
## Do not confuse with
- `omp-worktree-path-gotchas` — edits landing in the *wrong* worktree (cwd drift), not a stale main.
- `dotfiles-deployment-truth-audit` — classifies which files are tracked vs runtime state.
- `serverappconfigs-vault-page-registration` / `gitea-merge-form-encoded` — the PR mechanics upstream of this question.
@@ -0,0 +1,42 @@
---
name: new-service-deploy-gate-census
description: "When a newly added serverAppConfigs fleet service fails to deploy (\"is not deployed on target\" or \"Unit svc.service not found\"), run the full gate census; use pre-flight when adding any new service."
---
Adding a deployable service to serverAppConfigs touches SIX independent gates. PR #69 (blackbox-exporter, 2026-10-21) shipped with three of them missing and failed three separate deploys before all were found. Run this census pre-flight (new service) or as a debug ladder (deploy failure).
## The six gates, in failure order
| # | Gate | File | Failure signature |
|---|------|------|-------------------|
| 1 | client allow-list `valid_service()` | `scripts/deploy.sh` | `Service 'X' is not deployed on <target>.` (client-side, before anything runs) |
| 2 | remote allow-list `service_is_allowed()` | `scripts/deploy-remote.sh` | same message from the remote runner |
| 3 | dispatch case in `deploy_service()` | `scripts/deploy-remote.sh` | falls to catch-all `deploy_compose_service` which HARD-FAILS on dankclaw (no compose since 2026-09-16): `no quadlet handler` |
| 4 | planner entry + `SERVICE_ORDER` | `scripts/deploy-plan.py` | service silently absent from auto-plans (explicit `--service` still reaches gates 1-3) |
| 5 | quadlet unit validity | `<host>/.../quadlet/X.container` | `Failed to restart X.service: Unit X.service not found` — quadlet refuses to GENERATE a unit without `Image=`; also check every key the unit's comments claim (e.g. `Network=host` declared, not just argued) |
| 6 | health gate in the dispatch tail | `deploy-remote.sh` `wait_for_health` | deploy runs but hangs/fails at health |
## Pre-flight commands
```bash
# gates 1-3 in one shot (replace TARGET/SVC):
grep -c "$TARGET:$SVC" scripts/deploy.sh scripts/deploy-remote.sh # both allow-lists > 0
sed -n '/^deploy_service()/,/^}/p' scripts/deploy-remote.sh | grep "$SVC)" # dispatch case
python3 scripts/deploy-plan.py --target "$TARGET" --paths <the-service-paths> # gate 4
grep '^Image=' <host>/<svc>/quadlet/$SVC.container # gate 5 minimum
```
## Test seams that pin the class (exist post-2026-10-21)
- `tests/deployment_flow.bats` "the client and remote allowlists agree per target" — drift only; blind to a service missing from BOTH.
- `tests/deployment_flow.bats` "every gated <target> service is dispatched" — gate 3 (red-proof: plant a gated, undispatched name in a sandbox copy).
- `tests/observability_quadlet.bats` "every observability unit declares an Image" — gate 5.
## Deploy-execution traps (same tranche)
- `deploy.sh` MUST run from a checkout literally on `main` (branch-name gate) with a clean tree. A detached worktree pointed at the `main` ref fails (branch gate / dirty index). The primary checkout `~/Projects/serverAppConfig` is the deploy host; switching it to `main` needs explicit user OK when workspace policy blocks checkout.
- After merging, delete nothing by `git add -A` blind: a stray nested dir (e.g. a failed `git worktree add` stub) lands as an embedded-repo gitlink in the commit; `git rm --cached <dir> && git commit --amend` to excise, move debris to `~/.AGENTTRASH/`.
## Gitea merge invocation (consumed probe turns twice)
`POST /api/v1/repos/<o>/<r>/pulls/<n>/merge` with **JSON** body `{"Do":"merge"}` (capital D) + `Content-Type: application/json`. Form-encoded `Do=merge` 422s `[Do]: Required` on this Gitea despite older lore. Credential: `~/.config/secrets/gitea` holds `http://user:PW@host%3aport` (percent-encoded colon) — use **Basic** auth with `user:PW`; `Authorization: token` 401s. Never echo the password: build the header inside python from the file.
@@ -1,21 +1,21 @@
---
name: niri-monitor-flap-refresh-triage
description: "Diagnose niri/amdgpu monitor failures on blankpad — panel dropping, wrong refresh rate, black/flickering output — and fix the refresh without reloading the compositor or logging Frank out."
description: "Diagnose niri/amdgpu monitor failures on blankpad — panel dropping, wrong refresh rate, black/flickering output, letterboxed 1080p on an ultrawide — and fix the refresh without reloading the compositor or logging Frank out."
---
# niri monitor triage: flaps, phantom modes, 60 Hz fallback
# niri monitor triage: flaps, phantom modes, 60 Hz fallback, stale kernel mode list
Frank's reaction to a broken display is immediate and loud; the fix must never log him out.
Box: `blankpad` (Cezanne APU, `card1`, s2idle only), compositor `niri --session` as a user unit.
## Step 1 — separate three different diseases (do this before touching anything)
## Step 1 — separate the four diseases (do this before touching anything)
```bash
niri msg outputs # name, mode, position, full mode list per output
for c in /sys/class/drm/card*-*/status; do echo "$(basename $(dirname $c)) $(cat $c)"; done
systemctl --user show niri.service -p ExecMainStartTimestamp -p NRestarts # did niri restart?
journalctl --user -b --no-pager | grep -iE 'disconnecting connector|new connector|Page flip commit failed|DMCUB|falling back to preferred'
journalctl -k -b --no-pager | grep -iE 'amdgpu|PM: suspend exit|\*ERROR\*'
journalctl --user -b --no-pager | grep -iE 'disconnecting connector|new connector|Page flip commit failed|DMCUB|falling back to preferred|could not be found'
journalctl -k -b --no-pager | grep -iE 'amdgpu|PM: suspend exit|\*ERROR\*|enabling link|DP_LANE_COUNT_SET|DP_LINK_BW_SET'
```
Reading it:
@@ -25,16 +25,25 @@ Reading it:
- `configured mode M could not be found, falling back to preferred` → **config asks for a mode the panel does not offer**; the panel silently runs at EDID-preferred (usually 60 Hz). This is the one you can fix instantly.
- Config mtimes older than the symptom + no `niri` restart ⇒ the config did not cause it. Say so explicitly; agents get blamed for display problems they didn't cause.
## Step 2 — the phantom-mode trap (niri fails soft, so it looks like hardware)
## Step 2 — phantom-mode trap AND stale-mode-list trap (niri fails soft, so both look like hardware)
`output { mode "…" }` with a refresh the EDID never advertised does **not** error. niri logs a WARN and drops to `preferred` on every connect → janky 60 Hz ultrawide, blamed on the monitor.
Compare the requested mode against the panel's real list:
Two different lies, told by two different layers. `output { mode "…" }` with an unavailable refresh does **not** error: niri logs a WARN and drops to `preferred` on every connect → janky 60 Hz ultrawide, blamed on the monitor. Check the requested mode against the panel's real list:
```bash
niri msg outputs | sed -n '/<monitor name>/,/^$/p' | grep -E '^\s+[0-9]+'
```
If the kernel list lacks modes the EDID advertises (e.g. 3440x1440 absent, largest mode 1080p), discriminate parser gap vs stale cache — **compare the kernel's EDID blob with sysfs and decode it**:
```bash
command -v edid-decode # parse the live EDID: edid-decode /sys/class/drm/card1-DP-1/edid
modetest -M amdgpu -c 2>/dev/null | awk '/connected[[:space:]]+DP-1/,0' \
| sed -n '/value:/,/DPMS/p' # the 384-byte blob DRM actually holds
```
- Blob good (all extension blocks present, native DTD visible via `edid-decode`) **and** mode list bad → **stale mode list cached from a corrupted AUX/DDC read during a flapped hotplug**. DRM rebuilds modes only on a clean re-connect, and every subsequent flap re-caches garbage. Software cannot force the rebuild — the fix is a monitor **power cycle** (unplug power ~30 s; needs Frank's hands), then re-apply the mode. Do NOT conclude "kernel can't parse this EDID" and do not edit config for it.
- Blob itself degraded → AUX hardware read failure, same physical remedy, worse prognosis (cable/port).
## Step 3 — apply the mode WITHOUT a config reload (the no-logout path)
```bash
@@ -43,7 +52,11 @@ niri msg outputs | grep -A1 '<monitor>' # confirm the new mode
pgrep -c niri # compositor still alive, same PID
```
`niri msg output …` is documented as temporary ("not saved into the config file") — it changes one output and cannot log the session out. Avoid `niri action reload-config` when Frank is working: it reloads his whole config.
`niri msg output …` is documented as temporary ("not saved into the config file") — it changes one output and cannot log the session out. **`rc=0` proves nothing**: for a mode outside the kernel's published list niri exits 0 and silently no-ops — always re-read `niri msg outputs` to confirm. Avoid `niri action reload-config` when Frank is working: it reloads his whole config.
## Step 3.5 — flap census beyond the display: who else dies at the same second
Correlate DP flap timestamps against `journalctl -k | grep -E 'usb .*: (new |disconnect)|Synchronize Cache'`. A USB hub chain re-enumerating in lockstep with a DP flap means shared power/ground on the monitor or dock — and it identifies what else brown-outs (keyboard, webcam, USB SSD). `KT USB Audio` (idVendor 12d1 KTMicro) in the tree = the monitor's built-in DAC, i.e. the monitor's USB upstream port is in that hub chain. Kernel `enabling link 2 failed: 15` across *multiple boots* + `core_link_write_dpcd (DP_LANE_COUNT_SET/DP_LINK_BW_SET) failed` = physical layer (cable/dock/monitor PSU), not compositor, not config: prescribe cable swap / other input after a clean-boot power cycle, and say plainly which of the four diseases you did NOT fix.
## Step 4 — persist, and check you edited the file that is actually live
@@ -61,4 +74,4 @@ Desk config law (its own header): panels keyed by **EDID identity**, not connect
## Escalate honestly
Reboot clears DMCUB/EINVAL state → schedule it, don't take it. Persistent drops after a reboot → cable / dock / monitor DP-version setting, i.e. hardware, not config. Report which of the three diseases you found and which you did **not** fix.
Reboot clears DMCUB/EINVAL state → schedule it, don't take it. Persistent drops after a reboot → cable / dock / monitor DP-version setting, i.e. hardware, not config. Report which of the four diseases you found and which you did **not** fix.
@@ -0,0 +1,25 @@
---
name: okf-small-bundle-docs-audit
description: "Audit + dedupe a small repo's docs bundle for OKF conformance when the kb-lint/kb-coverage toolchain does not exist: inline conformance and link-audit scripts, owner-per-fact cuts, index trio, one log."
---
# OKF docs audit for a small repo bundle (no kb-lint toolchain)
Use for a repo whose KB is one docs directory (e.g. `dankvoice` → `dankvoice-docs/`), where `scripts/kb-lint.sh` / `kb-coverage.py` do not exist. For the big `dank/command-center` campaign see skill `docs-dedupe-verify-campaign`; same frame, no scouts — a corpus under ~1500 lines gets read fully yourself (scouts lie).
## Frame
- OKF spec lives at `~/.omp/omp-docs/references/okf-spec.md` (the command-center deploy checkout). v0.1 conformance = every non-reserved `.md` has parseable frontmatter with non-empty `type`; `index.md` = listing with NO frontmatter; `log.md` reserved, exactly one, bundle root.
- Branch off `origin/main` in a fresh worktree; docs-only + src-comment changes still need the repo's normal gate.
## Procedure
1. Inventory `wc -l` every page; read all fully. Ledger per fact: which page OWNS it (laws page owns laws, decisions own rationale, contributing owns process, README owns user-facing, GLOSSARY owns terms). Everything else becomes a pointer.
2. Hunt the rot classes that repeat: dead test citations (name in docs not in `grep -c tests/` — verify with `git log --all -S`), hand-maintained censuses (test counts, scenario lists) — replace with a pointer to the authoritative source, never refresh the copy; deleted doc lines for live env vars (compare `git show <fixed>:file | grep -c VAR` vs HEAD); sections contradicting the page's own head; open items already resolved in live state (check the live settings file, not the doc).
3. Frontmatter: `type` + one-line `description` + `tags` + `timestamp`. On edit-rejection the tool prints the file's current `#TAG` — reissue with it, no re-read needed.
4. Conformance check (inline python): walk bundle dirs, regex `^---\n(.*?)\n---` then `^type: \S` on each non-reserved `.md`.
5. Link audit (inline python): per md file, `\]\(([^)#]+)`; skip http; `os.path.normpath(join(dirname, target))` must exist. Report `N checked / ALL RESOLVE`.
6. Create `index.md` at bundle root + each subdir (listing, no frontmatter, relative links) and `log.md` at the bundle root seeded with laws/incidents only.
## Gotchas
- Src docstring fixes are artifact-free only if the bundler strips comments — prove with `bun run build; cmp dist/... old` before claiming no rebuild needed.
- README must NOT enumerate design records; point at `decisions/index.md` so new records need no README edit.
- Mark superseded decision records in their frontmatter `description` ("its X design is superseded by the Y record") — never rewrite history bodies.
@@ -1,14 +1,15 @@
---
name: omp-models-yml-runner-api-placement
description: "Add or edit a non-chat (tts/image/embedding) model row in ~/.omp/agent/models.yml, wire the speech role, and diagnose a failing speech device without blaming a healthy endpoint"
description: "Add or edit a row in ~/.omp/agent/models.yml — non-chat (tts/image/embedding) runners and authenticated/remote providers with command-backed apiKey values — wire the speech role, verify with a poisoned scratch HOME, and diagnose a failing speech device without blaming a healthy endpoint"
---
# Scope
`~/.omp/agent/models.yml` (or `models.yaml`) on a workstation: adding a **runner**
model row (TTS / image / embedding / rerank / video) alongside chat providers,
flipping `config.yml` `modelRoles` at it, and diagnosing a speech device that
refuses to speak.
adding an **authenticated or off-network** chat provider without committing its
secret, flipping `config.yml` `modelRoles` at it, and diagnosing a speech device
that refuses to speak.
# The landmine
@@ -51,6 +52,46 @@ which the runner transport never reads).
Roles live in `config.yml` (`modelRoles`), not in `models.yml`. Kind roles
(`image`, `web`, `speech`, `dictation`, `judge`) take no `:thinking` suffix.
# Authenticated / remote providers
Provider schema (`src/config/models-config-schema-bundle.ts`): `baseUrl`,
`apiKey`, `api`, `headers`, `compat`, `authHeader`, `auth`
(`"apiKey" | "none" | "oauth"`), `discovery`, `models`, `modelOverrides`,
`transport`. Defining `models` without `apiKey` and without `auth: none|oauth`
is a validation error, so an authenticated endpoint needs a key *shape* even if
the value comes from outside the file.
**Never put the literal secret in `models.yml`.** `apiKey` (and any `headers`
value) is resolved by `src/config/resolve-config-value.ts`:
- leading `!` → command-backed: `/bin/sh -c "<rest>"`, cwd = project dir,
10 s timeout, stdout **trimmed**, result cached for the process lifetime
(failure backs off 30 s);
- otherwise the string is looked up as an **exact env-var name**; the literal is
the fallback only when no such env key exists.
So the secret lives in a 600 file, and the tracked file stays clean:
```yaml
apiKey: "!cat ~/.config/secrets/openwebui_api_key" # QUOTE it: an unquoted
# leading ! is a YAML tag
```
Laws: a literal `sk-…` in this repo trips the pre-commit shape gate *and* lands
the key in every agent context on every box that pulls; rotation means rewrite
the secret file **then restart the harness** (the `!cmd` value is process-cached,
so a live session keeps serving the old key and a stale session reads exactly
like a revoked one); every workstation needs its own copy of the secret file.
**A row behind a proxy copies its direct-connected twin's `compat` block**, it
does not simplify it. Measured through an OpenAI-compatible gateway on
2026-10-21: SSE forwards `reasoning_content` and `tool_calls` deltas plus final
`usage` under `stream_options.include_usage`; `image_url` content parts reach
the vision tower; `chat_template_kwargs` is forwarded (same prompt:
`enable_thinking` false → 0 reasoning tokens, true → a real block). Dropping
kwargs/effort maps because "it is only a proxy" silently changes thinking
behaviour on that path. Check per gateway, never assume either direction.
# Verify after every edit (never assume the file loaded)
```bash
@@ -62,6 +103,25 @@ omp models | grep -c 'validation failed' # must be 0
A run that lists only built-ins means your row broke the file — revert before
yielding.
**Poisoned-scratch-HOME proof** for a remote/credential row (the only test that
shows a success came through the new path and not from a LAN endpoint):
```bash
mkdir -p /tmp/ompx/.omp/agent /tmp/ompx/.config
cp agent/models.yml agent/config.yml /tmp/ompx/.omp/agent/
ln -s ~/.config/secrets /tmp/ompx/.config/secrets # REQUIRED: ~ in a !cmd value
# expands to the scratch $HOME
sed -i 's#http://192\.168\.0\.[0-9]*:[0-9]*/v1#http://127.0.0.1:9/v1#g' /tmp/ompx/.omp/agent/models.yml
HOME=/tmp/ompx omp --model '<provider>/<id>:off' --tools bash --no-session --no-skills \
--no-rules --no-extensions --no-pty --no-lsp --no-title --cwd /tmp -p '<ask for a real tool call>'
```
Then `rm -rf /tmp/ompx` — that tree holds endpoint URLs. Cross-check the server
side before believing the client's echo (gateway access log / container log line
for the request you just made), and confirm the change never hit the live
checkout until you mean it to: `git -C <live checkout> status --porcelain agent/models.yml`
plus a grep for the new provider name.
# Speech transport contract (`openai-speech`)
Body is `{model, input, response_format, voice?, speed?}`; `voice` is forwarded
@@ -72,13 +132,17 @@ error, not a fallback — and `auth: none` makes a dead endpoint read as
*available*, so point `modelRoles.speech` at such a provider only after it has
answered a real synthesis.
A gateway's own `/api/v1/audio/speech` route is NOT an equivalent: OpenWebUI
substitutes its own voice enum before forwarding, so a cloned pack name comes
back 400 from the upstream. Keep `modelRoles.speech` on the direct endpoint.
# Three speech-role laws (bundle-read + measured 2026-10-18)
1. **No setting carries a voice for `openai-speech`.** The transport builds the
body with `voice` from the call's `voice_id` and nothing else.
`tts.localVoice` is the *local Kokoro backend's* key; `speech.voice` is the
TUI's **speak-aloud** voice, an enum of Kokoro names that never reaches an
HTTP request (setting it to a pack name breaks the vocalizer and does nothing
`tts.localVoice` is the *local Kokoro backend's* key; `speech.voice` is the TUI's
**speak-aloud** voice, an enum of Kokoro names that never reaches an HTTP
request (setting it to a pack name breaks the vocalizer and does nothing
for synthesis — read `cfgSpeechVoice` in `dist/cli.js` before believing
otherwise). So a voice-less call is the harness's **normal** shape: a
voice-required endpoint needs a server-side default (fleet answer:
@@ -0,0 +1,69 @@
---
name: omp-plugin-modal-bats-harness
description: "Drive a real omp-extension settings modal headlessly from bats (ui.custom stub, DV_KEYS key-replay) and ship the bundle with host pi packages external."
---
# Headless modal testing + host-external bundling for omp plugins
Friction-earned 2026-10-21 on `dank/dankvoice` PR #31 (select menu → vendored pi-common settings modal). Three failed driver iterations and one 43 MB bundle before green.
## The build contract (do this first)
A modal imports runtime values (`SelectList`, `matchesKey`, `getSelectListTheme`) from `@oh-my-pi/pi-tui` / `@oh-my-pi/pi-coding-agent`. Bundling the root barrels = 43 MB and trips `--target node` on bun builtins. The plugin host IS bun and hoists every pi package into `~/.omp/plugins/node_modules/@oh-my-pi/` — the same ground pi-common stands on. So:
```
bun build src/index.ts --outdir dist --target bun --format esm \
--external @oh-my-pi/pi-coding-agent --external @oh-my-pi/pi-tui \
--external omp-legacy-pi-modules
```
- `omp-legacy-pi-modules` is a harness-internal virtual module behind a lazy dynamic import in the pi-coding-agent barrel; externalizing is safe, omitting it fails the build.
- Declare the externals in `peerDependencies`.
- The packed-artifact load test must reproduce the hoisted host layout: extract the tarball, then `ln -s <repo>/node_modules/@oh-my-pi <work>/node_modules/@oh-my-pi` before importing the entry — a bare temp dir has nothing to resolve against and the test fails for the wrong reason.
## The driver stub for `ctx.ui.custom`
The modal mounts through `ctx.ui.custom(factory, opts)`; factory signature is `(tui, theme, keybindings, done)`. The bats driver stub:
```ts
custom: (factory: any) => {
customCall += 1;
let settleDone: () => void = () => {};
const closed = new Promise<void>((r) => { settleDone = r; });
// theme: Proxy answering EVERY style method; pass through the LAST arg —
// style calls are (colorName, text); returning args[0] prints color names,
// and a fixed {bg,fg,inverse} trio crashes the frame's theme.bold.
const style = (...a: unknown[]) => String(a[a.length - 1] ?? "");
const theme = new Proxy({} as Record<string, typeof style>, { get: () => style });
const component = factory(
{ requestRender: () => {}, terminal: { rows: 60, cols: 100 } }, // rows feeds list pagination; omit → NaN, nothing scrolls
theme, {}, () => settleDone(),
);
const census = () => { for (const l of component.render?.(100) ?? []) console.log(`ROW ${l}`); };
census();
// keys from DV_KEYS / DV_KEYS<n> per call: up|down|enter|tab|esc|ctrl+u named,
// name*N repeat, anything else typed.
(async () => { /* for each token: beats below; await ~140ms after enter (async
roster fetch), ~20ms otherwise; final ~150ms; settleDone(); */ })();
return closed;
},
```
## Key-replay discipline (each cost a red iteration)
- **Never zero-interval.** A burst of `\x1b[B` bytes in one tick coalesces in SelectList — `down*3` lands one step. Beat ~25 ms between repeats.
- **Type one character at a time.** A whole sentence written to the inline editor in one `handleInput` is swallowed (paste-shaped); iterate `[...text]` with ~10 ms beats.
- **Sleep after every `enter`**, not just fixed small delays: Enter opens submenus whose list contents arrive from an async fetch.
- Row math must be pinned: row order in `settingsFields` is the UI contract; tests reach row N with `down*N`.
## Vendored-modal semantics that bite
- Enum fields with `options.length <= cycleThreshold` (default) CYCLE inline on Enter instead of opening the picker — a curated list must set `cycleThreshold: 0` or an "open picker, down, enter" choreography silently cycles and the stray Enter activates the next row.
- Submenus render inside the body component's `render()` output — one census sees both.
- SelectList initial highlight = current value (`items.findIndex(value)`); choreography counts from there.
## Test layering that held
Component-level pins (field census, commits, audition) against fake deps with bun test; behavior bats drives the REAL bundle through the stub for wiring laws (persistence, byDir, notify strings). Don't duplicate choreographies across layers.
Related managed skills: `gitea-pr-api-cycle` (landing), `plugin-git-install-committed-artifact` (install validation).
@@ -0,0 +1,86 @@
---
name: omp-provider-row-sources
description: "Add a provider/model row to ~/.omp/agent/models.yml by reading the harness's own catalog tables (provider name, api, window, cost, compat) instead of guessing or web-searching, then prove it with a scratch-HOME call."
---
# Scope
Adding (or correcting) a provider/model row in `~/.omp/agent/models.yml` for a
provider the harness may already know: an upstream vendor API (DeepSeek, OpenAI,
Anthropic, Moonshot, …) or a new local endpoint. The rule this skill exists for:
**almost every field you are about to research on the web already exists in the
installed bundle** — research there first, and the web only confirms it.
Canonical install root (resolve the symlink first if the repo lives under a
worktree): `/home/blank/.bun/install/global/node_modules/@oh-my-pi/`.
# The three tables that own the facts
| You want | Read | Why |
|---|---|---|
| The legal **provider name**, its URL marker, and which quirks the harness gates on it | `pi-catalog/src/hosts.ts` (minified in `dist/cli.js` as the `VIn` map; consumed by `pi-catalog/src/compat/resolve.ts`) | The provider key is a **lookup key, not a display name**. e.g. `deepseekDirect: { providers: ["deepseek"], urlMarkers: ["api.deepseek.com"] }`, `anthropic: { providers: ["anthropic"], urlMarkers: ["api.anthropic.com"] }`. A label matching neither loses the behaviour set silently (the `max_tokens` field name, the thinking `extraBody`, the thinking-disable mode). |
| Model rows: ids, `contextWindow`, `maxTokens`, `cost` rates, per-model `compat` | `pi-catalog/src/provider-models/<api>.ts` — `openai-compat.ts`, `anthropic-messages.ts`, …; factory helpers `openAiCompletionsDescriptor(...)` / `anthropicMessagesDescriptor(...)` | Rows are fetched from the provider's models-dev key, so the bundle carries the same numbers the vendor publishes (and often rates the vendor page hides). The descriptor's `compat:` block is the authored truth for that provider's transport shape — copy it, never simplify it. |
| The **schema** itself (which keys are legal on a provider vs a model, and which are required) | `pi-coding-agent/src/config/models-config-schema-bundle.ts` (and the identical `dist/` copy) | Provider level and model level have *different* field sets. Cheaper and more correct than reading the minified bundle. |
Also in `pi-coding-agent/src/config/`: `model-registry.ts` (how `models.yml`
merges onto the built-in catalog — `#mergeCustomModels` → `mergeByModelKey`),
`resolve-config-value.ts` (how `apiKey` values resolve).
# Two traps this table pays for
- **Provider-level `api` only accepts chat/judge transports.** Naming a runner API
(`openai-speech`, `openai-images`, `openai-embeddings`) at provider level fails
whole-file validation and disables *every* custom provider, not just yours
(`omp models` prints "models.yml validation failed", lists only built-ins).
Runner api belongs on the **model** entry; see the managed skill
`omp-models-yml-runner-api-placement`.
- **A `models.yml` entry is an overlay, not a replacement.** Adding one model to a
provider the catalog already ships does not remove the provider's other rows —
`omp models <provider>` will show yours (yours wins on shared fields, including
`cost` and `contextWindow`) plus the catalog rows. Same-id rows patch with the
`replace` transport policy; a row that omits `contextWindow` inherits the
bundled row's window and its policy tiers.
# Secret handling (non-prose tracked file)
Never inline the credential. `apiKey` resolution order is: leading `!` → command
(`stdout` trimmed, cached for the process lifetime) → exact env-var name → literal.
So write `apiKey: "!cat ~/.config/secrets/<name>_api_key"` with mode 600.
- A bare `sk-…` trips the pre-commit **shape** gate (every file type).
- An unquoted `!` is a YAML tag — quote it.
- A quoted command value also dodges the *assignment* gate: the gate's value
charset rejects `~` and spaces, so only a literal key matches.
# Procedure
1. Read the three tables above for the provider you are adding. Take the exact
provider name, base URL, api, and compat from them; take `contextWindow` /
`maxTokens` / `cost` from the model row and cite the source in a comment.
2. Write the secret to `~/.config/secrets/<name>_api_key`, `chmod 600`.
3. Edit `models.yml`; for a repo-tracked config, follow that repo's own
branch/log/index rules (in `dank/command-center`: branch → issue → PR).
4. Verify with a **scratch HOME**, because `models.yml` loads at process start
and a live session keeps the old registry:
```bash
mkdir -p /tmp/ompx/.omp/agent /tmp/ompx/.config
cp agent/models.yml agent/config.yml /tmp/ompx/.omp/agent/
ln -sfn ~/.config/secrets /tmp/ompx/.config/secrets # ~ in a !cmd value = $HOME
HOME=/tmp/ompx omp models --no-extensions | grep -c 'validation failed' # must be 0
HOME=/tmp/ompx omp --model '<provider>/<id>:off' --tools bash --no-session \
--no-skills --no-rules --no-extensions --no-pty --no-lsp --no-title \
--cwd /tmp -p 'Reply with exactly the word OK and nothing else.'
```
Then the **negative control**, the step that separates a real auth success from
a cached/built-in answer: `sed` the `baseUrl` to a dead port in the scratch copy
and repeat — it must hang or error, not answer. Then `rm -rf /tmp/ompx` (it
holds endpoint URLs).
# Voice notes
- Prefer the bundled schema file over grepping `dist/cli.js`; grep the bundle only
when the source and the bundle disagree, or for a runtime default the schema does
not state (e.g. `reasoningDisableMode` per provider family).
- `cost` is optional in the schema. Omitting it makes the status line report `$0`
forever; one rate per direction is all the schema models, so a peak/off-peak
vendor table can only carry the peak value.
@@ -0,0 +1,91 @@
---
name: quant-pipeline-adversarial-review
description: "Review or audit the serverAppConfigs quantization campaign pipeline (convert-calib-kldref-score plus corpus builder/rechunker/pusher, fp8 stacker) for defects that silently waste GPU-hours or produce a number that is not the claimed number."
---
# Adversarial review of the quant campaign pipeline
Friction-earned 2026-10-29 across `scripts/convert-qwen38-w5nl64.sh`,
`run-qwen38-bf16.sh`, `make-calib-corpus.py`, `rechunk-kld-corpus.py`,
`calib-push.py`, `stack-qwen38-official-fp8.py`. Every class below was a real
defect in that tree. Review code, not vibes: quote the line, name the cost.
## Files in scope (one campaign = this set)
| Stage | File | Owns |
|---|---|---|
| corpus | `make-calib-corpus.py` | calib + kld JSONL, manifest |
| corpus | `rechunk-kld-corpus.py` | windows that fit the container's KV window |
| convert | `convert-<model>.sh` | plan → artifact per arm |
| tape | `run-qwen38-bf16.sh` | the rig's argv, mount, trap, gates |
| drive | `calib-push.py` | prefill traffic into the tap |
| fp8 path | `stack-qwen38-official-fp8.py` | expert stacking + PLE merge |
## Defect classes to hunt, in cost order
1. **Claimed identity that is only a length check.** A tool that writes TEXT back
out (`tok.decode(window)`) and asserts only counts is not proving token
identity — decode→encode can merge/split. Fix: re-encode each window and
compare id lists; refuse on mismatch. Same shape: an assertion on "N tokens
in, N tokens out" that counts windows rather than re-encoded text.
2. **A loop that cannot finish and cannot notice.** Drivers that re-push a corpus
`--max-rounds` times without measuring the sink. Compare the tap/output dir's
byte total per round and abort on zero growth; abort on zero successes; abort
on zero prompts. Without this the loss is silent for hours.
3. **`set -u` argument holes.** `--flag) VAR="$2"; shift 2` dies with "unbound
variable" when the flag is last. Guard `[ "$#" -ge 2 ]` in every such arm.
4. **Restore paths that leave the GPU holder alive.** `--rm` only fires when
podman itself exits, so a killed run leaves conmon holding the cards while
`systemctl is-active` says `active`. `restore()` MUST `podman rm -f "$C"`
before starting production. Prove with `podman ps` after a `kill`.
5. **Gates that run after the expensive phase.** A leftover container name, an
existing output, a non-writable tap dir, an unset figure — check them before
the plan/boot, not when podman/sed fails mid-run.
6. **Zero-exit with no artifact.** After `rc=0`, assert the output file exists
before `stat`/arithmetic on it.
7. **`assert` in a verifier.** `python -O` deletes it silently. Use `raise`.
8. **Warning where the code's own assumption breaks.** e.g. a 128-lane pack fed
an HD that is not a multiple of 128 — warning + continue writes a wrong table.
9. **Re-tokenization in loops.** A pool cycled 40× re-tokenizing each fragment
per pass; and a manifest recomputing a total by re-encoding the whole corpus.
Cache counts once, accumulate during generation.
10. **Substring tests standing in for key tests.** `"content" in line` before
`json.loads(line)["content"]` crashes on any list-shaped `.json`.
11. **Path guards compared as strings.** `A/../A/x` defeats an equality check;
add `realpath -m` on both sides.
12. **Shellcheck's honest false positive**: single-quoted `$CALIB` passed to a
converter that expands it from the container env is correct (SC2016). Say so
in the PR; do not "fix" it.
## Procedure
1. Inventory: `wc -l` the scripts + recipes; read each **completely** (ranges,
not whole files, but every line). Recipes carry the numbers scripts assert.
2. Read the tests that pin the argv (`tests/qwen38_reference.bats`'s `SHAPE`
fixture, `tests/qwen38_campaign.bats`) — they are the contract you must not
break, and the first place a stale number hides.
3. Mechanical gates:
```bash
python3 -m py_compile scripts/*.py
bash -n scripts/<each>.sh
shellcheck -S warning scripts/<each>.sh # never pass .py to it
./tests/run.sh
```
4. Fix the real ones only; keep diffs surgical; re-run the suite after the batch.
5. Smoke the changed path for real — smoke beats the suite:
```bash
# run a local script on the server's data via the corpus venv, no checkout needed
ssh dankserver '/home/dank/venvs/radcorpus/bin/python - --corpus <corpus> --out /tmp/<x> --force' \
< scripts/rechunk-kld-corpus.py
```
Import-level helpers can be exercised with
`importlib.util.spec_from_file_location` (e.g. a new `tap_bytes()`: empty→0,
one file→its size, absent→0).
6. Stacked PRs: a review commit goes on its own branch off the open one; a diff
that includes the parent commit is a stacked PR, not a mistake — say which PR
merges first in the body.
## Report
One table of defect → cost → fix. Name the failure the defect produced in
practice (hours wasted, wrong number), because that is what makes it reviewable.
@@ -0,0 +1,72 @@
---
name: qwen38-chain-rerun-dankserver
description: "Rerun the qwen38-w5nl64 campaign on dankserver: re-chunk the KLD corpus to the real KV pool, verify the rig's gate with --dry-run, deploy, and relaunch run-chained.sh detached — plus the three state-reading traps (root vs rootless podman, unit failed during an outage, the .rc sentinel)."
---
# Qwen38 campaign rerun on dankserver
Use when rerunning the `qwen38-w5nl64` campaign chain (kldref -> calib -> retire the reference `.rad`) on Dankserver, or when the chain died with `E kld: document N is <n> tokens and the KV pool holds <pool>` and needs its corpus fixed.
Order is fixed: **corpus -> gate -> deploy -> chain.** Skipping the gate burns the outage window.
## 1. Corpus must fit the pool, not the flag
The rig's `--max-model-len` is decorative: the recorder refuses any document longer than the **KV pool the boot actually got** (printed by the boot ledger, e.g. `7,436`), so a corpus authored at a wider window scores nothing.
```bash
ssh dankserver '/home/dank/venvs/radcorpus/bin/python \
/home/dank/campaign/qwen38-w5nl64/rechunk-kld-corpus.py \
--corpus /home/dank/corpora/out/kld.jsonl \
--out /home/dank/corpora/out/kld-7168.jsonl'
```
Read the summary, don't trust the run: `tokens_in` must equal `tokens_out` (that equality is the whole invariant — the tool exists to prove it), `longest_document` must equal the ceiling, and `scored_positions` grows because every window now carries its own `score_from`.
Chars-per-token is payload-specific: this file is chars/3.6, another corpus measured chars/4.97. Never size a window from chars alone.
## 2. Gate the rig before spending the outage
`scripts/run-qwen38-bf16.sh kldref --dry-run` runs the corpus gate and prints the argv **without touching the production unit**:
```bash
ssh dankserver 'cd /home/dank/serverAppConfigs && \
FRANK_QWEN38_KLD_CORPUS=/home/dank/corpora/out/kld-7168.jsonl \
bash scripts/run-qwen38-bf16.sh kldref --dry-run'
```
Check the printed line `corpus fits: longest record <chars> ... inside the <window>` **and** that the `-v ...:/data/kld.jsonl:ro` mount names the file you intended — the rig mounts whatever `FRANK_QWEN38_KLD_CORPUS` points at, and its default is the old file.
## 3. Deploy from a clean MAIN checkout, not a feature worktree
`deploy.sh` refuses off-main (`FAIL: deploy.sh must run from the main branch.`). Run it from the local main checkout and name the service — a bare target is a fleet-wide drain:
```bash
cd ~/Projects/serverAppConfig && ./scripts/deploy.sh dankserver --service frank-archon
```
If the remote pull aborts with `Your local changes ... would be overwritten by merge`, the remote copy is a committed blob written directly on the host. Prove it before discarding:
```bash
git -C /home/dank/serverAppConfigs hash-object <file> # working copy
git -C /home/dank/serverAppConfigs rev-parse <sha>:<file> # candidate commit
```
Equal -> `git checkout -- <file>` is lossless (the blob is in the object store), then re-deploy. Unequal -> archive to `.AGENTTRASH/<date>-<topic>/` first, then port into the repo.
## 4. Launch the chain detached, record the pid
```bash
ssh dankserver 'cd /home/dank/campaign/qwen38-w5nl64 && \
nohup bash run-chained.sh >> run-chained.log 2>&1 < /dev/null & \
echo "chain_pid=$!" > run-chained.pid'
```
- `run-chained.sh` is a server-side campaign artifact (not tracked in the repo); it chains the phases and must export `FRANK_QWEN38_KLD_CORPUS` for the re-chunked file.
- Move a prior `kld-ref/` into `.AGENTTRASH/<date>-qwen38-kldref-attemptN/` first, or the new recording's artefacts mix with the old ones.
- The chain writes `/home/dank/logs/qwen38-bf16-run.rc` as its completion sentinel; watch the pid, don't poll the log.
## 5. Reading state during the outage (three traps)
- **Container storage is split**: `frank-archon` is a root unit -> its container appears only under `sudo podman ps`; the media stack (romm, copyparty, exporters) is rootless -> only plain `podman ps`. Each view legitimately shows "nothing" for the other half. `pgrep -af radiance` settles which runtime is live.
- **`frank-archon` reads `failed` while the rig holds the card.** The rig stops the unit; the container exits status=1 on stop, so systemd marks `failed` until the phase restores it. Confirm with `journalctl -u frank-archon -n 6` — `Stopping...` then `Failed with result 'exit-code'` means the outage, not a crash. `systemctl show -p ActiveEnterTimestamp` dates it.
- **Engine stdout lives in `/home/dank/logs/`, not the journal** (for halogen); for radiance read the chain log.
@@ -0,0 +1,49 @@
---
name: radiance-calib-tap-produce-latest
description: "Produce the radiance calibration tap (layer Hessians for GPTQ calib=) on released radiance: which image actually arms it, the only shape that fits 2x R9700, the floor, the seal contract, and the two silent traps. Use when a w4-MoE quant needs a tap, not when merely verifying a build writes."
---
Friction-earned 2026-10-30 on dankserver (2x R9700 32 GiB, qwen3.8-flash-next bf16 container, campaign `/home/dank/campaign/qwen38-w5nl64`).
## The image decides whether the tap exists
`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS` arm the tap in the **source** (`mc.calib = calib_dir() != nullptr`, identical in v1.8.1 and v1.14.0 trees), but a boot either prints
```
I[0] recorded passes: off, the program accumulates calibration Grams (gram_accum)
```
or it does not — and if it does not, the tap writes **zero bytes for the whole run** while `/health`, `/stats` and the pusher all look healthy. Measured: production pin 1.8.1 = 17 hours, 0 bytes; `:latest` (v1.14.0, revision `4a61d7296bc5`, digest `69add58f7de1…`) on the same argv/env = sealed in minutes.
**Procedure:** boot the candidate image with the calib env and grep its log for `gram_accum` **before** pushing anything. Absence = wrong build; do not spend a window discovering it.
## The vendor calibration shape does not fit two 32 GiB cards
`docs/MOE-W4.md` prescribes `--max-num-batched-tokens 65536 --max-num-seqs 64` (~930 tok/s). On 2×32 GiB R9700s (27.95 GiB claimable) it dies in the VRAM resolver — arena alone 12.83 GiB; variants came up short by 3.19 GiB, 279 MiB, 59 MiB, 86 MiB, then a starved cache. Wider steps are refused by the arch before memory even matters:
```
radiance: qwen4exp: the QSA score plane would be 3125 MiB -- 16384 rows (--max-num-batched-tokens 16384)
by --max-model-len 200000 / compress ratio 4. Lower either.
```
What boots and seals: **4 slots, `--max-num-batched-tokens 2048`** (the quantised file tier's ceiling), `--kv-cache-dtype fp8`, `--weights-disk-tier`, `--placement expert_tiered`, `--ngram-placement disk`, `--max-model-len 200000` (pool 199,948 tokens: `kv cache 1.50 GiB … 1.0 full-length`). Prefill lands at 50–70 tok/s; the tap's cost is expert re-streaming per step, not the engine.
## Concurrency: 4 stalls, 2 survives
With `--parallel 4` the pusher sends four ~7.8 K-token prefills; **two complete (7,335 and 8,054 tokens), then a step never returns**: log frozen, `W step: nothing has been logged for 21 s` repeating, all threads in `futex`/`kfd_wait_on_events`, 0 MiB/s of reads, `/health` still 200 and `/stats` still `running: 4` with the pool 98 % idle. Reproduced on both a 32,768 and a 200,000 window, so it is not the pool. Production restarts fine afterwards (`chat 200` in 0.49 s) — the card is not wedged. Treat a frozen engine log with live requests as a hang, not a slow rig, and drive the pusher at `--parallel 2`.
## Floor and seal contract
- `RADIANCE_CALIB_TOKENS=1500000` is enough: the vendor's own measurement is that a pooled Gram from 10 M tokens beats one from 1 M by under 1 %; 6 M cost 29 h on this rig.
- One seal writes **~40 GiB** — 24,768 files `gram.r<rank>.blk.<l>.ffn_down_exps.<e>.bin`, 42.9 GB. Give the tap dir ≥60 GiB; a 37 GiB filesystem filled to zero and every later write came back short.
- The engine **exits by itself when the tap seals**, so a mid-flight client gets `429 rate_limit_error: the server is shutting down`. That is success.
- Grams are written only at the seal: a restart before it loses the accumulation but no artifact, so a self-healing driver (kill rig, reboot, re-push) is safe.
## Two traps that read as successes
- **A port that answers is not your rig.** After the runner stops production, the dying process still answers `:7770` for seconds; a health-only gate "passes" at 10 s on a container that is not up, and the pusher then refuses on container name (or worse, pushes into production). Gate on `podman ps` **naming the container**, then health.
- **A container-based GPU tool must own the cards.** An orphaned engine keeps `/dev/kfd` and the next boot dies with `E startup stopped: device error` (also produced by `kill -9`-ing an engine: let the container teardown run, or kill by exact pid from `sudo fuser /dev/kfd` — never `pkill -f`, which also matches your own command line).
## Downstream
The tap is consumed by `rad-convert` via `gptq:calib=<dir>`; the file contract (`RADGRAM2`, `gram.r<rank>.<stem>.bin`) is version-stable, so a latest-written tap converts on latest. `scripts/grams-to-imatrix.py <calib> -o imatrix.gguf` turns it into an imatrix. Arm output size is ~the served container's size (114 G here) — check free space before converting, not after.
@@ -0,0 +1,71 @@
---
name: radiance-calibration-tap-shape
description: "Run and measure a radiance GPTQ calibration tap on a disk-tiered BF16 rig — n-gram tier, step sized to the document, tap marker gates, and reading the real rate from /stats instead of the log."
---
# Running and measuring a radiance calibration tap
A calibration tap (`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS`) boots a BF16 reference container
on the serving cards under `--weights-disk-tier` and prefills a corpus until the pooled token floor
is reached, writing per-expert Grams for GPTQ. It is the one campaign phase whose cost is dominated
by **disk read volume**, not by compute, and the shape that serves production is the wrong shape for
it.
## The four things that decide the tap's wall time
1. **N-gram table tier.** On the BF16 payload the table is tens of GiB and `--ngram-placement ram`
PINS it. Beside a full host pool that leaves a few GiB of RAM and the boot dies at
`startup stopped: out of memory` while the same pool and slot count had already booted the
reference recording. Calib only needs the routed weights prefilled, so pass
`--ngram-placement disk` for calib and keep `ram` for the KL modes — their numbers are only
comparable at an identical shape.
2. **Step ≥ document length.** A document cut into N steps re-streams each layer's routed expert
union N times from the container. With `--max-num-batched-tokens` below the document length the
tap runs several times slower than with one wide step, and the KV pool does not change at all
(`kv_tokens_total` is unaffected). This is legal for calib and NOT for a scoring boot: scoring
numbers are only comparable at one shape, so the step is calib's own knob.
3. **KV pool, not slots.** When the pool holds barely one document, extra `--max-num-seqs` only
queue (and preempt) requests; raising slots does not raise throughput.
4. **The token floor is pooled, not per round.** One corpus pass may be smaller than the floor, so a
round that seals no bytes is NORMAL — a driver that treats a flat round as failure aborts one
round early. Fail on a tap that *loses* bytes; let "never sealed" fail at max rounds.
## Never push a tap that belongs to no rig
A tap dir is a directory; production is also an endpoint on the same port. Without proof, a pusher
will happily feed millions of tokens into whatever answers and record nothing.
- The rig writes a marker into the tap dir at boot (`container=`, `image=`, `pid=`), retires it
when its boot exits non-zero, and the pusher refuses unless the marker exists, names the
container it targets, and that container is running.
- The window script stops on a failed phase instead of falling through its health wait, and banks a
large reference artifact only when both phases exit 0 and the tap holds files.
## Measuring the rate (do not read it off the log)
The campaign log is `tee`'d and block-buffered, so its request count lags by minutes.
```bash
curl -s http://127.0.0.1:7770/stats | python3 -c 'import json,sys; d=json.load(sys.stdin); \
print({k: d[k] for k in ("uptime_s","prefill_tps","steps_prefill","total_requests","preemptions","kv_tokens_total")})'
```
`prefill_tps` is CUMULATIVE over uptime — `tokens ≈ prefill_tps × uptime_s`, so take two samples a
couple of minutes apart and difference them. `steps_prefill / total_requests` proves whether a
document is one step or many. Discard the first minutes after boot (warmup and the first requests
are not the steady state).
## Sizing the tap before paying for it
Probe one 5-minute sample at the candidate step, compute the ratio against the inherited shape, and
choose the floor from the measured rate — not from an earlier campaign's runtime on a different
corpus and shape. Then set `RADIANCE_CALIB_TOKENS` (via the runner's env knob) and launch ONE run;
if the probe already beat the bar, that probe run IS the real run and needs no restart.
## Deliverables that make it unattended
- a gated window script: boot → health with a hard timeout → push → stop → restore the service →
bank the reference artifact only on success;
- a pusher that is marker-gated and progress-aware;
- a ledger line recording the shape used and the target chosen, so the next session does not
re-derive it.
@@ -0,0 +1,68 @@
---
name: radiance-disk-tier-throughput-census
description: "Measure a radiance rig's real prefill rate and prove whether the bottleneck is disk, CPU or shape — for calibration taps, KLD recording or score boots; use when a rate looks wrong, an ETA must be quoted, or someone asks about CPU allocation."
---
# Census a radiance rig's throughput without lying to yourself
Ask three questions in order: what is the rate, what is the binding resource, and does the shape
explain it. Every number below comes from a kernel or engine counter — never from a gauge.
## 1. The rate: counters, never `prefill_tps`
`/stats` on :7770 exposes `prefill_tps`, and it is a **rolling gauge**. Reading 69.5 → 149.1 across
120 s and multiplying by `uptime_s` produced a bogus "320 tok/s / 10.4 h" verdict that had to be
retracted (measured 2026-10-29). Derive the rate from counters over a ≥240 s window:
```bash
python3 - <<'PY'
import json, time, urllib.request
def stats(): return json.load(urllib.request.urlopen("http://127.0.0.1:7770/stats", timeout=10))
a = stats(); t0 = time.time(); time.sleep(240); b = stats(); t1 = time.time(); dt = t1 - t0
docs = b["total_requests"] - a["total_requests"]
steps = b["steps_prefill"] - a["steps_prefill"]
MEAN = 4238.0 # tokens/doc, from the corpus: total_tokens / documents
print("docs %d in %.0fs -> %.0f tok/s | steps/doc %.2f" % (docs, dt, docs * MEAN / dt, steps / docs))
PY
```
`steps/doc` is the read-amplification signal: a document cut into N prefill steps re-streams each
layer's routed expert union N times when the weights are on the disk tier, so N > 1 is leaked time.
## 2. The binding resource: diskstats and the idle/iowait split
```bash
# per-device read bandwidth (parent device, not the partition)
awk '{print $3, $6*512}' /proc/diskstats | grep -E "^nvme"
# host busy = total - idle - iowait (summing /proc/stat's first 8 fields WITHOUT subtracting is
# the classic error: the sum already covers idle, so it reads ~800% on an 8-core box)
```
Then split CPU by process: sample `/proc/<pid>/stat` fields 14+15 (utimes + stimes) twice.
Two hard readings that decide the argument:
- `radiance` compute at ~0.26 cores while the disk delivers ~3.2 GB/s and `iowait` is ~29 % of the
box ⇒ **disk-bound**. More cores or a `--cpuset-cpus` pin cannot add bytes; pinning only caps it.
- A container with `cpu_quota=0` and an empty `CpusetCpus` (check `podman inspect -f
'{{.HostConfig.CpuQuota}} {{.HostConfig.CpusetCpus}}'`) is already unrestricted — "allocate more
CPU" is a no-op, and "allocate 6 of 8" is a ceiling.
Sanity relation: tok/s × (GiB streamed per document ÷ tokens per document) should match the measured
device bandwidth. If it does not, the shape, not the hardware, is the story.
## 3. The shape: what may legally be widened
- `kldref` and `score` must keep production's `--max-num-batched-tokens` and `--max-num-seqs`: their
KL numbers are only comparable across boots at an identical shape (the audit law).
- A **calibration tap** is exempt — its output is layer Hessians, nothing compares it. Give it a step
that covers a whole document (`Q38_CALIB_MNBT` ≥ the corpus window) and read `steps/doc`: on a
7,168-token corpus, 2,048 → 8,192 moved 2.5 → 1.00 steps/doc and 94 → 113 tok/s (**1.2×**, not the
4× the step-count ratio suggests — total bytes are unchanged, so do not promise more).
- The KV pool, not the step, bounds concurrency: when `kv_tokens_total` ≈ one document, requests queue
and preempt; extra slots buy nothing.
## Reporting
Quote tok/s, steps/doc, device GB/s and the busy/iowait split together, plus the shape each number
was taken at. A rate without its shape is not a measurement; a gauge without its window is not a rate.
@@ -0,0 +1,37 @@
---
name: radiance-pinned-ram-ledger-pricing
description: "Price any .rad tensor-group dtype change (e.g. fp8_e4m3 - plain bf16) against a radiance host's pinned-RAM / ttm ledger before agreeing it is possible; answers \"can we afford this encoding\" rather than \"does the engine support it\"."
---
# Price a dtype change against the pinned-RAM ledger
Two separate questions, never merge them: **(a) does the runtime represent this encoding?** and **(b) does the host's pinned-RAM ledger have room?** A "yes" to (a) plus a "no" to (b) is the usual answer.
## 1. Does the runtime represent it?
Read the pinned source for the tensor group, not the recipe docs. For a PLE n-gram table, `arch/common/rad_block_ple.h` shows the encoding is *derived from the tensor*, not requested:
```
table_bf16 = weight_enc(...) && rad_enc_is(&te, "plain") && te.n_planes == 1
... RAD_W(w_table), table_bf16 ? RAD_NONE : RAD_W(w_scale), ...
```
Consequences worth stating out loud:
- One tensor group = **one** encoding. You cannot hold bf16 and e4m3 side by side in one container; two encodings means two artifacts.
- Replication rules differ per table. The n-gram table is shared under TP (`need = align_up(ngram_bytes, 2 MiB)`, "all ranks gather through one shared row cache over the same file") while the token embedding charges `emb_pin = tp * align_up(shard, MiB)`. Reading one formula and applying it to the other doubles or halves the bill.
- Check whether the existing encoding is **lossless** against the source before calling a wider one an upgrade. A stacked-official-fp8 staging note can prove the e4m3 codes are restored exactly (code grid 2^-3 vs bf16 rounding 2^-9), in which case bf16 costs bytes and buys nothing for that payload.
- Compare the constant already baked into the tooling. `FRANK_ARCHON_TTM_MIN_MIB` / the preflight's floor term is an accounting of what the *current* payload pins; matching it to `elements x bytes_per_element` identifies exactly which groups it covers and which it does not.
## 2. Does the ledger have room?
```
bytes = rows x width x bytes_per_element # fp8/u8/int8 = 1, bf16/fp16 = 2
MiB = bytes / 1048576
ceiling = cat /sys/module/ttm/parameters/pages_limit x 4096 / 1048576
floor = (tp x HOST_POOL_MIB) + prefix_cache_MiB + MNBT + store + table_MiB
delta = new_table_MiB - old_table_MiB # this is the real cost of a dtype flip
feasible = floor_with_new_table <= ceiling
```
- Cross-check `ceiling` against `FRANK_ARCHON_TTM_TARGET_MIB` and `MemTotal`; a target recorded as the host ceiling means the kernel already refused higher, so a raise is a test (reversible sysfs write, re-read and refuse by name) and not an assumption.
- Compute the cheapest concession that closes the gap, in order: shrink the prefix-cache host tier, shrink the host pool per rank, then surrender the payload being planned. Say which one it costs; never present a dtype change as free when the pool is what pays.
## 3. Report shape
State (a), (b), the arithmetic table (old MiB, new MiB, ceiling, floor both ways), the concession required, and the confound it adds to any existing comparison. Keep the recommendation to one line; the numbers carry it.
@@ -0,0 +1,37 @@
---
name: radiance-step-shape-ledger-audit
description: "Price a radiance serving-shape change (MNBT / max-num-seqs / window) from the engine's own boot ledger instead of a comment, and catch a mode that carries a private step."
---
Use when a radiance endpoint or campaign rig boots with a step/shape that differs from the one already proven, or refuses with `E startup stopped: out of memory` while the container image and env are unchanged.
## The law
The step's **activation arena is charged to the card**, not to the disk tier. `--max-num-batched-tokens` sizes it (see `vram_budget.cpp:117`, the engine's own fix text: "lower --max-num-batched-tokens (the arena ...)"), so a wider step is taken out of the weight tiers and the KV cache. A comment claiming a wide step is "free throughput" is not evidence. Never let one mode of a multi-mode serve script carry its own `--max-num-seqs` or step; derive both from the env file that owns them (for frank-archon: `scripts/frank-archon.env` → `FRANK_ARCHON_MAX_SEQS`, `FRANK_ARCHON_MAX_BATCHED_TOKENS`, `FRANK_ARCHON_HOST_POOL_MIB`, `FRANK_ARCHON_GPU_HEADROOM_MIB`).
## The audit (one ssh call, two logs)
Radiance writes the ledger to the file named beside the serve script (`/home/dank/logs/qwen38_bf16_<mode>.log`, `/home/dank/logs/frank_archon.log`) — **not** the journal. Extract, per boot:
```
for f in <proven_mode> <failing_mode>; do L=/home/dank/logs/qwen38_bf16_$f.log
for k in "--max-model-len" "--max-num-seqs" "--max-num-batched-tokens" \
"--host-pool-mib" "--tp " "--kv-cache-dtype" "--placement"; do
grep -m1 -E "^I +\\${k%}" "$L" | head -1; done
grep -m1 "activation arena" "$L"; grep -m1 "claimable" "$L"
grep -m1 "vram_kv" "$L"; grep -m1 "weights by tier" "$L"
grep -m1 "full-length sessions" "$L"
done
```
Compare row by row: `activation arena` → `claimable` → `vram_kv` → `weights by tier` (VRAM vs container-on-disk). The refusal line names the shape it could not back, e.g. `kv cache 271.25 MiB ... 0% backing of the 8 x 32768 worst case: 0.0 full-length sessions`.
## Reading the rows
- Arena grows with MNBT and is subtracted before weights/KV are funded (measured: 1,024 → 504 MiB arena, 30.87 GiB claimable; 65,536 → 10.65 GiB arena, 16.85 GiB claimable, experts squeezed to 10.98 GiB, 56 GiB of weights stranded on the disk tier).
- `vram_kv` below one full-length sequence for the requested seats ⇒ OOM refusal, even when a narrower shape at the same window booted.
- `expert prefetch 0 B` / "no lender" means the engine already gave the expert slab everything the arena left.
## Then
Fix by inheriting the proven numbers, delete the per-mode literals (no shim, no `FRANK_*_MAX_SEQS` alias), update the mode's own comment so the false premise is destroyed rather than re-argued, and prove with `<script> <mode> --dry-run` that the argv matches the proven mode byte for byte.
@@ -0,0 +1,81 @@
---
name: radiance-tap-arm-and-seal-verify
description: "Prove quickly whether a radiance build writes calibration Grams (arming line, 8k-floor seal, engine exits on seal, 43 GB seal, GPU-exclusivity preflight, pgrep/health gate traps)."
---
# Verify a radiance calibration tap in 15 minutes
Prove whether a radiance build writes calibration Grams (`RADIANCE_CALIB_DIR`) before
committing hours of drained serving to it. The whole answer is two log lines and a
directory size.
## The two lines that decide it
1. **Arming** — on the boot ledger, rank 0:
`recorded passes: off, the program accumulates calibration Grams (gram_accum)`
Absent = the tap is not in the program: nothing will ever be written, and
`/health`, `/stats` and a healthy push prove nothing. A 17-hour run on the
production pin (radiance 1.8.1) wrote **zero bytes** with no arming line.
2. **Seal** — when the pooled token floor is reached:
`radiance: calibration wrote layer Hessians after N tokens to /data/calib`
printed once per rank. Then the directory holds `gram.r<rank>.<block>.<stem>.bin`
files (~24,768 files, ~43 GB). Check `du -sb`; a plausible-looking empty dir is
the failure mode.
## The seal ends the server
The engine **shuts itself down** once it seals. In-flight requests then receive
`429 rate_limit_error: the server is shutting down`. That is SUCCESS, not an error:
wait for the seal line, and treat a post-seal 429 as done. Do not retry into it.
## Minimum proof run
- Boot the candidate image with `RADIANCE_CALIB_DIR=/data/calib` and
`RADIANCE_CALIB_TOKENS=8192` (a small floor, not the campaign floor).
- Push three or four ~7k-token documents concurrently; at ~64 tok/s aggregate the
seal lands in ~2 minutes.
- One seal costs ~43 GB: point the tap at a filesystem with ≥60 GB free and check
before booting.
- Consumer side is version-stable: `rad-convert` takes `gptq:calib=<dir>` and reads
`RADGRAM2` grams written by any pin, so the producer and the converter may be
different builds.
## Shape: the vendor recipe does not fit two 32 GiB cards
Vendor `docs/MOE-W4.md` prescribes `--max-num-batched-tokens 65536 --max-num-seqs 64`
(~930 tok/s). On two 32 GiB R9700s the activation arena alone is 12.83 GiB of the
27.95 GiB claimable and every variant stops in the VRAM resolver.
The qwen4exp arch prices a **QSA score plane** of `rows × window / compress`:
`the QSA score plane would be 3125 MiB -- 16384 rows by --max-model-len 200000 / 4.
Lower either.` → 16,384 rows is refused outright; 8,192 rows costs ~1.56 GiB.
The shape that fits and seals: production's own — 4 slots, `--max-num-batched-tokens
2048`, fp8 KV, `expert_tiered` + weights disk tier, n-gram table on disk.
## Exclusivity is not optional
A leftover calibration container silently owns the cards and poisons every later
boot with `E startup stopped: device error`. Before any boot:
```
sudo fuser /dev/kfd # must print nothing
podman ps --format '{{.Names}}'
```
After stopping a rig, `podman rm -f <name>` **and** kill any `radiance` pid still
holding `/dev/kfd` (`kill -9` by the pid from `fuser`, never `pkill -f`). Then verify
`fuser` is empty before booting the next shape — a 21-second `device error` at boot
is a dirty device, not a shape refusal.
Two gate traps, both measured:
- `curl /health` right after stopping the service can be answered by the **dying**
production process, so "up at 10 s" is a lie. Gate on the container name in
`podman ps` **and** the marker file the rig writes, not on the port alone.
- `pgrep -f "script.sh"` matches the polling shell's own command line; a
`while pgrep -f` wait loop then never exits. Gate on a file, a container name, or
the pid you captured at launch.
## Report
Write the outcome into the page/README that owns the campaign, not a diary: which
build arms the tap, the fitting shape, the floor and its warrant, and the seal size.
Numbers from a shape that is not running now are dead weight — keep the law.
@@ -1,18 +1,39 @@
---
name: radiance-w4-campaign
description: "Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration → protected experts → convert → KLD gate → deploy, with the engine-pin and MNBT laws."
description: "Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration tap pin and engagement proof, protected experts, convert, KLD gate, deploy — plus the tap-disk and orphan-engine laws."
---
Pipeline for quantizing a BF16 Qwen3.8-Flash-Next checkpoint into a servable `qwen4exp` `.rad` on dankserver (frank-primary). Reference run: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`; scripts in `scripts/` (`run-swift15-bf16.sh`, `calib-push.py`, `derive-protected-experts.py`, `apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`, `kld-gate-swift15.sh`, `fleet-probes-swift15.sh`).
Pipeline for quantizing a BF16 MoE checkpoint into a servable `.rad` on dankserver (frank-primary). Reference run: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`; scripts in `scripts/` (`run-swift15-bf16.sh`, `run-<model>-bf16.sh`, `calib-push.py`, `derive-protected-experts.py`, `apply-protected-cast.sh`, `convert-<model>.sh`, `kld-gate-<model>.sh`, `fleet-probes-<model>.sh`).
Sequence:
1. **Fetch** checkpoint with per-shard LFS sha256 verification (`-f`, byte counts, no token on argv).
2. **BF16 container**: `rad-convert` no-recipe (pure copy) — serves as the KLD reference engine.
3. **KLD reference**: record logits on the SERVE pin (1.0.13) at MNBT 32768, `--max-model-len 16384`, corpus ≤16k prompts. Writes atomically at completion (`rows:` line = receipt). Never scp a running bash script (lazy-read law).
4. **Calibration**: engine **1.2.0** (only pin with `RADIANCE_CALIB_TOKENS`); push ~10M tokens of the model's own distribution; per-expert down-proj Grams land as RADGRAM2 files (`gram.r<rank>.blk.L.ffn_down_exps.<e>.bin`). Grams are engine-invariant across pins — the version boundary is by design.
3. **KLD reference**: record logits on the SERVE pin at the production geometry, `--max-model-len` under the pool, corpus ≤ that window. Writes atomically at completion (`rows:` line = receipt).
4. **Calibration tap** — see the pin law and the engagement test below.
5. **Protected experts**: derive from gram energy (share ≥3 % AND ≥3× routing share), splice between `# PROTECTED-BEGIN/END` markers of the recipe. Plugin law: ONE encoding per expert table; only whole bf16 experts legal beside it — no gate_up/down split.
6. **Final convert** on the SERVE pin (CPU GPTQ, hours; disk gate ≥146 GiB free first).
7. **KLD gate** on the serve pin: **MNBT ≤ 2048** (w4 streams in place only there; 32768 → `DID NOT FIT ... stored form is not the container's`), pool 40960, fp8 KV matching the production KV dtype. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
8. **Deploy** `./scripts/deploy.sh dankserver --service frank-primary`; verify `/v1/models` id = served-model-name (alias independent of file — harness models.yml untouched), boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥256-token budget + `reasoning_content` fallback).
7. **KLD gate** on the serve pin at the production MNBT (a w4 does not stream in place outside the shape it was measured at), pool and KV dtype matching production. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
8. **Deploy** `./scripts/deploy.sh dankserver --service frank-archon`; verify `/v1/models` id = served-model-name, boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥512-token budget + `reasoning_content` fallback).
Laws: stop frank-primary for every GPU phase (relinked unit: stop, mask fails); verify docs against `podman inspect <name> --format {{join .Config.Cmd " "}}` (rootless, as dank) before writing shapes; ff-pull fails on untracked files the merge tracks — trash first.
## The tap pin law (measured 2026-10-30, cost: one 17-hour night)
`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS` arms the per-expert tap on the **1.2.x** line only.
- Every plugin from 1.2.4 through 1.8.1 still **contains** the machinery (`CalibExperts`, `RADGRAM2`, `calib_write_host`, the seal line `calibration wrote layer Hessians after %lld tokens`), so `strings` proves nothing about the pin.
- A runner that boots `FRANK_ARCHON_IMAGE` (1.8.1) for `calib` serves perfectly — `/health` 200, 1,700 documents at 118 tok/s, `.rig-boot` written, no error anywhere — and writes **zero bytes**: the tap never arms.
- The engine exposes **no tap counter** in `/stats` and prints **no arming line** in the boot ledger, so a dead tap is indistinguishable from a slow one for hours.
- Default `calib` to the 1.2.0 digest (`stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa` on dankserver as of 2026-10-30); override only by naming `--image` on purpose.
**Engagement test — 3 minutes, run it before any long tap:** boot with `RADIANCE_CALIB_TOKENS=8192`, push 2–3 real corpus documents, then require the seal line *and* bytes in the tap dir. On 1.2.0 that reads `calibration wrote layer Hessians after 10207 tokens to /data/calib`. No seal line in ten minutes = the pin or the arming is wrong; stop.
## The tap's disk and process laws
- **One seal writes ~40 GB** (39.4 GiB measured for an 8,192-token floor): the output is expert-shaped, not token-shaped. Put the tap dir on a volume with ≥60 GiB free and gate it before boot (`Q38_CALIB_MIN_FREE_MIB`). On a 37 GiB root fs it filled `/` to **0 bytes**; the engine's later writes came back `cannot write` / `write was short`, and *other* tooling on the host then failed too.
- **Removing the container does not kill its engine.** `podman rm -f <rig>` leaves `podman-init -- radiance …` alive holding `/dev/kfd`; production then fails `ar_oneshot_2rank_wht6 … device error` at kernel init, exits **0**, and `Restart=always` loops it (counter reached 100) while `/health` never answers. Diagnose with `sudo fuser -v /dev/kfd`; kill **by pid** from `fuser /dev/kfd` + `/proc/<pid>/comm == radiance`, never `pkill -f` (the pattern matches the caller's own argv). Put that gate in the runner's restore trap.
- The service stop/restore path needs `sudo systemctl` — polkit denies the unprivileged call intermittently (`Access denied … interactive authentication`), and a failed stop means the rig boots *beside* production.
## Rate measurement
`/stats` `prefill_tps` is a **rolling gauge**, never a cumulative counter: `gauge × uptime` invents rates (it once produced a bogus 4.5×). Derive tok/s from `total_requests` deltas × the corpus mean tokens/document, or from `steps_prefill`.
Laws: stop production for every GPU phase; verify shapes against `podman inspect <name> --format {{join .Config.Cmd " "}}` (rootless, as dank) before writing them; ff-pull fails on untracked files the merge tracks — trash first.
@@ -0,0 +1,36 @@
---
name: screenshot-walkthrough-in-static-docs
description: "Embed a real-browser user-flow walkthrough (screenshots as data URIs) into a single-file zero-JS static docs page; includes the selector laws, cookie isolation, and secret-gate proof earned driving franksign's full demo."
---
name: screenshot-walkthrough-in-static-docs
description: "Embed a real-browser user-flow walkthrough (screenshots as data URIs) into a single-file zero-JS static docs page."
# Screenshot walkthroughs in a single-file static guide
Friction-earned 2026-10-21 on `dank/franksign` `guide/index.html` §13: 19 un-retouched captures of every user flow (signup → studio → template → send → sign → freeze → signed PDF → claim shelf → void → 410 door), embedded as data URIs, guide still single file, zero JS.
## Recipe (end to end)
1. **Boot the prod-shaped stack** on a non-collision port (`PORT=3210`; e2e seams squat :3100, demos :3000). Kill it before running any test gate.
2. **Walk flows as distinct identities sequentially** — the managed browser shares ONE cookie jar across tabs. Between roles: `tab.clearCookies()` then `tab.reload()` **on the app's origin, never on `about:blank`** (domain-scoped clear on about:blank clears nothing and the previous identity's session silently survives — you'll screenshot the admin where you meant the signer).
3. Fresh per-checkout demo state is fine; re-running flows with new identities leaves harmless history that actually *helps* later shots (roster with completed + pending + voided rows).
4. **Selectors**: prefer `role/textbox[name="Label *"]` / `role/button[name="Sign"]` / `testid/…`. Forms render required markers INTO the label text (`Email *`) — `label="Email"` misses; `text/…` clicks may silently no-op where `role/button[...]` works. Snapshot once per screen (`tab.ariaSnapshot(undefined,{compact:true,interactive:true})`) instead of guessing.
5. **Screenshot to disk** (path option silently ignored — see `browser-screenshot-disk-and-cookies`): `tab.run(async ({page}, p) => { const r = await page.screenshot({encoding:"base64", type:"png"}); await Bun.write(p, Buffer.from(typeof r === "string" ? r : r.data, "base64")); }, { args: [path] })` — this variant returns a bare string, not `{data}`; handle both.
6. **Shrink before embedding**: `for f in *.png; do magick "$f" -resize 1000x "$f"; done` → 17–83 kB each; 19 shots ≈ 800 kB raw, ~1.1 MB final guide.
7. **Non-browser payoff shots**: rasterize artifacts the flow produces (e.g. signed PDF) with `pdftoppm -png -r 90` + resize — the deterministic-output image is the best single screenshot of a PDF pipeline.
8. **Assembly**: write the section HTML with `__IMG:name__` placeholders via the write tool; a one-shot node script replaces tokens with `data:image/png;base64,…` and splices before `<footer>`. Never paste base64 through the edit tool.
9. Update nav + add a pointer paragraph from the prose-journeys section to the walkthrough section.
## Verify
Serve the dir (`python3 -m http.server`), open, assert via one evaluate: `img` count == captured, `failed == 0` (`complete && naturalWidth>0`), and every `href^="#"` resolves to an id. Screenshot the section top + a mid scroll. Then kill the stack BEFORE the gate.
## Secret gate vs data URIs
A base64 wall trips shape patterns (`AKIA/ASIA[A-Z0-9]{16}` etc.) via case-insensitive grep at law-of-large-numbers rates. Prove noise, don't just override: `git diff --cached | grep -oEi '<patterns>' | sort -u` — hits that are substrings of PNG streams (mixed-case `AkIa…`, random tails) are image bytes. Then use the hook's documented deliberate override (`ALLOW_SECRETS=1 git commit`) and record the proof in the commit body. Confirm no real credential is visible in any shot first (password fields render masked; note demo tokens in captions are dev-DB throwaways).
## Voice/law notes
- Captions must state what the demo fakes (no SMTP → "link travels by copy-paste") and that captures are un-retouched with real demo values.
- The walkthrough routinely surfaces UI laws the prose hid (e.g. Void `disabled={r.state !== "pending"}` while the API accepts completed voids) — write those discoveries into a closing law box, and check the prose section against them.
@@ -0,0 +1,77 @@
---
name: script-refactor-parity-vs-previous-revision
description: "Prove a refactor of sibling data-pipeline scripts is behavior-identical: run the pre-refactor revision from git HEAD beside the new one on the host that owns the venv/data, compare parsed records rather than bytes, and bootstrap the shared module so file, runpy and container paths all import it."
---
Refactoring a pipeline of sibling scripts (corpus builders, re-chunkers, pushers) whose only
tests are smokes. The test suite cannot see a moved record boundary, so parity has to be
*measured against the previous revision*.
## 1. Keep the old revision runnable
Never trust the diff. Materialize both sides where the data lives (the host with the venv,
tokenizer, parquet, corpus):
```sh
git show HEAD:scripts/<tool>.py > /tmp/qc-old/<tool>.py # pre-refactor
cp scripts/<tool>.py scripts/<newmodule>.py /tmp/qc-new/ # post, module beside it
```
`ssh host 'cat > /tmp/...'` with the bytes on stdin is enough; no repo on the target needed.
Run both with **identical seeds and flags**, into separate output dirs.
## 2. Compare records, not bytes — unless the escaping is unchanged
- If the refactor funnels writes through one helper, that helper's `ensure_ascii` choice moves
every non-ASCII character. A byte diff then reports a false regression while the rows are
identical.
- Prove it correctly: parse both sides and compare in order —
```python
a = [json.loads(l) for l in open(old) if l.strip()]
b = [json.loads(l) for l in open(new) if l.strip()]
print(a == b, hashlib.sha256(json.dumps(a, sort_keys=True).encode()).hexdigest()[:16])
```
- Where the refactor does not touch serialization (a tool that already wrote literal UTF-8),
demand the stronger proof: `cmp` + md5 equality of the artifact.
- Order matters: shuffled output with an unchanged seed proves the RNG draw sequence survived
the hoist of nested closures. Compare the *sorted* file only as a second check — permutation
equality alone hides a reordered `rng.choice`.
- Hashing the record set with `sort_keys=True` also catches dict-key drift in manifests.
## 3. Sibling-module bootstrap that survives every invocation
A new shared module must import when the tool is run three ways: `python3 scripts/tool.py`,
`runpy.run_path('/work/tool.py')` from a container mount (`-v scripts:/work:ro`), and from any
cwd. Standard preamble:
```python
_HERE = os.path.dirname(os.path.abspath(__file__)) if "__file__" in globals() else os.getcwd()
if _HERE not in sys.path:
sys.path.insert(0, _HERE)
import <module> # noqa: E402 - sibling module, same directory
```
Piping the script through `python3 -` has no `__file__` and defeats this — run tools by path and
say so in the module docstring. Verify both forms explicitly before claiming it works.
## 4. Unit-test the refusals without the heavy dependency
Hoist validation into a function so it can be driven with a stub, in-process:
```python
spec = importlib.util.spec_from_file_location("<module>", ".../<module>.py")
m = importlib.util.module_from_spec(spec); sys.modules["<module>"] = m; spec.loader.exec_module(m)
```
Registering the module in `sys.modules` under its real name *before* loading the dependent
script is required, or the script's own `import <module>` re-imports it and your stub is invisible.
A stub object with `encode`/`decode` that mangles ids past a threshold exercises a
"token identity" check in one page of output, no tokenizer file needed.
## 5. Gate order
`py_compile` every touched file → run the tool's `--help` through both invocation forms →
old-vs-new parity on the host → suite → commit. A suite that enumerates shell seams will not
notice a corpus-tool regression; the parity run is the only evidence there.
@@ -0,0 +1,100 @@
---
name: sftp-client-unreachable-triage
description: "Diagnose \"SFTP/SSH from box A to box B doesn't work\" when box A is unreachable from the fleet: exonerate the server with a log+key census, then discriminate sleep vs firewall vs name-resolution vs missing key from outside."
---
# Triage a client-side SFTP/SSH failure (client box unreachable)
Use when Frank reports "FileZilla/SFTP/scp from <client> to <server> isn't working" and the
client box is not reachable from the fleet. Produces a ranked, evidence-backed verdict
without access to the client. **Read-only probes only**; no restarts, no config writes.
## Order of operations (each step can end the investigation)
### 1. Server-side exoneration — gather before theorising
```bash
ssh -o BatchMode=yes <server> 'sudo -n sshd -T | grep -Ei "^(port|listenaddress|pubkeyauth|passwordauth|kbdinteractive|allowusers|subsystem|authorizedkeysfile)"'
ssh -o BatchMode=yes <server> 'ls -l /usr/lib/openssh/sftp-server; sudo -n ls -l /etc/ssh/ssh_host_*key.pub' # host-key mtime = known-hosts churn check
ssh -o BatchMode=yes <server> 'sudo -n iptables -S | head; sudo -n nft list ruleset | head -20' # INPUT ACCEPT = no firewall
```
Host keys unchanged for years ⇒ FileZilla cannot be tripping a host-key warning.
### 2. Censuses that turn "maybe" into "never" (the payload of this skill)
```bash
# keys trusted by the server, by comment
ssh <server> 'ssh-keygen -lf ~/.ssh/authorized_keys'
# every IP that ever authenticated, and with which key
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
| grep -oE "Accepted publickey for [a-z]+ from [0-9.]+" | awk "{print \$NF}" | sort | uniq -c'
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-90 days" \
| grep -oE "ED25519 SHA256:[A-Za-z0-9+/=]+" | sort | uniq -c'
# did the client IP EVER appear, anywhere?
ssh <server> 'sudo -n journalctl -u ssh --no-pager --since "-45 days" | grep -c "<client-ip>"'
ssh <server> 'sudo -n zgrep -h "<client-ip>" /var/log/auth.log* | grep -v sudo | tail'
```
Count 0 across journal **and** rotated logs ⇒ no TCP ever established ⇒ the failure is
DNS, client egress, or a sleeping host — **not** credentials. Say so explicitly, because
"permission denied" and "connection timed out" have different fixes.
### 3. Is the client even at the IP everyone believes? (DHCP drift)
```bash
avahi-resolve -n <Client>.local # mDNS truth
ip neigh | grep -E '\.(old|new) ' # FAILED vs REACHABLE
nmap -sn -n 192.168.0.0/24 # works unprivileged; arp-scan needs root
```
macOS **Private Wi-Fi Address** rotation changes the MAC ⇒ new DHCP lease ⇒ every
hardcoded IP for that box rots. Check both old and new addresses for mentions in
`/var/log/auth.log*` and `journalctl`.
### 4. Discriminate sleep vs firewall (ARP/mDNS alive, everything else dead)
- ARP replies + live mDNS but **zero** ICMP and **zero** TCP on a spread of ports
(22, 80, 445, 548, 5900, 8080, 5000, 7000, 3283) from **multiple** hosts including a
wired one: probe from a wired box on the same L2 — that also rules out Wi-Fi AP isolation.
- Same result ⇒ either the host is asleep (Bonjour sleep proxy / AP answers ARP+mDNS on its
behalf) **or** a host packet filter (macOS "Block all incoming", LuLu/Little Snitch,
ProtonVPN kill switch) is dropping it.
- **Do not overclaim which.** State both, and note the built-in macOS firewall only blocks
*inbound* — so if Frank is actively using that Mac, a built-in-firewall verdict cannot
explain an *outbound* SFTP failure; a third-party filter/VPN or DNS is then the cause.
### 5. Name resolution (the silent failure)
```bash
getent hosts <server> <server>.local <server>.<domain>
dig +short <server> @<pihole-ip>
```
Traps seen on this fleet:
- Bare `dankserver` resolves **nowhere** — no Pi-hole custom record, gateway DNS refuses
client queries. Only mDNS `<host>.local`, which may advertise a **link-local IPv6 only**.
- `dankserver.franksdank.cloud` resolves to **Caddy (192.168.0.143)**, not the SSH box —
a wildcard vhost. Never suggest a `*.franksdank.cloud` name for SSH/SFTP.
- **FileZilla ignores `~/.ssh/config`** — `Host dankserver` aliases do not apply; the Host
field needs a literal IP that resolves.
### 6. Fix recipe handed to the user (client side)
```bash
ping -c2 <server-ip> && nc -vz <server-ip> 22
ls ~/.ssh/id_ed25519.pub || ssh-keygen -t ed25519 -C "<client-name>"
ssh-copy-id -i ~/.ssh/id_ed25519.pub dank@<server-ip>
sftp -i ~/.ssh/id_ed25519 dank@<server-ip>
```
FileZilla: Protocol **SFTP**, Host literal IP, Port 22, Logon Type **Key file** →
`/Users/dank/.ssh/id_ed25519` (or Normal + password). The "Encryption" dropdown is FTP-only
noise for SFTP sites.
### 7. Report collateral you proved, don't guess it
Dead client IP ⇒ grep the fleet for hardcoded references and *observe* the breakage:
```bash
grep -rn --exclude-dir=.git "<old-ip>" ~/Projects/serverAppConfig | wc -l
curl -sk -o /dev/null -w '%{http_code}\n' --resolve "radarr.franksdank.cloud:443:<caddy-ip>" https://radarr.franksdank.cloud/
```
(`-H Host:` alone gives `000` — curl sends the IP as TLS SNI and the handshake fails. Use
`--resolve` to keep SNI correct before concluding anything about a vhost.)
## Laws
- Zero packets at the server beats any credential theory; log census first, key census second.
- ARP/mDNS answering is **not** proof a host is awake; the AP or a sleep proxy can answer.
- Probe from a wired host before blaming Wi-Fi isolation.
- A rotating private Wi-Fi MAC will re-break every hardcoded IP: recommend a DHCP
reservation as the fix, not 40 file edits.
@@ -0,0 +1,32 @@
---
name: snapcompact-reader-recall-probe
description: Measure bitmap-frame OCR recall and true per-frame token billing of a custom vision endpoint to pick snapcompact shape and price it correctly.
---
# Snapcompact reader recall probe
Decide snapcompact settings for a non-frontier (custom/local) vision model: which frame shape its tower actually reads, and what the endpoint really bills per frame (vs the harness's blind `FRAME_TOKEN_ESTIMATE`=5024 floor).
## Fact sources (in trust order)
1. `omp://compaction.md` (Snapcompact method section) + `cfg://snapcompact` + `cfg://compaction` — mechanism + resolved values/defaults.
2. Upstream ground truth: `raw.githubusercontent.com/can1357/oh-my-pi/main/packages/snapcompact/CHANGELOG.md` — per-model eval winners and the `toolbench` re-tune (spacing-tuned cells beat dense; ~16px/char OCR floor). `research/` dir has the eval scripts.
3. Live bundle truth: grep minified `dist/cli.js` (`~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/`) for string literals: `grep -o '\[\[/[^;]\{0,400\}' cli.js` shows the ideal-shape regex table (`claude→11on16-bw, gemini→8on22-bw@2048, gpt|codex→8on22-bw, kimi→8on22-bw, glm→8on16-bw`); `unknown:"8on22-bw"` is the openai-compat wire fallback. Unmeasured ids (no qwen matcher) get the wire-family shape.
4. Reachability of pricing knobs (proved 2026-10-27 on 18.12.5): the user-row merge chain (`grep -o '.\{80\}t\.imageCountBudget.\{200\}' cli.js`) forwards `imageCountBudget` ONLY — `imageTokenization` resolves through the compiled catalog identity (no user-row key reaches it, so custom endpoints can never mark frames `verified` and always eat the 5024 floor in sizing). Regimes: `openai-patch | anthropic-patch | deepseek-patch | fixed` (`"fixed":return e.tokens`) — catalog-side only.
## Probe recipe
Scratch dir: `bun init -y && bun add @oh-my-pi/snapcompact` (ships TS source; native renderer via pi-natives). npm version may drift a patch ahead of the harness bundle — geometry caveat, note it.
`probe.ts` legs against the endpoint's `/v1/chat/completions`:
- Build ~120k-char tool-traffic transcript in snapcompact scope format (`¶user:`, `¶call:` + `<out>…</out>`, `¶ai:`) with N planted unique spelled-out facts (spell digits as words so filler can't collide; keep filler digit-free); score recall by `content.includes(value)`.
- Legs: plain-text baseline, then images per candidate shape via `resolveShape({api, id}, variant)` + `renderMany(text, {shape})`; send as `{type:"image_url", image_url:{url:"data:<mime>;base64,<data>", detail:"original"}}` with a reading-guide text part.
- Replicate the harness's real request shape: same `chat_template_kwargs` (enable_thinking/preserve_thinking) and sampling from the models.yml row.
- True per-frame billing = `(prompt_tokens − text-scaffold) / frames`, compare vs `plannedFrameTokens(shape)`/`frameBilling`.
Then: declare `imageCountBudget: <proven count>` on the models.yml row (custom providers inherit the unknown-provider floor of 5 images/request; llama.cpp has no such limit) — that lifts the count clamp; token pricing stays floor-bound until upstream prices the wire family. Measured baseline (frank-archon, omp 18.12.5): auto `8on22-bw` 8/8 recall at 47 % of text's billed tokens, ~2,370 tok/frame true vs 5,024 charged; dense grids (8on16/11on16) drop characters — keep `shape: auto`.
## Gotchas
- `log()` is an eval-kernel global, NOT a bun global — define a console wrapper or the script dies at first call.
- bash `async` + `name` = service mode (rejects async); drop `name` for finite jobs; `timeout: 0` disables the 600s deadline for multi-leg vision runs.
- edit tool needs the absolute path (not bare filename) for /tmp files; `PUT <N` inserts BEFORE line N.
- Vision gate: row must declare `input: [text, image]` or snapcompact is skipped silently.
- Fleet follow-ups live in `omp-docs/core/snapcompact.md` (dank/command-center).
@@ -0,0 +1,27 @@
---
name: static-page-headless-verify-traps
description: "Verify a static HTML artifact headlessly after writing it: serve on a scratch port, assert structure, screenshot — with the omp JS-eval/browser API traps (browser.open options object, no tab.wait, tab.run is Node-isolated, log() swallowed in JS cells)."
---
# Headless verify of a static HTML file — traps that cost cells
Use after writing/rewriting a self-contained HTML artifact (guide, report, landing page) that must be proven to render before commit.
## Loop
1. Serve: `Bun.spawn(["python3","-m","http.server","<port>","-d","<dir>"])` inside the eval cell; kill at cell end (`proc.kill(9)`), or `pkill -f "http.server <port>"` first if a prior cell died mid-run.
2. Open + assert + screenshot in ONE cell; also cross-check anchors statically with node (regex `href="#x"` vs `id="x"`) — no browser needed for that half.
3. Entity/markup contamination sweep with grep on the FILE (`&quot;`, `&#`, raw `<` inside text): what renders small can hide real damage; conversely a suspicious screenshot is often just low-res misread — settle label text via `tab.text("svg")`/`tab.text(selector)` before "fixing" markup.
## Eval/browser API traps (all hit live 2026-10-21)
- `browser.open({ url })` — options OBJECT; `browser.open(url)` is a TypeError.
- `tab.wait(...)` does NOT exist — use `await Bun.sleep(ms)` or better `tab.waitForSelector/waitForText`.
- `tab.run(fnOrString)` executes in the isolated Node-side runtime: `document` is undefined there. For page-world expressions use `tab.evaluate("...string...")` (expression, no top-level `return`; wrap a function IIFE when you need one) or the direct helpers (`tab.text(sel)`, `tab.count`, `tab.attr`).
- **`log()` output from JS eval cells is silently swallowed** — even after successful browser calls. Extract data by `return`-ing the value from the cell (Python cells log fine; JS cells must return). A cell that "ran but printed nothing" usually means it printed via `log()` — re-run returning the payload.
- Screenshots come back as webp images in the tool result AND save via `screenshot({path})`; image text under ~13px is unreliable to read — verify exact strings via `tab.text()` instead of squinting.
## Static anchor/claim audit before rendering
- Counts, schema/table names, endpoint bodies: re-grep the SOURCE per claim; never trust numbers carried in from conversation memory (this caught "public vs franksign schema", "OR signer_user_id join that doesn't exist", wrong test-counts pre-commit).
- `git diff --stat` regenerated binary fixtures (PDF CreationDate churn) before committing docs — don't commit fixture noise the setup script wrote.
@@ -0,0 +1,32 @@
---
name: tts-synth-timeout-cap-run-triage
description: "Triage a dankvoice \"The operation timed out.\" toast: prove/disprove a qwentts engine cap-run (2048-frame, no EOS) via Dankclaw journals before suspecting the client."
---
# TTS synth-timeout triage
Symptom: omp shows `Error: dankvoice: The operation timed out.` — a real 60 s stall, not a client bug. `SYNTH_TIMEOUT_MS = 60_000` (`src/audio-io.ts`) + the notify at `utterance.ts:181` is the ONLY emitter of that text.
## Known root cause (chronic as of 2026-10)
The pinned `qwentts.cpp tts-server` (frank-tts-engine, Dankclaw, container port 8880) can miss EOS and run to its hard-coded 2048-frame cap: ~164 s of audio from a short sentence, ~90–100 s wall. Measured ~5/day, 68 in 14 days. The engine is a serial FIFO (`QWENTTS_MAX_BATCH=1`), so every narration queued behind a cap-run also times out — the toast comes in bursts. Shim logs `BrokenPipeError` at `wfile.write` (client gone when buffered audio finally ships). No frame-cap/deadline flag exists (`/app/tts-server --help`); the shim can't cancel (engine buffers before writing); `SHIM_TIMEOUT=600` deliberately outwaits it. **Fix belongs in `serverAppConfig` (watchdog restart or upstream feature request), never in dankvoice.**
## Procedure
1. **Endpoint is alive?** `curl -fsS -o /dev/null -w "%{http_code} %{time_total}s\n" -m 10 http://192.168.0.143:7772/v1/audio/voices` and time one synthesis with the session's actual voice (voice may be set per-directory in `~/.omp/cache/dankvoice.json` `byDir` — read that first).
2. **Read the journals — from bash ssh, NOT the `ssh://dankclaw/` device.** The ssh:// device lands as a different user (blank) than the ssh config (dank@192.168.0.143) and sees no units, no sockets — false negatives everywhere:
```bash
ssh -o BatchMode=yes dankclaw 'journalctl --user -u frank-tts --since "-6h" --no-pager | grep -B1 -A20 BrokenPipeError | tail'
ssh -o BatchMode=yes dankclaw 'journalctl --user -u frank-tts-engine --since "-6h" --no-pager | grep -E "Generation done : (2048|[89][0-9]{2}) frames"'
```
Signature = a facade `BrokenPipeError` whose timestamp is ~60 s after an engine `Admit slot=0`, and/or `Generation done : 2048 frames`. Window-correlate with the engine: `--since "HH:MM" --until "HH:MM"` and skip `[Sample]`/`Generated` noise.
3. **Chronicity count:** `journalctl --user -u frank-tts-engine --since "-14 days" --no-pager | grep -c "2048 frames"`.
4. **Verdict:** cap-run confirmed → the client behaved correctly (toast + next utterance speaks). Route the fix to `dank/serverAppConfigs` (`dankclaw/voiceservice/`): host-side watchdog on the journal signature restarting `frank-tts-engine.service`, and/or upstream per-request frame budget.
## Traps
- `systemctl --user` / `journalctl --user` on Dankclaw require the `dank` user — rootless quadlet units live there.
- zsh on Dankclaw eats `===` separators and unquoted URLs as globs: quote args or avoid `echo ===`.
- `tts-server` is not on the exec PATH: use `/app/tts-server`.
- Roster may contain only cloned voices (narrator1-4 retired) — a 400 on a stale voice name is a separate bug; timeouts are the cap-run.
- Elevated RTF right after idle (~2 min gap) is iGPU clock ramp, not contention.
@@ -0,0 +1,34 @@
---
name: upstream-engine-reference-into-vault
description: "Author a multi-page reference for an upstream engine into the serverAppConfigs OKF vault, passing the docs seams."
---
# Upstream engine → vault reference set
Use when asked to "deeply read and document" an upstream inference engine (radiance, llama.cpp, halogen) into `serverAppConfigs-docs`. Distill a repo's docs + ABI + option registry into progressive-disclosure pages, not a transcript.
## Read order (source beats docs)
1. `README.md`, then `docs/` TOC; read the serving guide, tools reference, plugin guide, architecture guide, op vocabulary.
2. Find the **authoritative option table** in source (`core/config.cpp` for serve flags, `tools/<tool>.cpp` `usage()` for tools, `abi/*.h` for plugin fields). Docs lag: radiance's `TOOLS.md` omitted `--chat-template`, `--lora`, `--lora-map`.
3. Version/ABI: `CMakeLists.txt` project VERSION and `#define RAD_ABI_VERSION`; these are independent.
4. Read one complete real plugin/example end to end (e.g. `plugin/spechub/exports.cpp`, `arch/llama_fp8/llama_fp8.cpp`) before writing "how to write a plugin".
## Page set (one owner per fact)
Hub + Options + HTTP API + Conversion + Model Families + Plugins + Architecture Plugins + Tools + Operations. The engine pages own engine defaults/flags/ABI; the fleet endpoint page (e.g. `LLM/frank-archon.md`) keeps live shape, Eras, measured laws, rollback. Link, never restate.
State the upstream commit + release + ABI, and separately the fleet pin; a flag newer than the pin is a boot refusal, not a no-op.
## Vault mechanics that fail the seams
- Frontmatter: `type` ∈ reference/howto/tutorial/explanation, `description` non-blank, ≤120 chars, not the title, no fleet quantity; ISO `timestamp` not future.
- Every page ≤500 lines. Split rather than exempt.
- Tables: escape `|` inside cells as `\|` (pipes inside backticks still split GFM tables).
- Register pages: edit the directory `index.md` rows with the **exact** description, then `python3 scripts/gen-vault-index.py serverAppConfigs-docs` (must be a no-op on a second run).
- Run: `check-okf.py`, `check-vault-links.py`, `bats tests/docs_completeness.bats`; then `./tests/run.sh`.
- Do not add a second log/status file; the vault's `log.md` policy is owned by the repo.
## Commit
Branch `docs/<slug>` off `origin/main`; commit docs + generated indices together. If main moved, merge it (never rebase) and re-run the full suite. Report commit hash, page list, test evidence; ask before opening a PR.
@@ -0,0 +1,46 @@
---
name: worktree-identical-file-tag-collision
description: "Diagnose and repair edit-tool tag collisions when two git worktrees hold byte-identical copies of the same-named file, so edits land in the wrong checkout."
---
## Symptom
You edit `path/to/file.ext` (bare or `~`-relative name) while another worktree under `~/.omp/wt/` holds a byte-identical copy of that file:
- the edit lands in the OTHER worktree (the most-recently-read copy), or
- the tool rejects with "header needs this file's current tag #XXXX" even though you just read the target's #XXXX, or
- after the edit, `grep -c "your marker"` in the intended worktree returns 0.
Root cause: the edit tool keys `[path#tag]` headers partly by hash; identical content ⇒ identical tag ⇒ ambiguous match, resolved to the freshest-read copy.
## Detection (before editing anything)
```bash
grep -c "<distinctive marker you are about to add or just added>" <target-worktree>/<file> # 0 after a "successful" edit = collision
```
Never trust the edit tool's success echo alone when sibling worktrees exist — verify the marker in the TARGET file.
## Repair (stray edit landed in the other worktree)
Restore that copy from its own HEAD — never hand-reverse:
```bash
cd <other-worktree> && git show HEAD:<file> > <file>
```
## Fix the collision, then re-edit
Give the two copies different hashes so tags stop colliding — cheapest is a scratch marker on the NON-target copy:
```bash
cd <other-worktree> && echo "# <id> scratch copy" >> <file>
```
Then re-issue the edit against the target with its original tag, and verify with `grep -c` again. (Editing the stray copy to a different content also works; anything that diverges the hashes.)
## Prevention
- When sibling worktrees exist, `read` the TARGET file by its worktree-scoped path immediately before the edit, and use that exact path in the header.
- Prefer including a distinguishing context line in the edit anchor so a mis-landed edit is rejected instead of applied.
- Same trap bites tests that copy fixtures (`$REPO_ROOT` resolves via the helpers file's location, not `$PWD`): red-proof a seam by copying the whole tests+fixture tree to a temp dir and running bats from there, or by testing the embedded command directly against a mutated fixture copy.
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792532762570
"lastConsolidatedAt": 1792630061849
}
@@ -1,6 +1,6 @@
{
"v": 1,
"lastConsolidatedAt": 1792439707623,
"lastConsolidatedAt": 1793314235443,
"lastResult": {
"at": 1791599865775,
"sessions": 1,
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792557324990
"lastConsolidatedAt": 1793396927634
}
@@ -1,4 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792469747088
"lastConsolidatedAt": 1793151249086
}
@@ -0,0 +1,15 @@
## Probe Network
- DO NOT ALLOW PROBES TO LEAVE HOME NETWORK
## Device Access
- AUTHORIZED TO ANY HOME NETWORK DEVICES BUT NO OTHERS
- PRESERVE LOCAL API SETTINGS
## Change Policy
- DO NOT ALTER OR MAKE CHANGES TO HOME NETWORK OR DEVICES
## Safety
- DO NOT BREAK ANYTHING
## Recon Method
- PURE DEEP RECON
@@ -1,14 +1,14 @@
## Agent Output Patterns
- Typed intro: "Update, [randomly generated epithet]!"
- Heckle mode option
- Typed outro: "More later, over and out!"
- Generation frequency control
## Documentation Flow
## Model Connectivity
- CONNECT TO MODELS FROM OUTSIDE NETWORK
- Put any created docs in the existing knowledgebase instead of creating a new one
## Documentation Flow
- Put any created docs in the existing knowledgebase
## Log Organization
- Logs kept explicitly to one file inside the knowledge base
@@ -1,10 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1792572225299,
"lastConsolidatedAt": 1792627495365,
"lastResult": {
"at": 1792557082020,
"at": 1792626257549,
"sessions": 1,
"deltas": 1,
"deltas": 2,
"model": "frank-zealot"
}
}
@@ -0,0 +1,5 @@
## Okf Format
- "final HTML report"
## Human Readability
- "human readability"
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1792572021650,
"lastConsolidatedAt": 1792632011734,
"lastResult": {
"at": 1792571693973,
"at": 1792625888033,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -0,0 +1,3 @@
## Formatting
- okf format
- formatted for human readability
@@ -1,4 +1,6 @@
## Templates
- Template submissions proceed as an agentically-assisted process.
## Agent Skills
- 16 is a project level agent skill, rather than a strict application thing
- 16 is a project level agent skill, rather than a strict application thing
## Design Assumptions
- All users are studios.
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1792572038693,
"lastConsolidatedAt": 1792627453281,
"lastResult": {
"at": 1792556461435,
"at": 1792609389951,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -0,0 +1,2 @@
## Documentation Guides
- Step-by-step guide with platform screenshots for all user flows
@@ -3,5 +3,9 @@
- Compaction applied
- The suite shouldn't loop through every epithet
## Operation flow
- The operation timed out
## Testing behavior
- Tests play ONE random Epithet
- Tests play ONE random Epithet
- Tests must not depend on the external API
@@ -1,8 +1,8 @@
{
"v": 1,
"lastConsolidatedAt": 1792572210019,
"lastConsolidatedAt": 1792627572197,
"lastResult": {
"at": 1792566095222,
"at": 1792622122061,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792574109325
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792607743914
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792627428845
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792631623051
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792646196892
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1792692724183
}
@@ -0,0 +1,4 @@
## Structural Standards
Quant quality must hold regardless of implementation choice.
Container versions must be digest-pinned to latest snapshot.
@@ -0,0 +1,5 @@
## Product Quality Standards
## Behavioral Updates
Merge any fixes immediately.
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793398614775,
"lastResult": {
"at": 1793397014344,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793162949974,
"lastResult": {
"at": 1793156304686,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793215583618
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793308515603
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793308588721
}
@@ -0,0 +1,4 @@
## Models
- Models settings include /switch to switch to it
- hdmi monitor to the left and the other to the right of eDP-1
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793312076890,
"lastResult": {
"at": 1793312076890,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793310990286
}
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793315256444,
"lastResult": {
"at": 1793314927653,
"sessions": 1,
"deltas": 1,
"model": "frank-zealot"
}
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793322540660
}
@@ -0,0 +1,4 @@
{
"v": 1,
"lastConsolidatedAt": 1793398691951
}
@@ -0,0 +1,3 @@
## Runtime Decisions
Avoid subagent spawns when GPU saturation occurs on Dankserver.
@@ -0,0 +1,3 @@
## Reliability
No reboots on deployed containers.
@@ -0,0 +1,10 @@
{
"v": 1,
"lastConsolidatedAt": 1793398788553,
"lastResult": {
"at": 1793397235249,
"sessions": 1,
"deltas": 14,
"model": "frank-zealot"
}
}
+235 -10
View File
@@ -47,6 +47,18 @@ providers:
# (arch/common/rad_block_vit.h). Video preprocessing configs ship
# too; images are the contract that matters here.
input: [text, image]
# Catalog image-count rules never match custom providers, so the
# unknown-provider floor (5 images/request) would silently cap the
# snapcompact archive at ~5 frames per rebuild. Radiance has no
# image-count limit and accepted 14 frames in one measured probe
# (2026-10-27, recall 8/8); the real clamp stays the archive sizing
# law (frames get half the room under the compaction threshold).
# NOTE: per-frame token pricing (imageTokenization) is catalog-only
# — no user-row key exists — so frames keep being charged the
# 5,024-token floor estimate against a measured ~2,400. Archive
# capacity is count-bound until upstream prices openai-completions
# vision honestly.
imageCountBudget: 16
compat:
# radiance's --reasoning-format defaults to auto: thinking
# comes back in `reasoning_content` (config.cpp). No parser
@@ -96,19 +108,20 @@ providers:
# halogen flash-server on the Strix Halo iGPU. Payload, shape, eras,
# laws, rollback: serverAppConfigs-docs/LLM/frank-dragoon.md — and
# scripts/frank-dragoon.env is the authority for every number there.
# contextWindow sits deliberately BELOW the engine ceiling
# (HALOGEN_CTX): the clamp below leaves the gap that absorbs the
# harness's chars/4 under-count, so a window equal to the ceiling
# would make every long request a guaranteed 400.
contextWindow: 155648
# contextWindow: 200,000 under the native 262,144 engine ceiling
# (HALOGEN_CTX, 2026-10-21) — the 400-zone law: the window MUST sit
# ~45k under the ceiling, never at it.
contextWindow: 200000
# The clamp is max_tokens = min(maxTokens, contextWindow − est −
# 4096); the chars/4 est UNDER-counts the rendered prompt by ~3 %
# on code/agent traffic (measured 2026-10-07: est 152,938 ->
# server 157,388). Worst-case request = window − 4096 + 3 % est
# error ≈ 156.2k, inside the engine ceiling with ~44k of margin
# (robust to a ~25 % under-count). Compaction fires at 85 % of the
# window and rides on frank-zealot: input ≈ threshold −
# keepRecent(20k), comfortably inside its per-slot window.
# server 157,388). Worst-case request = (200,000 − 4,096) × 1.03
# ≈ 201.8k; the compaction trigger fires at 95 % (config.yml
# thresholdPercent): 190,000 × 1.03 + 45,056 ≈ 240.8k — both inside
# 262,144 (margin 60k / 21k). Compaction rides frank-zealot:
# input ≈ 190,000 − keepRecent(20k) ≈ 175k ≤ its 200,704 slot.
# Recompute all of it after ANY window or threshold change; tests/
# frank_dragoon_unit.bats pins this arithmetic at 95 %.
# HALOGEN_MAX_TOKENS_CAP in frank-dragoon.env must stay >= this.
maxTokens: 45056
reasoning: true
@@ -305,3 +318,215 @@ providers:
name: "frank-tts (dankclaw) — cloned voices"
api: openai-speech
input: [text]
# Vendor cloud, not a fleet host: DeepSeek's own API. The label is the bare
# `deepseek` — NOT the house `<api> (<host>)` pattern — because the harness
# matches provider names against a built-in service table
# (`src/config/model-registry.ts` profiles, `dist/cli.js` `VIn` map), and
# `deepseekDirect` lists `providers: ["deepseek"]` with urlMarkers
# `["api.deepseek.com"]`. A name like `deepseek (platform)` matches neither
# form, and the row loses the direct-DeepSeek behaviour set the catalog ships
# for its own built-in provider (see the compat block below). The baseUrl
# carries the host the label can't name.
"deepseek":
baseUrl: https://api.deepseek.com/v1
api: openai-completions
# Command-backed, like every other credential in this file: the key lives in
# ~/.config/secrets/deepseek_api_key (600) and never enters the tracked
# tree, where the pre-commit shape gate would read it. Quoted because an
# unquoted leading `!` is a YAML tag. The resolved value is cached for the
# process lifetime — rotation is: rewrite the file, then restart omp.
# Written 2026-10-29 from the key Frank issued; every other workstation that
# pulls this repo needs its own copy of the file (nothing here carries it).
apiKey: "!cat ~/.config/secrets/deepseek_api_key"
models:
- id: deepseek-flash
name: "deepseek-flash"
# Official name (api-docs.deepseek.com): `deepseek-flash` is served by
# DeepSeek-V4.1-Flash-1017. The legacy names deepseek-v4-flash and
# deepseek-v4-flash-vision-exp are still accepted but retired — the
# model card says to use `deepseek-flash`; so does Frank.
# The model card states context_window 1M; 262,144 is its `max_tokens ≥
# 256K` recommendation. The two are the engine's own pair: any prompt +
# budget over 1,048,574 is a refusal, and this row states them together
# so the harness clamps before the API has to.
contextWindow: 1048576
maxTokens: 262144
# Natively multimodal (V4.1 = image-text-to-text on the hub), despite
# the retired `-vision-exp` alias suggesting otherwise.
input: [text, image]
# Rate card, per 1M tokens, peak tier ($0.15/$0.60 off-peak is not
# modelled — the harness has one field per direction and a session
# cannot know which window it is billing into). Overriding these here is
# the only way a non-catalog row reports cost at all.
cost:
input: 0.3
output: 1.2
cacheRead: 0.006
cacheWrite: 0
compat:
# Copied from the harness's own built-in descriptor for this provider
# (`pi-catalog/src/provider-models/openai-compat.ts:7422`), never
# simplified — the same law as the WAN rows:
# V4 thinking mode rejects the `tool_choice` parameter outright; tool
# calls still work and default to auto when tools are present.
supportsToolChoice: false
# V4 has no `developer` role.
supportsDeveloperRole: false
# Reasoning arrives in `reasoning_content`; the harness's default for a
# direct DeepSeek provider is `zai-thinking-disabled` (an explicit
# `thinking: {"type":"disabled"}`), which is what this endpoint
# honours — `reasoning_effort: "none"` is not accepted here. Left unset
# so the built-in default stays in charge rather than a guess.
reasoningContentField: "reasoning_content"
# The documented request enables thinking with the `thinking` toggle
# AND `reasoning_effort`; when thinking is on, the harness strips the
# toggle from extraBody and keeps the effort value, so the request
# carries what the endpoint expects.
# Card-recommended sampler: temperature 1.0, top_p 1.0 (in thinking
# mode the API floors top_p at 0.95 — 1.0 is inside the allowed set).
# top_k is not accepted through the OpenAI surface, so it is not sent.
# extraBody is merged last and wins, so pinning the trio survives a
# server-side sampler edit — same law as the LAN rows.
extraBody:
thinking:
type: enabled
temperature: 1.0
top_p: 1.0
# `max_tokens` is the field this API speaks (not
# max_completion_tokens) — set by the built-in default, and this row
# cannot mis-name it because it names no override of its own.
# ─── WAN door ───────────────────────────────────────────────────────────
# The fleet's own off-network door. Since 2026-10-29 the `deepseek` block
# above is also reachable from anywhere (a vendor URL, not a LAN IP), but it
# is not this door: this block is the route back to FRANK'S engines. Every
# fleet provider here is a LAN address, and `modelRoles` (config.yml) still
# points at those, so away from home you must NAME one of the rows below:
# omp --model 'openwebui (franksdank.cloud)/frank-dragoon:medium'
# The door is the OpenWebUI vhost `ai.franksdank.cloud` — Caddy → OWU on
# Dankclaw:3005, Anubis-exempt because OWU enforces its own auth, public A
# record present (LAN clients resolve 192.168.0.143 via Pi-hole, measured
# 2026-10-21). It exposes an OpenAI-compatible surface at /api/v1 and
# proxies to the SAME engines as the LAN rows — no second inference stack,
# so every engine-side law below still holds through it.
# CREDENTIAL LAW: the key is not in this file. `apiKey` accepts a
# command-backed value (`!cmd`, resolved by src/config/resolve-config-value.ts:
# stdout trimmed, cached for the process lifetime; anything else is tried as
# an env-var name, then taken literally), so the secret lives in
# ~/.config/secrets/openwebui_api_key, mode 600, like every other credential
# on these boxes. A literal `sk-…` here would trip the pre-commit shape gate
# and then live in every agent context on every box that pulls this repo.
# Rotate: rewrite the secret file, then RESTART omp — the `!cmd` result is
# cached per process, so a live session keeps serving the old key.
# WHAT THE DOOR SHOWS (probed 2026-10-21 with this key): /api/v1/models
# answers frank-dragoon and frank-zealot only. frank-archon is NOT published
# in OpenWebUI's picker and the API answers 400 "Model not found" for it, so
# the fleet's biggest endpoint has no WAN path until it is published
# server-side on Dankclaw — a fleet change, not a harness change.
# CONTRACT PROBED THROUGH THE DOOR (2026-10-21): streaming SSE carries
# `reasoning_content` deltas and `tool_calls` deltas, finish_reason
# `tool_calls`, and a final `usage` under stream_options.include_usage;
# `image_url` content parts reach both vision towers (a 64 px red PNG comes
# back "Red" from each); and `chat_template_kwargs` IS forwarded — proven on
# frank-zealot, where enable_thinking false gives reasoning 0 chars and true
# gives a real thinking block on the same prompt. Every compat block below is
# therefore a copy of its LAN twin, never a simplification: dropping the
# kwargs here would quietly change thinking behaviour on the WAN path.
# Chat completions made with an API key create NO OpenWebUI chat row (the
# chat list is unchanged after probing), so an off-network session leaves no
# trail in the family's chat history.
# NOT BEHIND THIS DOOR: speech. OWU does route /api/v1/audio/speech and it
# does reach frank-tts:7772, but OWU substitutes its own voice enum first —
# a cloned pack name comes back 400 from the upstream. `modelRoles.speech`
# stays on `frank-tts (dankclaw)`; do not repoint it here.
"openwebui (franksdank.cloud)":
baseUrl: https://ai.franksdank.cloud/api/v1
api: openai-completions
# Quoted because an unquoted leading `!` is a YAML tag, not a command.
apiKey: "!cat ~/.config/secrets/openwebui_api_key"
models:
- id: frank-dragoon
# Same halogen flash-server as `llamacpp (dankclaw)` above, one hop
# further out; the picker row and the engine share this id, so there
# is no alias to keep in sync.
name: "frank-dragoon (WAN)"
# Off-network the compaction hub must be reachable too: the LAN
# frank-zealot row is a dead address from anywhere but the house.
# Same engine, same slot arithmetic, so the keepRecent/threshold
# math in the LAN rows transfers untouched.
compactionModel: openwebui (franksdank.cloud)/frank-zealot
# Inherited from the engine, not invented: 200,000 under the native
# 262,144 ceiling (the 400-zone law — never sit at the ceiling). The
# proxy neither widens the window nor truncates: an over-ceiling
# request is still a 400.
contextWindow: 200000
# Same clamp arithmetic as the LAN row (see frank_dragoon_unit.bats);
# HALOGEN_MAX_TOKENS_CAP on the engine is unchanged.
maxTokens: 45056
reasoning: true
input: [text, image]
compat:
# Verified through the door: reasoning arrives as
# `reasoning_content` in the deltas, not folded into content.
reasoningContentField: "reasoning_content"
# `:off` must be a real switch, and it is: this engine honours
# reasoning_effort "none"; the harness default would only send the
# lowest effort, which still thinks.
reasoningDisableMode: none-effort
# Identity map, same reason as the LAN row — flash-server lists
# minimal|low|medium|high|xhigh|max|none and 400s anything else.
reasoningEffortMap:
minimal: minimal
low: low
medium: medium
high: high
xhigh: xhigh
max: max
# Pinned so a server-side HALOGEN_TOP_K edit cannot change this
# session's recipe; the engine accepts all three keys.
extraBody:
temperature: 1.0
top_p: 0.95
top_k: 20
- id: frank-zealot
# The 3090 endpoint, same llama.cpp engine behind the proxy. Stays
# the widest slot in the file, so it is the WAN compaction target.
name: "frank-zealot (WAN)"
compactionModel: openwebui (franksdank.cloud)/frank-zealot
# PER-SLOT window: llama.cpp divides --ctx-size across slots and
# answers a longer prompt with a 400, never a truncation. Follows
# every reslice on the engine (3 × 200,704 today).
contextWindow: 200704
# Prompt + thinking budget inside the slot; compaction fires at 85 %
# of the window on this row set (see the LAN twin for the full
# worst-case arithmetic).
maxTokens: 16384
reasoning: true
input: [text, image]
compat:
reasoningContentField: "reasoning_content"
# LOAD-BEARING, exactly as on the LAN row: the Bonsai-2 template
# accepts ONLY xhigh|medium|low and raises on anything else, so an
# unmapped harness level is a hard failure on the role's first
# request. Probed 2026-10-21 that the mapped value, not the
# harness-level, is what reaches the template through this proxy.
reasoningEffortMap:
minimal: low
low: low
medium: medium
high: xhigh
max: xhigh
# The payload's own sampling recipe plus the thinking switches.
# chat_template_kwargs survives the proxy (verified), so
# preserve_thinking keeps prior assistants' thinking blocks in
# history for multi-turn tool traffic — same contract as at home.
extraBody:
temperature: 1.0
top_p: 0.95
top_k: 20
min_p: 0.0
presence_penalty: 0.0
chat_template_kwargs:
enable_thinking: true
preserve_thinking: true
+2
View File
@@ -24,4 +24,6 @@ input {
focus-follows-mouse // Automatically focus windows under the mouse pointer
workspace-auto-back-and-forth // Enable workspace back & forth switching
trackpoint
}
Loaded 100 of 116 files, more files were not shown because too many files have changed in this diff. Show more