Compare commits
177
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dcdf84719c | ||
|
|
8c3dc6bfc9 | ||
|
|
27da5cd9ec | ||
|
|
ec9bf77b10 | ||
|
|
6a12ffaef4 | ||
|
|
d33ee26e67 | ||
|
|
b8f66e22b2 | ||
|
|
07768ce8b5 | ||
|
|
d845160e62 | ||
|
|
3ab48bf1f2 | ||
|
|
51f9846ac6 | ||
|
|
4085ee403f | ||
|
|
1dc01bd006 | ||
|
|
de7dd6f897 | ||
|
|
64bb23b329 | ||
|
|
336201e814 | ||
|
|
e0b1466f74 | ||
|
|
d368394656 | ||
|
|
ec19771b8b | ||
|
|
8b87b5ae1a | ||
|
|
047f264fcf | ||
|
|
54bb0202c2 | ||
|
|
fb36a1b0fd | ||
|
|
c37ad6e870 | ||
|
|
c6e2a1daca | ||
|
|
88d486b37b | ||
|
|
49f67b2192 | ||
|
|
e49537c2ac | ||
|
|
62a70caebb | ||
|
|
4735702965 | ||
|
|
22d2f3bcd8 | ||
|
|
205b54adcb | ||
|
|
8af40c402b | ||
|
|
6e89c63e5d | ||
|
|
e690a2ca66 | ||
|
|
d4d6dd0c44 | ||
|
|
66e1129e6c | ||
|
|
a280c29d2c | ||
|
|
8393dc1090 | ||
|
|
d0587a5bc5 | ||
|
|
334e5d0ee0 | ||
|
|
66fc40de65 | ||
|
|
71601f72a8 | ||
|
|
f3ce42eb80 | ||
|
|
812be6a0a6 | ||
|
|
06f45e83c1 | ||
|
|
40b9725433 | ||
|
|
671222b881 | ||
|
|
71a7e5cb9a | ||
|
|
97b65067b1 | ||
|
|
23372e96f5 | ||
|
|
ab94c91bcf | ||
|
|
5cf7e6bd8e | ||
|
|
1cc4b76479 | ||
|
|
59920f849d | ||
|
|
bfd6a6340e | ||
|
|
4849f46eb2 | ||
|
|
dd91ef1d59 | ||
|
|
289f74062a | ||
|
|
d7bd05398d | ||
|
|
004962ffce | ||
|
|
56fc899763 | ||
|
|
35ceae6d7b | ||
|
|
655a39ffb6 | ||
|
|
b4fc881164 | ||
|
|
6df45d709a | ||
|
|
5ab0a49ea4 | ||
|
|
4666d0dce0 | ||
|
|
d013795922 | ||
|
|
2b7b0bc3cd | ||
|
|
3809e1ed22 | ||
|
|
e278e4b863 | ||
|
|
c05f5ad451 | ||
|
|
ecc089c6ef | ||
|
|
7523ca5c19 | ||
|
|
3022277167 | ||
|
|
689ada428f | ||
|
|
9ebf5fe85f | ||
|
|
86280608e5 | ||
|
|
411b32ff59 | ||
|
|
04f93133fc | ||
|
|
7da5e9dfa8 | ||
|
|
0d43eb0948 | ||
|
|
5110060c24 | ||
|
|
6f1f55345a | ||
|
|
6af3714bd0 | ||
|
|
495e2229f5 | ||
|
|
37ef4897ae | ||
|
|
06e549cb27 | ||
|
|
c96741794a | ||
|
|
e14d0906ea | ||
|
|
d5be16127b | ||
|
|
b7aef40266 | ||
|
|
fafdaa2617 | ||
|
|
7213068d91 | ||
|
|
05dbe8c5c1 | ||
|
|
806352f8f6 | ||
|
|
34a7af2847 | ||
|
|
635c64d773 | ||
|
|
64d9c54657 | ||
|
|
aec7f0f068 | ||
|
|
4023b21365 | ||
|
|
e14fc884a3 | ||
|
|
b3a84c5bb8 | ||
|
|
9e6254108a | ||
|
|
ce87958b77 | ||
|
|
802014fea9 | ||
|
|
e22acf4913 | ||
|
|
c14c64d6c0 | ||
|
|
6e3e5594eb | ||
|
|
36ff3742b4 | ||
|
|
fd6a07bb58 | ||
|
|
1921d76319 | ||
|
|
d302151a8d | ||
|
|
5b6f4381b0 | ||
|
|
cae49fcd09 | ||
|
|
ee292ffd10 | ||
|
|
dd4713fa81 | ||
|
|
a8727ddf0b | ||
|
|
2faeab665a | ||
|
|
62b3b92e69 | ||
|
|
ec6a707fe4 | ||
|
|
96553979fb | ||
|
|
ac5b377750 | ||
|
|
8f5a229cc6 | ||
|
|
ea514c8fd2 | ||
|
|
bc60fa33be | ||
|
|
e4ecc93f29 | ||
|
|
d055eb1e7f | ||
|
|
70346f3d57 | ||
|
|
1251cfbd40 | ||
|
|
e67bbd0de0 | ||
|
|
986081bf1d | ||
|
|
a9b3c495da | ||
|
|
c99eb8e11b | ||
|
|
cd20d6ddfd | ||
|
|
82dd0c7d09 | ||
|
|
99b9a6a17c | ||
|
|
39385636be | ||
|
|
d99c1c3c4f | ||
|
|
651924a4bc | ||
|
|
e86e60af39 | ||
|
|
01d9d3fbd5 | ||
|
|
570b5509d0 | ||
|
|
3579c7ebc5 | ||
|
|
0075ca3bbf | ||
|
|
bcd52bb0f4 | ||
|
|
edd7e0cf68 | ||
|
|
f033639b54 | ||
|
|
6fffd83957 | ||
|
|
dd8f3974c1 | ||
|
|
acf67182e9 | ||
|
|
38bf9c4897 | ||
|
|
54f15e2a58 | ||
|
|
1e7e212e5c | ||
|
|
475211d2e7 | ||
|
|
0258ef70f8 | ||
|
|
0c7fe8b4bf | ||
|
|
765b686859 | ||
|
|
332067a516 | ||
|
|
66e3f220e7 | ||
|
|
6134e1168e | ||
|
|
c379d16858 | ||
|
|
ca8b46241e | ||
|
|
3fa0242f56 | ||
|
|
fe039a0b73 | ||
|
|
d4092f2f65 | ||
|
|
cedef97c99 | ||
|
|
0ee08bc368 | ||
|
|
6664f9090a | ||
|
|
7de07ee35c | ||
|
|
fcc7226c3b | ||
|
|
a8011c3d4d | ||
|
|
08f90c6df3 | ||
|
|
57e255f930 | ||
|
|
b3d3df4bdb | ||
|
|
5d523f6e9f |
@@ -1,3 +1,13 @@
|
||||
# LFS routing for image types. Measured 2026-10-21: this repo tracks ZERO LFS files
|
||||
# (`git lfs ls-files` = 0) — the wallpaper library became live theme state and is gitignored
|
||||
# (omp-docs/tools/dotfiles-deploy.md), so these four rules match nothing today. They are kept
|
||||
# because tracked media is a plausible future and idle filters cost nothing; deleting the file
|
||||
# is a pruning call for a session with Frank, not a cleanup an agent runs alone.
|
||||
#
|
||||
# Do not cite this file as the reason `git-lfs` is installed on a box. The manifest carries its
|
||||
# own justification (packages/common.pkgs), and a tool justified by dead config is a tool nobody
|
||||
# can defend when the config goes.
|
||||
|
||||
*.jpg filter=lfs diff=lfs merge=lfs -text
|
||||
*.jpeg filter=lfs diff=lfs merge=lfs -text
|
||||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
+130
-10
@@ -1,25 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
# Secret gate: block any commit whose staged diff contains live-secret shapes.
|
||||
# This repo is pushed to a Gitea remote; anything committed is instantly visible
|
||||
# Secret gate: block any commit whose staged diff contains live-secret shapes or
|
||||
# credential assignments. This repo IS the deployment and every workstation
|
||||
# pulls it, so a committed secret is instantly in every agent's context window
|
||||
# on the fleet. Deliberate override: ALLOW_SECRETS=1 git commit ...
|
||||
#
|
||||
# Two rules, deliberately different in scope:
|
||||
#
|
||||
# 1. Token SHAPES are scanned across the ENTIRE staged diff, whatever the file
|
||||
# type. The first version of this gate whitelisted extensions
|
||||
# (*.md *.yml *.yaml *.json *.toml *.sh *.ts *.js), so a `postgres.env`, a
|
||||
# `.container` quadlet, an `.ini`, or an extension-less dotfile — exactly
|
||||
# the shapes the dotfiles tier is made of — was never read at all.
|
||||
# 2. Credential ASSIGNMENTS (PASSWORD=…, secret_key: …) are scanned only in
|
||||
# non-prose files. The auto-learned memory tier is prose ABOUT credentials,
|
||||
# and a gate that fires there gets overridden until it means nothing; the
|
||||
# 2026-10-20 audit measured 7 such prose hits on the tracked tree. Markdown
|
||||
# keeps shape detection only, which is what it had before.
|
||||
#
|
||||
# Bcrypt hashes and Basic-auth headers joined the shape list on 2026-10-20
|
||||
# (issue #40) after a live Caddy basicauth hash plus its plaintext password —
|
||||
# and the Pi-hole internal password — were found in a tracked memory file.
|
||||
set -uo pipefail
|
||||
|
||||
staged=$(git diff --cached --diff-filter=ACM -U0 -- '*.md' '*.yml' '*.yaml' '*.json' '*.toml' '*.sh' '*.ts' '*.js')
|
||||
[ -z "$staged" ] && exit 0
|
||||
shapes='sk-[A-Za-z0-9]{18,}|ghp_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[A-Z0-9]{16}|ASIA[A-Z0-9]{16}|BEGIN [A-Z ]*PRIVATE KEY|aes_[A-Za-z0-9/+]{40}|eyJ[A-Za-z0-9_-]{20,}\.ey|\$2[aby]\$[0-9]{2}\$[./A-Za-z0-9]{20,}|Authorization:[[:space:]]*[Bb]asic[[:space:]]+[A-Za-z0-9+/=]{24,}'
|
||||
|
||||
patterns='sk-[A-Za-z0-9]{18,}|ghp_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[A-Z0-9]{16}|ASIA[A-Z0-9]{16}|BEGIN [A-Z ]*PRIVATE KEY|aes_[A-Za-z0-9/+]{40}|eyJ[A-Za-z0-9_-]{20,}\.ey'
|
||||
# `newpass1234`, `secret_key: "rotated-value"` — an assignment with a literal
|
||||
# value. `$VAR`, `${VAULT}`, <placeholders> and empty values cannot match.
|
||||
assign='[A-Za-z0-9_]*(PASSW|SECRET|TOKEN|API_?KEY|ACCESS_KEY|PRIVATE_KEY|CREDENTIAL)[A-Za-z0-9_]*[[:space:]]*[:=][[:space:]]*["'"'"']?[A-Za-z0-9/+._@-]{8,}'
|
||||
placeholders='changeme|change[-_]|placeholder|redacted|example|sample|dummy|nokey|no[-_]key|not[-_]needed|your[-_]|localhost|xxxx|yyyy|\$\{|<[a-z]|\*\*\*|%s|null|false|true'
|
||||
|
||||
# Exemptions: a variable naming a location (`…_FILE=`, `…_PATH=`) or a value
|
||||
# that is a filesystem path is a pointer, not a credential. Flagging those is
|
||||
# how a gate gets permanently overridden.
|
||||
assign_exempt='(_FILE|_PATH|_DIR|_URL|_ENDPOINT|_SOCKET)[[:space:]]*[:=]|[:=][[:space:]]*["'"'"']?(/|\.\./)'
|
||||
# Fail closed: a gate that cannot read the staged diff must not report success.
|
||||
#
|
||||
# `-z`, and the list lives in a file. Two traps decide this: git quotes any path it cannot print
|
||||
# as UTF-8 (core.quotePath is on by default), and a quoted name handed back to
|
||||
# `git diff -- <path>` matches nothing — the file is skipped, and a skipped file used to be a
|
||||
# silent pass. Command substitution cannot hold the list either, because $() deletes NUL bytes,
|
||||
# so the names go to a temp file and the loop reads them with a NUL delimiter.
|
||||
TMPD=$(mktemp -d) || TMPD=""
|
||||
if [ -z "$TMPD" ]; then
|
||||
echo "SECRET GATE: cannot create a temp dir — refusing to commit." >&2
|
||||
[ -n "${ALLOW_SECRETS:-}" ] && exit 0
|
||||
exit 1
|
||||
fi
|
||||
trap 'rm -rf "$TMPD"' EXIT
|
||||
if ! git diff --cached --diff-filter=ACM --name-only -z > "$TMPD/files" 2> "$TMPD/err"; then
|
||||
{
|
||||
echo "SECRET GATE: cannot read the staged diff — refusing to commit."
|
||||
sed 's/^/ /' "$TMPD/err"
|
||||
echo "Override deliberately: ALLOW_SECRETS=1 git commit ..."
|
||||
} >&2
|
||||
[ -n "${ALLOW_SECRETS:-}" ] && exit 0
|
||||
exit 1
|
||||
fi
|
||||
# git's stderr stays out of the name list on purpose: a `warning:` line mistaken for a path
|
||||
# becomes a file that "cannot be read", which is how a gate starts skipping the real ones.
|
||||
[ -s "$TMPD/files" ] || exit 0
|
||||
|
||||
report=""
|
||||
exempt=""
|
||||
count=0
|
||||
while IFS= read -r -d '' f; do
|
||||
[ -z "$f" ] && continue
|
||||
# Self-exemption, deliberately an exact filename list and not `tests/`: the
|
||||
# gate's fixtures must contain credential-shaped strings or there is nothing to
|
||||
# test against, and this hook documents the shapes it looks for. A secret parked
|
||||
# in any other test file is still caught.
|
||||
case "$f" in
|
||||
.githooks/pre-commit|tests/secret-gate.bats) exempt="$exempt $f"; continue ;;
|
||||
esac
|
||||
diff=$(git diff --cached -U0 -- "$f")
|
||||
[ -z "$diff" ] && continue
|
||||
# Only ADDED lines can leak. Matching removals too blocks the very commit that
|
||||
# deletes a credential — which is exactly what happened on 2026-10-20, when the
|
||||
# gate refused the redaction of the password it exists to catch.
|
||||
added=$(printf '%s\n' "$diff" | grep -E '^\+' | grep -vE '^\+\+\+' | sed 's/^+//' || true)
|
||||
[ -z "$added" ] && continue
|
||||
# Shape hits are judged on the MATCHED TOKEN, not on the whole line. Documentation about
|
||||
# credentials quotes vendor placeholders — AWS publishes an all-zeros access-key id that ends
|
||||
# in EXAMPLE for exactly this purpose — and
|
||||
# a gate that blocks prose over an inert example gets worked around until it means nothing:
|
||||
# the live-sync heartbeat failed three ticks on 2026-10-21 because a managed skill page quotes
|
||||
# exactly that id. A line is exempted only when EVERY shape on it is inert, so one live token
|
||||
# sharing the file with a documented fake still blocks.
|
||||
hits=$(printf '%s\n' "$added" | while IFS= read -r line; do
|
||||
live=0
|
||||
while IFS= read -r m; do
|
||||
# The shape regexes carry fixed quantifiers, so the match they return is TRUNCATED:
|
||||
# `AKIA` + exactly 16 characters stops inside the word EXAMPLE and the placeholder
|
||||
# disappears from what is judged. Judge the match plus what follows it instead.
|
||||
ctx=$(printf '%s' "$line" | awk -v m="$m" '{ i = index($0, m); print (i > 0 ? substr($0, i, length(m) + 8) : m) }')
|
||||
printf '%s' "$ctx" | grep -Eiq 'EXAMPLE|X{4,}|Y{4,}|Z{4,}|_{4,}|REDACTED|CHANGEME|PLACEHOLDER' || live=1
|
||||
done < <(printf '%s\n' "$line" | grep -Eio "$shapes" || true)
|
||||
[ "$live" = 1 ] && printf '%s\n' "$line"
|
||||
done)
|
||||
if [ "${f##*.}" != "md" ]; then
|
||||
# The placeholder filter reads the VALUE, never the whole line. A trailing comment that
|
||||
# says `# example` says nothing about the token the same line assigns; scoping the filter
|
||||
# to what follows the separator is what keeps `API_TOKEN=<real> # placeholder` a block.
|
||||
assigns=$(printf '%s\n' "$added" | grep -Ei "$assign" | grep -Ev "$assign_exempt" \
|
||||
| while IFS= read -r line; do
|
||||
# The judged value is the assigned token alone: it ends at whitespace or `#`, so a
|
||||
# trailing comment cannot vouch for it. Truncating a quoted value early only ever errs
|
||||
# toward blocking, which is the safe direction for a gate that can be overridden.
|
||||
value=${line#*[:=]}
|
||||
value=${value#"${value%%[![:space:]]*}"}
|
||||
value=${value%%[[:space:]]*}
|
||||
value=${value%%#*}
|
||||
printf '%s\n' "$value" | grep -Eiq "$placeholders" && continue
|
||||
printf '%s\n' "$line"
|
||||
done)
|
||||
hits=$(printf '%s\n%s\n' "$hits" "$assigns" | sed '/^[[:space:]]*$/d')
|
||||
fi
|
||||
if [ -n "$hits" ]; then
|
||||
n=$(printf '%s\n' "$hits" | wc -l)
|
||||
count=$((count + n))
|
||||
report+=" $f"$'\n'"$(printf '%s\n' "$hits" | cut -c1-100 | sed 's/^/ /')"$'\n'
|
||||
fi
|
||||
done < "$TMPD/files"
|
||||
|
||||
# Say so out loud: an exemption that prints nothing is how a carve-out becomes
|
||||
# invisible, and invisible is the property a secret wants.
|
||||
if [ -n "$exempt" ]; then
|
||||
printf 'secret-gate: self-test fixtures exempt:%s\n' "$exempt" >&2
|
||||
fi
|
||||
|
||||
hits=$(printf '%s\n' "$staged" | grep -Ei "$patterns" || true)
|
||||
if [ -n "${ALLOW_SECRETS:-}" ]; then
|
||||
[ -n "$hits" ] && printf 'secret-gate: ALLOW_SECRETS set, allowing %s secret-shaped string(s) through.\n' "$(printf '%s\n' "$hits" | wc -l)"
|
||||
[ -n "$report" ] && printf 'secret-gate: ALLOW_SECRETS set, allowing %s secret-shaped string(s) through.\n' "$count"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ -n "$hits" ]; then
|
||||
if [ -n "$report" ]; then
|
||||
{
|
||||
echo "SECRET GATE: staged diff contains $(printf '%s\n' "$hits" | wc -l) secret-shaped string(s):"
|
||||
printf '%s\n' "$hits" | sed 's/^/ /'
|
||||
echo "SECRET GATE: staged diff contains $count secret-shaped string(s):"
|
||||
printf '%s\n' "$report"
|
||||
echo "Fix it, or override deliberately: ALLOW_SECRETS=1 git commit ..."
|
||||
echo "A credential that reaches a tracked file is already in history on every"
|
||||
echo "machine that pulls this repo — deleting it later does not un-leak it."
|
||||
} >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
+36
@@ -26,6 +26,10 @@ auth.json
|
||||
**/auth.json
|
||||
.env
|
||||
**/.env
|
||||
*.env
|
||||
# `*.env` is the rule that matters: `.env` alone does not match `postgres.env`
|
||||
# or `frank-job-hunt.env`, and those are the files that actually carry
|
||||
# POSTGRES_PASSWORD / API keys on this fleet (issue #40).
|
||||
trust.json
|
||||
**/trust.json
|
||||
*.pem
|
||||
@@ -74,3 +78,35 @@ wt/
|
||||
# A session run inside a git worktree mints a bucket named after the worktree
|
||||
# path; the worktree is deleted, the bucket is orphan state, not config.
|
||||
agent/memories/--home-blank-.omp-wt-*
|
||||
|
||||
# --- theme media: content the pipeline consumes, not configuration ---------
|
||||
# 23 wallpapers, 80 MiB. Tracked until 2026-10-20: every clone of a config repo was
|
||||
# carrying a library no diff can review. The live library at
|
||||
# ~/.local/share/noctalia/wallpapers is a real directory of real files (measured
|
||||
# 2026-10-20), so this path in the repo was a duplicate of it, not its source — and
|
||||
# note that untracking deletes the copy in every other checkout regardless of
|
||||
# `git rm --cached` on the branch. A fresh box starts with an empty library; every
|
||||
# filename carries its wallhaven ID; `/w/<id>` answers `text/html`, so resolve the CDN URL at
|
||||
# `https://wallhaven.cc/api/v1/w/<id>` and download `data.path` — the recipe, with the reason it
|
||||
# is not a plain `curl -L`, is on omp-docs/tools/dotfiles-deploy.md
|
||||
dotfiles/noctalia/.local/share/noctalia/wallpapers/
|
||||
|
||||
# --- generated theme output: machine-written, never reviewable, never deployed ---
|
||||
# noctalia/matugen rewrite these in place; a tracked copy is a stale snapshot that
|
||||
# deploy.sh would push back over a live palette. These patterns are the ignore half of the
|
||||
# exclusion list in `deploy_file` (scripts/deploy.sh): change one, change the other, or a
|
||||
# generated file becomes tracked-but-never-deployed — the fiction class #41 removed. The
|
||||
# capital-N rule is noctalia's own palette-directory naming, and it was in deploy.sh only.
|
||||
# Law: omp-docs/inventory/home-config-census.md, Refs #42, #60
|
||||
dotfiles/**/themes/noctalia*
|
||||
dotfiles/**/colors/noctalia*
|
||||
dotfiles/**/noctalia.*
|
||||
dotfiles/**/noctalia-*
|
||||
dotfiles/**/Noctalia*
|
||||
|
||||
# --- harness natives: unpacked runtime payloads, never configuration ----------
|
||||
# 2.4 MB of tree-sitter grammars the harness unpacks into the repo root on first use (measured
|
||||
# on blankpad 2026-10-21). Regenerable, unreviewable, not config — every clause of AGENTS.md's
|
||||
# runtime-state doctrine applies, and it sat unnamed in the root, one `git add -A` away from
|
||||
# being committed as if it were configuration.
|
||||
natives/
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
---
|
||||
name: command-center-dotfiles-import
|
||||
description: Runbook for importing client env-layer configs into the command-center repo and deploying them via stow on new fleet machines; decisions agreed with Frank 2026-09-09 (pre-execution — verify execution state before assuming done).
|
||||
---
|
||||
|
||||
Design agreed with Frank for importing client application/environment configs into the `~/.omp` command-center repo and deploying them on fresh machines. **Status: plan agreed, not yet executed** — check repo state (does `dotfiles/` or `scripts/deploy.sh` exist?) before re-deriving; if already implemented, this becomes the operating model, not a to-do.
|
||||
|
||||
## Agreed decisions (each was an explicit user choice)
|
||||
1. **Target**: unknown/TBD fresh machine → design stays host-agnostic; deploy derives overrides from `$(hostname)` at run time. Nothing hardcodes a target.
|
||||
2. **Scope**: environment layer only — niri (+cfg includes), gtk-3.0/4.0 settings, ghostty, alacritty, shell rc files (`.zshrc`, `.bashrc`, `.profile`, `.p10k.zsh`), fish, systemd user units, autostart, dconf. App *data* (browser profiles, vaults, DBs) never imported.
|
||||
3. **Layout**: stow-style package dirs `dotfiles/<pkg>/<home-relative path>` inside the repo; repo clones to `~/.omp` on every host that runs the harness (full command-center clone, not env-layer only).
|
||||
4. **Deploy mechanism**: symlink farm — `stow -t ~` from `~/.omp/dotfiles`; relative symlinks within `$HOME` work because repo root lives inside `$HOME`.
|
||||
5. **Dogfood order**: convert the workstation itself (real dirs → symlinks via adopt) BEFORE any target import; iterate where the compositor reload is observable.
|
||||
6. **Secrets policy**: never commit secret values. Split rc files into committed logic + untracked sourced-if-exists stub (`~/.config/environment.d/secrets.env` or `.zshrc.local`); real values provisioned out-of-band only on machines that need them. Pre-commit secret scanner stays as safety net, not gate of first resort.
|
||||
7. **Divergence**: `zz-<hostname>` override package stowed LAST with `--override`; only per-host diffs live there. (Stow descends into conflicting symlinked dirs — per-file overrides only work via this last-stowed `--override` pattern.)
|
||||
8. **Deploy trigger**: explicit `scripts/deploy.sh` invocation, human- or agent-invoked. NO auto-deploy on pull — pull and deploy are separate decisions; `--check` = drift report mode.
|
||||
9. **Ownership**: Frank is sole push authority; exactly one editing agent at a time → no multi-writer race machinery needed. Each host's agent edits only its own `zz-<hostname>` package.
|
||||
10. **Conflict policy**: pre-existing regular file that differs from repo truth → move to timestamped `.quarantine/` dir on the target (never `rm`, same convention as this repo's trash rule), symlink anyway, print every replacement in deploy output and `--check` report. Repo truth always wins; divergence recoverable from quarantine.
|
||||
11. **First-clone auth**: human types credential once console-side (URL-with-token or prompt); the new machine's secrets exist only there afterward.
|
||||
|
||||
## Implementation gotchas baked into the design
|
||||
- `core.hooksPath` is a per-clone local setting — does NOT survive clone; bootstrap/deploy must set it on every fresh clone or the secret-gate pre-commit hook silently doesn't run there.
|
||||
- dconf/gsettings are databases, not files: import as `dconf dump /` text into repo; deploy runs `dconf load` instead of symlinking.
|
||||
- p10k installer and toolchain hooks append to rc files at install time — expect conflicts on `.zshrc`-class files at first adopt; quarantine-and-replace handles it, but diff live-vs-repo before adopting so pre-existing drift is seen, not silently adopted.
|
||||
- Known secret at import time: `GITEA_TOKEN` export lived in `.zshrc` (line ~22) on the workstation — quarantine BEFORE importing rc files; consider token rotation at cutover since it had been exposed in plaintext rc + remote URL.
|
||||
|
||||
## Deploy script spec agreed
|
||||
`scripts/deploy.sh`: sets hooksPath → stows all base packages → stows `zz-$(hostname)` with `--override` → first run uses `--adopt` for pre-existing identical files → divergent files quarantined-and-reported; `--check` mode emits drift report (modified tracked files vs deployed symlinks) without mutating.
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
name: stash-supersession-audit-before-drop
|
||||
description: "Before dropping a stale git stash, prove it's superseded by HEAD via per-file line-set diff of git show stash@{0}:file vs HEAD; archive as an annotated tag and drop only after auditing unique facts — zero-loss alternative to blind drop."
|
||||
---
|
||||
|
||||
# Stash Supersession Audit Before Drop
|
||||
|
||||
Use when a repo (especially a live-deployment checkout where the working tree IS the deployment, e.g. `~/.omp` command-center) has an old stash whose content may have been superseded by later commits, and dropping blindly risks losing unique facts.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Inventory the stash** — see which files it touches:
|
||||
```bash
|
||||
git stash show --stat stash@{0}
|
||||
```
|
||||
2. **Probe for unique content per file.** For each touched file, diff stash version against HEAD and count lines that exist only in the stash version:
|
||||
```bash
|
||||
git show "stash@{0}:<path>" > /tmp/stash_copy
|
||||
# compare normalized line sets; report lines present in stash but absent from HEAD
|
||||
```
|
||||
A file with `unique-vs-HEAD = 0` is fully superseded — safe to drop for that file.
|
||||
3. **Classify each unique-content hit**:
|
||||
- *Stale fact* (superseded by a later refactor, e.g. old model/provider naming) → drop is correct.
|
||||
- *Live durable fact* (e.g. an operational invariant never re-recorded) → extract it into the live memory/docs file or flag to the user before dropping.
|
||||
4. **Archive before dropping** — zero-loss: tag the stash commit, then drop from the stash list:
|
||||
```bash
|
||||
git tag archive/stash-<YYYY-MM-DD>-<base-branch>-wip "stash@{0}"
|
||||
git stash drop "stash@{0}"
|
||||
```
|
||||
The tag pins the full stash object (including its index parent); recoverable later via `git show archive/stash-...` or `git checkout archive/stash-...^3 -- <path>` for the untracked-files third parent if present.
|
||||
|
||||
## Why
|
||||
Stashes are invisible in normal history and expire from reflogs (~90 days). A stash that *looks* superseded can still hold the only copy of a durable fact; tagging converts "drop" into a reversible archive with zero workflow cost.
|
||||
@@ -86,29 +86,61 @@ never committed, only surfaced. Conflicts abort and alert (ntfy); it never
|
||||
force-pushes, auto-resolves, or uses `--no-verify`. Details:
|
||||
[omp-docs/tools/live-sync.md](omp-docs/tools/live-sync.md).
|
||||
|
||||
## Agent skills
|
||||
|
||||
The engineering skills (`triage`, `to-tickets`, `to-spec`, `research`,
|
||||
`implement`, …) read their per-repo configuration from three pages in the
|
||||
knowledge base. Those pages own their facts; this is only the pointer.
|
||||
|
||||
### Issue tracker
|
||||
|
||||
See [`omp-docs/skills/issue-tracker.md`](omp-docs/skills/issue-tracker.md).
|
||||
|
||||
### Triage labels
|
||||
|
||||
See [`omp-docs/skills/triage-labels.md`](omp-docs/skills/triage-labels.md).
|
||||
|
||||
### Domain docs
|
||||
|
||||
ADRs: `omp-docs/decisions/`. `omp-docs/glossary.md` is intentionally not created.
|
||||
See [`omp-docs/skills/domain-docs.md`](omp-docs/skills/domain-docs.md).
|
||||
|
||||
## Secret gate (HARD rule)
|
||||
|
||||
The pre-commit hook (`.githooks/pre-commit`, enabled via
|
||||
`git config core.hooksPath .githooks`) scans every staged diff for live-secret
|
||||
shapes (`sk-…`, `ghp_…`, `github_pat_…`, `glpat-…`, `xox…`, `AKIA…`, PEM headers,
|
||||
JWTs) and blocks the commit. Quarantine list — never tracked, ever: `auth.json`,
|
||||
`.env`, `trust.json`, `*.pem`, `*.key`, `identity*`. Deliberate override:
|
||||
`git config core.hooksPath .githooks`) blocks the commit in two ways. Token
|
||||
**shapes** (`sk-…`, `ghp_…`, `github_pat_…`, `glpat-…`, `xox…`, `AKIA…`, PEM
|
||||
headers, JWTs, bcrypt hashes, `Authorization: Basic` headers) are scanned across
|
||||
the **entire** staged diff — every file type, because the first version of the
|
||||
gate whitelisted extensions and never read a `.env`, `.container`, `.ini`, or
|
||||
extension-less dotfile at all. Credential **assignments** (`PASSWORD=…`,
|
||||
`secret_key: …`) are scanned in non-prose files only; markdown keeps shape
|
||||
detection, because the auto-learned memory tier is prose *about* credentials and a
|
||||
gate that fires there gets overridden until it means nothing. The gate **fails
|
||||
closed**: if it cannot read the staged diff it refuses the commit rather than
|
||||
passing it. Quarantine list — never tracked, ever: `auth.json`, `.env` and
|
||||
`*.env` (`postgres.env`, `frank-job-hunt.env` are the ones that carry real
|
||||
values), `trust.json`, `*.pem`, `*.key`, `identity*`. Deliberate override:
|
||||
`ALLOW_SECRETS=1 git commit …`. Remember why: a secret in a tracked file is a
|
||||
secret in every agent's context window on every machine that pulls this repo.
|
||||
secret in every agent's context window on every machine that pulls this repo, and
|
||||
redacting it later does not un-leak the history.
|
||||
|
||||
## Machine facts (verify before trusting)
|
||||
|
||||
| box | role | GPU | notes |
|
||||
|-----|------|-----|-------|
|
||||
| Dev workstation | client box, never runs inference | laptop iGPU + dGPU (non-unified) | mainline llama.cpp at `~/GitPackages/llama.cpp/build/bin/`; this repo lives here at `~/.omp` |
|
||||
| Dev workstation (`blankpad`) | client box, never runs inference | Lenovo laptop, AMD Ryzen 5 PRO 5650U / Cezanne APU (**unified memory**, s2idle sleep only, no dGPU) | this repo lives here at `~/.omp`; the earlier "iGPU + dGPU / `~/GitPackages/llama.cpp`" description belonged to the sold `dankware` box |
|
||||
| Dankclaw | bench target, secondary models + service stack | Strix Halo APU (**unified memory**) + RTX 3090 (moved over from Dankserver 2026-09) | prod services `frank-dragoon` (halogen-flash-server, :7770, iGPU) and `frank-zealot` (llama.cpp PrismML fork, :7771, 3090); runbooks in the serverAppConfigs vault (`~/Projects/serverAppConfig/serverAppConfigs-docs/LLM/`); ssh: `ssh dankclaw` |
|
||||
| Dankserver | main-model box | 2× Radeon AI PRO R9700 — non-unified (was RTX 3090 until 2026-09) | runs `frank-archon` (radiance, TP=2, :7770) — **this repo's operator session runs on it**; ssh: `ssh dankserver` |
|
||||
|
||||
## Environment gotchas
|
||||
|
||||
- Gitea API needs a token: `GITEA_TOKEN` env (gitignored `.env`). No `tea` CLI
|
||||
installed anywhere. Remote URLs embed the token locally in `.git/config` — that
|
||||
file is never pushed; it is not a substitute for the secret gate.
|
||||
installed anywhere. Remote URLs carry NO embedded token (2026-10-09
|
||||
credential-store cutover) — git and the API both read
|
||||
`~/.config/secrets/gitea` (`store --file`, URL-line format; see
|
||||
`omp-docs/tools/gitea/client-access.md`). That file is never pushed; it is
|
||||
not a substitute for the secret gate.
|
||||
- Passwordless sudo (`sudo -n`) works on Dankclaw and Dankserver for systemctl
|
||||
verbs only; user units need no sudo at all.
|
||||
- SSH into both servers is BatchMode-capable from here:
|
||||
|
||||
+29
-3
@@ -54,13 +54,27 @@ Suggest these tools or languages as possible solutions to problems if they are a
|
||||
|
||||
### 1. Maintain your projects source of truth.
|
||||
|
||||
The knowledge base at `docs/` is the source of truth for your project. Your training is out of date. Default to the documentation and the code before you assume anything. Maintain a work log of actions taken and their result. Refer to the work log to remember the progress of your tasks.
|
||||
Documentation exists for ONE reason: **point of use**. A page earns its bytes only if it helps you perform a task at the moment you perform it — the shape to deploy, the flag that must be passed, the failure mode to recognise, the rollback to run. It is NOT a museum, NOT a lab notebook, and NOT a place to keep track of numbers.
|
||||
|
||||
**THE PURPOSE TEST — apply it before you write, and again before you keep:**
|
||||
- Would an agent or Frank, doing a real task tomorrow, read this line and act differently because of it? If no: **delete it.**
|
||||
- Is this a measured number (t/s, tokens, ms, MiB, %, bench ladder, acceptance rate) for a model, runtime, or hardware configuration that is NOT currently deployed and NOT currently being compared? Then it is **dead weight — destroy it**, do not archive it, do not "preserve it in an Eras row", do not cite it. AI advances make such a figure wrong within weeks, and a wrong number in a runbook is worse than no number: it gets believed.
|
||||
- "It is the only copy of that fact" is NOT a reason to keep it. Sole custody of a useless fact is still useless. Losing information is an acceptable — often correct — outcome; do not hesitate to lose it.
|
||||
- What survives the test: live configuration and its exact numbers, the procedure that must be followed, the law/causal story that prevents a repeat failure, security posture, and the interface contract a consumer depends on. Qualitative laws outlive the measurement that taught them — keep the law, drop the numbers.
|
||||
- Never let a tool's "this is duplicated / this is the only record" census decide what to keep. That is a question of USEFULNESS, which only the operator answers.
|
||||
|
||||
1. Read the relevant `index.md` before you touch a topic.
|
||||
2. Follow `omp-docs/references/okf-spec.md` for every doc you create or edit.
|
||||
3. Read the target file completely before you edit it.
|
||||
4. Always update the documentation when you've completed a task.
|
||||
5. Compact the work log when necessary.
|
||||
4. Always update the documentation when you've completed a task — and when updating means deleting a page or a paragraph, delete it.
|
||||
5. Compact the work log when necessary; a work log holds laws and incidents, never bookkeeping.
|
||||
|
||||
#### THE WORK LOG IS ONE FILE.
|
||||
The work log is a single page in the knowledge base: `omp-docs/log.md` in this repo, or whichever change-log page the repo's docs index names. Write the entry when the change lands — in the same commit as the change — and read that page to recover where things stand before starting. It follows its own entry law: a dated entry is what changed plus the durable lesson, a few lines.
|
||||
|
||||
- NEVER create a second log: not per directory, not per project, feature, branch or run, not `NOTES.md` / `PROGRESS.md` / `STATUS.md` / `worklog/`, not a dated diary. In `dank/command-center` the KB linter reports any `log.md` that is not the bundle root's; the rule and its reasoning are owned by `omp-docs/references/okf-spec.md`.
|
||||
- A log holds laws and incidents, never bookkeeping: no per-file inventories, no run-by-run numbers, no transcripts. A number in a log still has to pass the purpose test above.
|
||||
- If a knowledge base has no log page, add one at its root and list it in the index. That is the fix — never a log somewhere else.
|
||||
|
||||
#### USE PROGRESSIVE DISCLOSURE.
|
||||
Always ensure that all produced documentation is laid out in files following WIKI conventions: start with a top level README.md index file that provides a brief, concise introduction to the item being documented, and then SPLIT OFF individual aspects, features, or arguments into their own interlinked pages so that future agents can read the documentation in a piecewise fashion, only taking in the details they need. Where relevant, crosslink any created documentation with other relevant documentation. This structure may also be nested, if necessary.
|
||||
@@ -90,12 +104,24 @@ Never assume the user reports correct information. He is human, and humans make
|
||||
- Before starting new work, `git pull` from the remote, and start a new branch in a separate worktree using `master` or `main` as the base branch.
|
||||
- Commit atomically: after every change that forms a complete functional unit, commit. Do so early and often.
|
||||
- When the work is complete, ask the user if they would like to open a pull request.
|
||||
- **Report a pull request as link AND title, every time.** A bare `#54` is not a reference: numbers are per-repository, so a number alone does not say which work, in which repo, is being reported. Give the PR's web URL on that forge — on the Dankclaw Gitea that is `https://gitea.franksdank.cloud/<owner>/<repo>/pulls/N` — together with the PR's title. Applies everywhere a PR is mentioned: chat replies, status summaries, issue and commit bodies, alerts. Another forge (GitHub, GitLab, …): the equivalent URL its web UI serves, same pairing.
|
||||
|
||||
### 5. Test Driven Development
|
||||
|
||||
- Always practice Test Driven Development practices; do UNIT TESTS exist for the functionality about to be edited? Is there a LINTER? Is there a FORMATTER? Verify that they exist, and that they are configured as a pre-commit hook. If they do not exist, they MUST be written PRIOR to the code being written. Invoke `/skill:tdd` for preferred testing style. If tests do exist, ensure that they still match the planned changes. If you believe a PRE-EXISTING test must be changed prior to commencing work, ASK THE USER ABOUT IT.
|
||||
- Always ensure tests run on code changes after every commit, and all project tests run before every push using git hooks, EXCEPT in the event that the individual test in question takes an excessive amount of time. Always ask the user before disabling an individual due to running time or performance concerns.
|
||||
|
||||
|
||||
### 6. Keep the Todo List LIVE, Not Retrospective
|
||||
|
||||
The todo list is the user's cockpit view of your work. A list that only flips at the end is a history dump, not tracking - the user sees nothing while you work.
|
||||
|
||||
- Multi-step work: `todo init` BEFORE the first substantive step - the list must exist while the work is unseen, not after.
|
||||
- Batch the todo transition with the step it describes: `done` + `start` ride along in the same message as the first tool calls of the next step. NEVER defer flips until the whole bill of tasks is finished.
|
||||
- A task is `done` the moment its work is verified - not when the surrounding phase ends.
|
||||
- Blocked on external input (user answer, subagent, service)? `block` it with a reason so the stall is visible; `unblock` the instant it is actionable.
|
||||
- NEVER make a todo call the only call in a message - that is ceremony, not progress; pair every flip with real work.
|
||||
|
||||
## FORBIDDEN ACTS AND MISTAKES TO NOT REPEAT (DO NOT DO THESE):
|
||||
- Editing node_modules or similar import module directories. This is third party code that will be overwritten next time `npm update` or a similar command is run.
|
||||
- `git push` - unless directed, except if it's our local gitea instance on Dankclaw.
|
||||
|
||||
+19
-6
@@ -1,6 +1,10 @@
|
||||
providers:
|
||||
openrouterVariant: online
|
||||
fetch: auto
|
||||
maxInFlightRequests:
|
||||
llamacpp (dankserver): 3
|
||||
llamacpp (dankclaw): 2
|
||||
llamacpp (dankclaw-3090): 3
|
||||
modelRoles:
|
||||
slow: llamacpp (dankclaw)/frank-dragoon:medium
|
||||
vision: llamacpp (dankclaw-3090)/frank-zealot:xhigh
|
||||
@@ -13,7 +17,7 @@ modelRoles:
|
||||
default: llamacpp (dankserver)/frank-archon:medium
|
||||
smol: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
web: web/searxng
|
||||
speech: local/kokoro
|
||||
speech: frank-tts (dankclaw)/frank-tts
|
||||
judge: openrouter/~typesafe/jev-latest:auto
|
||||
dictation: local/parakeet-tdt-0.6b-v3
|
||||
memory: llamacpp (dankclaw-3090)/frank-zealot:medium
|
||||
@@ -64,7 +68,7 @@ retry:
|
||||
- llamacpp (dankclaw)/frank-dragoon
|
||||
maxRetries: 10
|
||||
searxng:
|
||||
endpoint: search.franksdank.cloud
|
||||
endpoint: http://192.168.0.143:8081
|
||||
exa:
|
||||
enabled: false
|
||||
memory:
|
||||
@@ -145,17 +149,25 @@ display:
|
||||
subagentLivePreview: true
|
||||
task:
|
||||
showResolvedModelBadge: true
|
||||
maxConcurrency: 3
|
||||
maxConcurrency: 2
|
||||
agentModelOverrides:
|
||||
designer: llamacpp (dankclaw-3090)/frank-zealot
|
||||
librarian: llamacpp (dankclaw-3090)/frank-zealot
|
||||
reviewer: llamacpp (dankserver)/frank-archon
|
||||
scout: llamacpp (dankserver)/frank-archon
|
||||
scout: llamacpp (dankclaw-3090)/frank-zealot
|
||||
security-reviewer: llamacpp (dankclaw)/frank-dragoon
|
||||
sonic: llamacpp (dankclaw-3090)/frank-zealot
|
||||
task: llamacpp (dankserver)/frank-archon
|
||||
seance: llamacpp (dankclaw-3090)/frank-zealot
|
||||
eager: default
|
||||
enableLsp: true
|
||||
maxRuntimeMs: 1800000
|
||||
softRequestBudget: 120
|
||||
maxRecursionDepth: 1
|
||||
disabledAgents: []
|
||||
isolation:
|
||||
enabled: true
|
||||
completionProbe: false
|
||||
snapcompact:
|
||||
systemPrompt: none
|
||||
toolResults: false
|
||||
@@ -203,8 +215,8 @@ vault:
|
||||
tts:
|
||||
localVoice: bm_george
|
||||
speech:
|
||||
enabled: true
|
||||
mode: yield
|
||||
enabled: false
|
||||
mode: all
|
||||
enhanced: true
|
||||
voice: am_fenrir
|
||||
secrets:
|
||||
@@ -232,6 +244,7 @@ followUpMode: one-at-a-time
|
||||
tools:
|
||||
approvalMode: yolo
|
||||
format: auto
|
||||
maxTimeout: 600
|
||||
marketplace:
|
||||
autoUpdate: notify
|
||||
bash:
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: amd-apu-vulkan-tenant-onboarding
|
||||
description: "Add a ggml/Vulkan inference tenant to an AMD APU (gfx1151 / Strix Halo iGPU) that already serves another engine: APU memory census, RADV heap gate, render-node resolution, build-side glslc/ICD deps, co-tenancy measurement."
|
||||
---
|
||||
|
||||
Use when a ggml-based engine (llama.cpp, qwentts.cpp, anything built on ggml) must move from CPU to the **iGPU** of an AMD APU (gfx1151 / Strix Halo / Ryzen AI Max) that **already serves another engine on the same card**. Answers two questions only a booted host can: what gate the second tenant must clear, and which files the build needs to clear it.
|
||||
|
||||
## 1. The usual VRAM reasoning is wrong on an APU
|
||||
|
||||
`/sys/class/drm/cardN/device/mem_info_vram_total` reports a small **carve-out** (2 GiB class), and `rocm-smi`'s "dedicated" figure is not the working set either. The real working set lives in **GTT / system pages**: `/proc/meminfo` → `GTT` and `System pages`. A second GPU tenant therefore consumes host RAM from the same pool the first tenant's weights and the OS use — no supplementary groups, no separate aperture to reserve.
|
||||
|
||||
- Price the tenant from a **booted** footprint, never from `weights + KV + buffers`: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check `free` *after* the load, not before.
|
||||
- Never subtract a "dedicated" figure from it to find headroom.
|
||||
- Unheld fabric clock (`pp_dpm_mech` = `auto`) is a hazard only when the NPU joins too; a pure iGPU pair is unaffected — check it before adding a third tenant.
|
||||
- Swap on a host with GPU tenants protects co-tenants; paging a KV pool is worse than the OOM it prevents.
|
||||
|
||||
## 2. The gate that decides feasibility: the RADV heap a container sees
|
||||
|
||||
`rocm-smi` and `free` cannot see this number; only a container off the **engine's own base image** can:
|
||||
|
||||
```bash
|
||||
# one-off container, no model, no load, no compute — safe beside production
|
||||
podman run --rm --device /dev/dri --group-add keep-groups <engine-base> bash -c \
|
||||
'apt-get update -qq >/dev/null 2>&1; apt-get install -y -qq --no-install-recommends mesa-vulkan-drivers vulkan-tools glslang-tools >/dev/null 2>&1; vulkaninfo --text'
|
||||
```
|
||||
|
||||
Read `VkPhysicalDeviceMemoryProperties` → the `DEVICE_LOCAL` heap size. On a hybrid Strix Halo box Mesa splits the aperture (~1/3 : 2/3), so `DEVICE_LOCAL` is the budget the ggml tool will actually allocate against — measured 77.33 GiB *with the first tenant loaded*, against a 2 GiB DRM carve-out. Two traps, both measured:
|
||||
|
||||
- `vulkaninfo --batch` does not exist in Ubuntu 24.04's `vulkan-tools` (prints help, exits 1). Every `vulkaninfo` parse needs `--text`.
|
||||
- The container may have **no NVIDIA ICD**, so Vulkan index `0` *is* the iGPU — but do not trust ordering by index: `--text` and read `deviceName` + `driverID` (`RADV_GFX1151` vs `NVIDIA`). A hand-picked `/dev/dri/renderDN` is worse than useless on a hybrid host: **`renderD128` there is the NVIDIA card and `renderD129` the amdgpu iGPU**; resolve the node with `readlink -f /sys/class/drm/renderD*/device/driver` and pass `/dev/dri` wholesale, letting the driver pick by name.
|
||||
- A rootless container that drops its supplementary groups enumerates **nothing**. `--group-add keep-groups` (quadlet: `GroupAdd=keep-groups`) plus the host udev rule that makes the render nodes group-writable is the whole requirement; `--device /dev/dri` alone is not enough.
|
||||
|
||||
## 3. Build-side deps: the configure fails before the device ever matters
|
||||
|
||||
ggml's `src/ggml-vulkan/CMakeLists.txt` requires, in order: `find_package(Vulkan COMPONENTS glslc REQUIRED)` then `find_package(SPIRV-Headers CONFIG REQUIRED)`, and it compiles its shaders with **glslc**, never `glslangValidator`. On Ubuntu 24.04 (measured against the digest-pinned `ubuntu:24.04` base, 2026-10-19):
|
||||
|
||||
- `glslang-tools` ships `glslangValidator` and **NOT `glslc`** → `command -v glslc` empty, `find_package(Vulkan … glslc REQUIRED)` fails, and the Vulkan backend silently never exists.
|
||||
- The package is the standalone **`glslc`** (universe, `2023.8-1build1`). Working build-stage set: `libvulkan-dev glslc spirv-headers` — `libvulkan-dev` provides `vulkan/vulkan.h` + the `VulkanHeaders`/`VulkanLoader` CMake configs `find_package(Vulkan)` resolves to, `spirv-headers` ships `/usr/share/cmake/SPIRV-Headers/SPIRV-HeadersConfig.cmake`. A configure probe in the pinned base prints `Found Vulkan: … found components: glslc` then `Including Vulkan backend`.
|
||||
- **The archive's glslc is old.** 2023.8 compiles `GL_KHR_cooperative_matrix` (the path ggml's vectored quantized matmul needs) but reports `cooperative_matrix2`, `integer_dot_product`, `bfloat16`, `float_e2m1/e4m3` as unsupported, so those shader families are compiled out. If a Vulkan prefill reads as inexplicably slow, suspect the **shader compiler before the device**; the fix is a pinned glslang build passed as `-DVulkan_GLSLC_EXECUTABLE=<path>`, not a different runtime.
|
||||
- Runtime stage needs the loader **and an ICD**: `libvulkan1` + `mesa-vulkan-drivers`. Loader-without-ICD enumerates zero devices, and ggml-based servers die in device lookup — often *after* reading the weights, so the journal reads like a model fault.
|
||||
- Ship **no NVIDIA ICD** on purpose: one Vulkan device in the enumeration is what makes the *name* `Vulkan0` safe to write into config. Prefer the name over an index in every flag.
|
||||
|
||||
## 4. Prove enumeration without allocating anything
|
||||
|
||||
Do not boot a scratch second engine beside production to ask which devices exist — a second model load is a real GPU tenant, and fleet law stops that. Instead run the binary with a device name that cannot exist and read its error; it names what it enumerated, with no allocation and no model resident:
|
||||
|
||||
```bash
|
||||
podman run --rm --device /dev/dri --group-add keep-groups --entrypoint /app/tts-server <image> \
|
||||
--model /dev/null --codec /dev/null --device NoSuchDevice 2>&1 | tail -20
|
||||
```
|
||||
|
||||
## 5. The rollback must be a flag, not a rebuild
|
||||
|
||||
Put the device in the env file as a value the entrypoint passes only when set, and make **set-but-empty** mean "pass no flag at all" — the binary's own default (CPU, always built) is the rollback. `: "${VAR:=default}"` is the WRONG form: `:=` fires on an empty value too, so the documented rollback line `VAR=` is silently rewritten back to the GPU and the only way out is a rebuild. Use:
|
||||
|
||||
```bash
|
||||
if [[ ! -v QWENTTS_DEVICE ]]; then QWENTTS_DEVICE=Vulkan0; fi
|
||||
```
|
||||
|
||||
(`podman`'s `--env-file` passes `VAR=` through as set-and-empty, and `source` does the same, so both readers agree.) Pin it in a test: empty device ⇒ the flag is absent from argv.
|
||||
|
||||
## 6. Co-tenancy measurement (only after the tenant is live)
|
||||
|
||||
The gate is the **first tenant's** performance with a real request from the second in flight — not the second tenant's own speed. And price the tenant from a booted footprint, never from weights+KV+buffers: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check `free` after the load, not before.
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: audit-never-run-container-deploy
|
||||
description: "Audit a locally built podman/quadlet deploy path that has never actually run: prove base-image assumptions, env-file reader divergence, and image-hash coverage before the first deploy."
|
||||
---
|
||||
|
||||
# Audit a never-run container deploy path
|
||||
|
||||
A deployable that has never been built or started has never been *executed*: every
|
||||
fault in it is invisible to tests that stub the runtime. Audit it as static code
|
||||
against the tools it calls, then prove each suspicion cheaply against the real
|
||||
tool. The faults worth hunting are the ones that **report success while doing
|
||||
nothing** — those are the ones that survive a green `/health` and a green e2e run.
|
||||
|
||||
## The three fault classes (all measured 2026-10-18, frank-tts-engine)
|
||||
|
||||
1. **A command the base image does not ship.** e.g. `adduser` in a `RUN` line
|
||||
against a trimmed `ubuntu:24.04` — absent, and not pulled in by
|
||||
`ca-certificates curl libgomp1`. Dies at build time (loud), but only the first
|
||||
time anyone builds it.
|
||||
Prove it in the base, never from memory:
|
||||
`podman run --rm <base> sh -c "command -v adduser || echo absent; useradd --system --uid 10000 --home-dir /app --shell /usr/sbin/nologin tts && id tts"`
|
||||
Note the accepted cosmetic warning (`uid 10000 is greater than SYS_UID_MAX 999`)
|
||||
in a comment, or a future agent "fixes" it by dropping `--system` and the uid
|
||||
stops matching the sibling container.
|
||||
2. **Two readers of one env file disagree.** systemd units that pass
|
||||
`EnvironmentFile=` hand it to podman `--env-file`, which does **no shell
|
||||
parsing** — it takes the literal remainder of the line — while bash `source`
|
||||
strips one quoting layer. So `WARM_TEXT="Warm up, one two three."` reaches the
|
||||
container *with its quotes*, and a script that interpolates it into a JSON body
|
||||
produces invalid JSON: the container boots, `active`, every voice registered,
|
||||
every voice cold. Fix in the consumer: strip exactly one matched pair **only**
|
||||
when both ends are the same quote AND the inner content carries another quote or
|
||||
backslash (never mangle a value that legitimately contains quotes), then escape.
|
||||
3. **A build-skip hash that under-covers the image.** `build_compose_image`-style
|
||||
helpers skip `podman build` when a label hash matches; hashing the `Containerfile`
|
||||
alone means editing a `COPY`'d app/script rebuilds **nothing** — deploy reports
|
||||
success, `/health` answers, container serves old code. Each case arm must name
|
||||
the paths its `Containerfile` copies (`extra=(…)`) and hash Containerfile+extras.
|
||||
Arms that have no extras keep their old hash, so they do not needlessly rebuild.
|
||||
|
||||
## Test it as behaviour, never as a grep of the handler
|
||||
|
||||
- Drive the **real** function: `awk '/^build_compose_image\(\) \{/,/^\}/' deploy-remote.sh > bci.sh`,
|
||||
`source` it, put a stub `podman` first on `PATH`, assert the observable question —
|
||||
*did the deploy ask podman to build, or did it skip?*
|
||||
- Stub traps that cost real time: podman is invoked `podman image inspect <image> --format …`
|
||||
so the tag is **`$3`**, not `$2`; tags contain `/` so a state file named after one
|
||||
needs `${tag//\//_}` or the write silently lands in a nonexistent directory and
|
||||
every image looks permanently unbuilt; `--label cfhash=…` arrives **before** `-t <tag>`,
|
||||
so collect the label and write it after the arg loop.
|
||||
- For an env-file bug, write one test that reads the file **both** ways — `source`
|
||||
and the `--env-file` way (parse `KEY=literal remainder`) — and assert the program
|
||||
receives the same value in both shapes, including a value that carries its own
|
||||
quotes and a backslash. A suite that only sources the file is how the bug shipped.
|
||||
- Prove red: `git show HEAD:scripts/deploy-remote.sh > scripts/deploy-remote.sh`,
|
||||
run the new test (must fail at the exact intended step), restore. Keep each fix in
|
||||
its own commit and run the seam at each one so the branch bisects.
|
||||
|
||||
## Base-image and host facts to check while auditing
|
||||
|
||||
- `useradd` lives in `login(1)` (base carries it); `adduser` does not.
|
||||
- Rootless podman **cannot raise rlimits** — `--ulimit nofile=…` above the host hard
|
||||
limit dies at container create, not at runtime.
|
||||
- A referenced `Network=` `.network` file the generator cannot find means the
|
||||
container unit is **not produced at all**.
|
||||
- Two containers talk by DNS name only if they attach to a **common** network;
|
||||
check both unit files rather than trusting a comment that says they share one.
|
||||
- Container DNS name + the network that owns it, never `host.containers.internal`
|
||||
or a published loopback port, for service-to-service traffic.
|
||||
|
||||
## Do not
|
||||
|
||||
- Do not deploy the audited thing because the audit is finished: first build of a
|
||||
heavy image (recursive source clone ~10 min) plus a new unit on a host serving
|
||||
people is the operator's call.
|
||||
- Do not add a Prometheus target for a container with no `/metrics` — say the probe
|
||||
is a decision (blackbox HTTP), do not fake a scrape.
|
||||
@@ -1,61 +1,41 @@
|
||||
---
|
||||
name: audit-subagent-deliverables
|
||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), adversarial-review-report triage, resource-safety pass (fd/memory), and cutover with running-process restarts."
|
||||
description: "Integrate subagent-written code safely: line-by-line audit, mechanical gates (bash test/lint), merge shared-function collisions, fix scope creep, verify env var propagation."
|
||||
---
|
||||
|
||||
# Audit subagent deliverables
|
||||
Integrating code written by subagents (fan-out implementation). The code is usually correct; the failures are structural — collisions between agents, scope creep, and assumptions that never reach the runtime.
|
||||
|
||||
A subagent that wrote code is a source of plausible output, not a verified change. Integrate it the way you'd integrate an untrusted PR — with extra suspicion, because the author is an agent that shares your blind spots and its prose is more confident than its evidence.
|
||||
## Workflow
|
||||
|
||||
Order of operations: read everything → gates → tests (as evidence, not proof) → resource-safety pass → fix in-place → cutover.
|
||||
1. **Read every deliverable line by line** before integrating — not the summary.
|
||||
2. **Run the mechanical gates immediately**, in parallel: `bash -n` on every script; `bats tests/ 2>&1 | grep "not ok"` for the failure list, never `tail` (tail shows a passing tail and hides the red in the middle).
|
||||
3. **Merge shared-function collisions.** When two agents add different versions of the same function, pick the superset and re-point the other's callers at it. (Real case: one agent's `fleet_base()` carried `-b`, the other's `resolve_fleet_base()` carried `--base-url`; resolve to the one flag set the tests use.)
|
||||
4. **Delete scope creep.** Code added for a goal nobody asked for — a new env var, a new CLI flag, a new output file — goes, with its docs, unless the operator asked.
|
||||
5. **Verify env var propagation.** A var the orchestrator computes must be exported before the `bash -c` / subshell that reads it, or the child silently takes a default. Check the export site, not the doc.
|
||||
|
||||
## 1. Read every produced file, line by line
|
||||
Full file, not a diff skim. You are the reviewer who must catch the line the author never thought about.
|
||||
## Audit checks
|
||||
|
||||
## 2. Mechanical gates before logic review
|
||||
```bash
|
||||
bash -n <script> # syntax (scripts)
|
||||
bunx tsc --noEmit # types (TS)
|
||||
bun test # suite
|
||||
```
|
||||
Run the suite for information, not as the stopping point — it only proves the author's own scenarios.
|
||||
| Check | How |
|
||||
|---|---|
|
||||
| Flag names real | `bash script.sh --help` |
|
||||
| Paths exist | `test -e` each literal in the diff |
|
||||
| Env reaches the runtime | grep the export/inline-assignment site, then the read |
|
||||
| Tests assert behaviour | not `[[ -n "$X" ]]`, not `test -f <new file>` |
|
||||
| Tests not weakened | `git diff HEAD~ -- tests/` — a deleted assertion is a red flag |
|
||||
|
||||
## 3. Hunt the three classic agent defects
|
||||
- **Quoting bugs:** `set -e` + `"${@/$p/$r}"` = substitution regex containing user input (bash can't escape `/` in a `${var/re/pat}` pattern: `/` reads as the pattern/replacement separator). Fix with a separator absent from inputs (e.g. `%`) and `printf '%q'` for shell-interpolated values.
|
||||
- **`cd` assumptions:** parallel `runShell` calls share one CWD (pool mode). Any command sequence that assumes CWD (e.g. `git -C .` after a sibling's `cd`) must use explicit absolute paths.
|
||||
- **Unbounded resources:** every fd opened must be closed on ALL paths (success, throw, validation fail) — `using` for fs.sinks (closes on throw; sinks are `AsyncDisposable`, not `fs.FileHandle`, no `.close()`); bounded `Map` for caches (`MAX` + FIFO eviction); explicit caps (lines read, file sizes).
|
||||
## Red flags
|
||||
|
||||
## 4. Check the test file is real
|
||||
Tests that assert internal state (`expect(wcwd).toBe(...)`) over output contracts; tests of mock echo; tautologies. Delete them per house rules, keep behavior contracts.
|
||||
- Test asserting a new file exists rather than behaviour.
|
||||
- Doc claiming a flag the script never defines.
|
||||
- Function added with no caller.
|
||||
- Doc example that contradicts the code it documents.
|
||||
- Same helper defined twice with different flag spelling.
|
||||
- New env var read but never exported.
|
||||
|
||||
## 5. Cutover audit (repo-wide)
|
||||
`grep -r` the symbol namespace the subagent touched — `wcwd`-style globals are the classic blast radius. Grep every `cd`/`process.chdir` site and verify each paired command sequence uses explicit paths. Stale comments describing the old design must go — comments that lie are time bombs for the next reader.
|
||||
## Re-verify after the operator's own edits
|
||||
|
||||
## 6. Cutover restarts running processes
|
||||
A code fix on disk is NOT live in any process that loaded the file at spawn (bun/node load-at-spawn). After shipping fixes: restart every long-running supervisor/daemon/UI-server that carries the affected code path; prefer takeover protocols (stale-pid detection) that make restarts cheap. Symptom of forgetting: "fixed" code producing old-code symptoms in production (observed: 812 poisoned ledger rows).
|
||||
Run the gates again — their edit may undo an integration (a re-typed helper, a reverted flag).
|
||||
|
||||
## 7. Triaging an adversarial REVIEW report (not code)
|
||||
When a subagent produced findings, treat each finding as a hypothesis with a base rate ~40% (measured over 3 rounds, 2026-10):
|
||||
1. **Walk every finding end-to-end against the CURRENT tree** before accepting — cite file:line yourself. Reviewers frequently read a mid-edit snapshot; many findings are stale by construction.
|
||||
2. **Classification that kills most findings:**
|
||||
- *Stale read* — code changed during review; verify against live lines.
|
||||
- *Inverted logic* — "failed rows are skipped" when the filter makes failed rows RETRY. Re-read the filter.
|
||||
- *Unreachable* — trigger requires breaking a stated design law (single-worker, append-only). Not a defect.
|
||||
- *By-design* — the reviewer labels a spec law a bug (serial execution, non-stream client). Check the spec, not the reviewer's confidence.
|
||||
- *Constant-trust* — "XSS via manifest field" where the field is git-pinned config; "path traversal via REMOTE_TMP" where it's a hardcoded const. Ask: what untrusted input reaches this sink?
|
||||
- *Self-refuting* — the reviewer's own probe output disproves its claim (raw==dedup probe in a "dedup" finding). Quote it back.
|
||||
3. **Acceptance bar:** concrete trigger with real inputs + full path walked. Real accepts cluster in: resource caps (pids/timeout/awaited-cleanup), honest labels (default signatures), escaping discipline, self-healing probes.
|
||||
4. **One-line verdicts + evidence per finding**, shipped in the commit message; calibration section ("genuinely solid") accepted as free verification of past fixes.
|
||||
## Report
|
||||
|
||||
## Verification (the non-negotiable)
|
||||
Build/run the real artifact with its real inputs. Observe behavior at the seams:
|
||||
- concurrent calls → no CWD races;
|
||||
- resource ceilings → caps enforced;
|
||||
- cleanup → no fd/pid/dir leaks;
|
||||
- security → injection probes inert (quotes, slashes, metachars);
|
||||
- normal case → output identical to pre-audit.
|
||||
|
||||
## Evidence (2026-10-07 + 2026-10-10/11 sessions)
|
||||
- Shell-engine subagent: 426 LOC, `bun test` 39/39 green. Line-audit found: 5 leaked fds per call (`fd` sink never closed), CWD-race in `git -C .`, `set -e` + `${@/\/g\//}` slash-escaping bug, unbounded 10k-entry cache, 45s timeout on git-status.
|
||||
- Review-triage census over 3 blind adversarial rounds (same repo, ~41 findings): accepted 9, rejected ~32 — rejects dominated by stale reads (protocol rewritten mid-review), inverted filter logic, unreachable-under-law triggers, base64-alphabet "quote injection" (charset has no quotes), SIGPIPE non-leaks, "no memory cap" quoting the `--memory 4g` line above the claim. Real accepts: fork-bomb pids cap, stalled-ssh upload cap, stale-flag self-heal probe, honest census label, freshness gate on pid reuse.
|
||||
- Poison-law incident: sandbox fix shipped while supervisor ran old build → 812 rows graded by the dead protocol; takeover made recovery cheap. The restart step (#6) exists because of this.
|
||||
Per deliverable: correct / changed / removed, with file:line, plus the gate output that proves it (`bats` failure list before and after).
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
name: authentik-rp-sso-login-diagnosis
|
||||
description: "Diagnose/repair a dead OAuth2 relying-party login against authentik (Grafana-style): authorize 500, missing email claim, Grafana /emails fallback. Live probe harness + ak-shell gotchas."
|
||||
---
|
||||
|
||||
# Authentik OAuth2 relying-party (RP) SSO login diagnosis
|
||||
|
||||
Use when a vhost authenticates through authentik via the OIDC authorization-code
|
||||
flow (Grafana, Seerr OIDC, any "Sign in with <authentik>" button) and the login
|
||||
fails — even though the guard-chain e2e (Anubis + outpost) passes green.
|
||||
|
||||
## Key insight: the edge test lies
|
||||
|
||||
The outpost-guard spec (e.g. `13-sso`) only proves an authenticated session
|
||||
reaches the app's *login page*. It never follows the RP OAuth button. So the RP
|
||||
login can be 100% dead and the suite still passes. **Always drive the real RP
|
||||
round-trip** to get ground truth (harness below).
|
||||
|
||||
## Symptom → cause map (all verified live on 2026-10-20, grafana)
|
||||
|
||||
| Signature | Root cause | Fix |
|
||||
|---|---|---|
|
||||
| Browser hits `…/application/o/authorize/?client_id=…` → "Server Error — Powered by authentik" (500), traceback `AttributeError: 'NoneType' object has no attribute 'slug'` in `flows/planner.py` | provider `authorization_flow` is NULL (raw `ak shell` `create()` leaves it unset) | set `authorization_flow = default-provider-authorization-implicit-consent` |
|
||||
| Grafana log `logger=oauth.generic_oauth level=error msg="Error getting email address" url=<api_url>/emails … 404/403` | the granted scope mapping returns a **bare string**, not a dict → no `email` claim emitted | fix the ScopeMapping expression to `return {"email": user.email, "email_verified": True}` |
|
||||
| authorize 500 on scope-overlap, log `Application requested scopes not configured` for a scope like `user:email` | client requested a scope authentik doesn't define (GitLab-ism) | client must request authentik's real scopes: `openid email profile` |
|
||||
| Grafana `?api=1` on API URL → GET `<…>/userinfo/?api=1/emails` → 403 | Grafana v13 appends `/emails` to a `?api=1` URL | use the standard `…/userinfo/` with **no `?api=1`** |
|
||||
|
||||
Crucial causal note: **Grafana v13 GETs `<api_url>/emails` for the address ONLY
|
||||
when the token carries no `email` claim.** So the `/emails` 404 is a *symptom of
|
||||
the missing claim*, not a Grafana misconfig. Do NOT chase it with `USE_ID_TOKEN`
|
||||
or attribute paths; fix the claim. `USE_ID_TOKEN=true` does NOT stop the
|
||||
`/emails` fetch. Once the claim exists, standard userinfo carries `email` and no
|
||||
`USE_ID_TOKEN` is needed.
|
||||
|
||||
## Live RP-login probe harness (ground truth)
|
||||
|
||||
A `qa-e2e` authentik session storage state exists at
|
||||
`tests/e2e/.auth/qa-e2e.json` after an e2e run; the e2e node_modules live in the
|
||||
main checkout (`tests/e2e/`). Throwaway probe:
|
||||
|
||||
```js
|
||||
// node _p.cjs (run from a checkout with node_modules + .auth)
|
||||
const { chromium } = require('playwright');
|
||||
(async () => {
|
||||
const b = await chromium.launch({ headless: true });
|
||||
const ctx = await b.newContext({ storageState: '.auth/qa-e2e.json',
|
||||
userAgent: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36" });
|
||||
const p = await ctx.newPage();
|
||||
await p.goto('https://<vhost>/', { waitUntil: 'domcontentloaded' });
|
||||
for (let i=0;i<30;i++){ const t=await p.locator('body').innerText().catch(()=>''); if(!t.includes('not a bot')) break; await p.waitForTimeout(3000);}
|
||||
await p.goto('https://<vhost>/login/generic_oauth', { waitUntil: 'domcontentloaded' });
|
||||
await p.waitForTimeout(9000);
|
||||
const r = await ctx.request.get('https://<vhost>/api/user');
|
||||
console.log(p.url(), r.status(), (await r.text()).slice(0,200));
|
||||
await b.close();
|
||||
})();
|
||||
```
|
||||
|
||||
Success = final URL on the app host (not `/login`) AND `/api/user` 200 with an
|
||||
email. Before the fix it is `/login` + 401. Move the throwaway to
|
||||
`~/.AGENTTRASH/…` after (never `rm`).
|
||||
|
||||
## ak-shell census: operational traps (authentik 2026.8)
|
||||
|
||||
- **Run it via `ssh dankclaw bash -s <<'EOF' … EOF`, NEVER direct
|
||||
`ssh host '… "{{…}}"'`.** Direct ssh eats inner quotes locally; `podman inspect
|
||||
--format '{{…}}'` and multi-line `ak shell -c "…"` only survive inside a quoted
|
||||
heredoc. Authentik runs **rootless** (`podman exec authentik-server …`, no sudo).
|
||||
- `OAuth2Provider` has **no `properties` attribute** in 2026.8; scopes are derived
|
||||
from `p.property_mappings`. Accessing `.properties` raises and silently aborts
|
||||
the whole `-c` payload.
|
||||
- `ScopeMapping` is **not importable** from `authentik.core.models`. To read
|
||||
scopes: iterate `p.property_mappings.all()`, then `m.scopemapping.scope_name`
|
||||
(reverse one-to-one; the base `PropertyMapping` row has no `scope_name`).
|
||||
- Introspect before guessing: `[f.name for f in Model._meta.get_fields()]`. Don't
|
||||
trial-and-error attribute names — that wastes turns.
|
||||
- Claims-into-id_token field is `include_claims_in_id_token`.
|
||||
- Provider census that works:
|
||||
```
|
||||
from authentik.providers.oauth2.models import OAuth2Provider
|
||||
for p in OAuth2Provider.objects.all():
|
||||
af=p.authorization_flow
|
||||
print(p.name, af.slug if af else None)
|
||||
# email mapping shape (the real bug):
|
||||
p=OAuth2Provider.objects.filter(name='grafana').first()
|
||||
for m in p.property_mappings.all():
|
||||
print(m.name, getattr(getattr(m,'scopemapping',None),'scope_name',None), repr(getattr(m,'expression','')))
|
||||
```
|
||||
|
||||
## Repair = idempotent script (DB state, not deployable)
|
||||
|
||||
Authentik config lives in its DB, not in any repo file; a blueprint re-import or
|
||||
an `ak shell` re-creation reverts provider fields silently. Ship an idempotent
|
||||
`scripts/authentik-<svc>-sso-fix.sh` that `ssh HOST 'podman exec -i
|
||||
authentik-server ak shell' <<'PYEOF' … PYEOF` and re-applies each fact guarded by
|
||||
"already set?" checks. Match the style of
|
||||
`scripts/authentik-set-login-session-duration.sh`; add a header note "DB state —
|
||||
re-run after a rebuild". Reference it from the service page. Grafana's half
|
||||
(env) lives in `grafana.container` and deploys normally.
|
||||
|
||||
## Repo test-seam gotchas this triggers
|
||||
|
||||
- **literal-secret seam** (`observability_quadlet.bats`, "no literal secret")
|
||||
globs keys `*_SECRET|*_PASSWORD|*_TOKEN|*_API_KEY` and demands a `${VAR}`
|
||||
value. A boolean env ending in `_TOKEN` (`…_USE_ID_TOKEN=true`) is a **false
|
||||
positive**. Prefer not to set such a key (drop `USE_ID_TOKEN`); otherwise the
|
||||
seam must be refined (ask the owner before editing a pre-existing test).
|
||||
- Quadlet `Environment=` **word-splits on spaces** (systemd semantics) — a spaced
|
||||
list must be quoted: `Environment="KEY=a b c"`, else it truncates to `a`. The
|
||||
repo already uses this (`open-webui.container` OAUTH_SCOPES).
|
||||
- **OKF conformance gate** rejects a page frontmatter `timestamp` in the future.
|
||||
The fleet host clock reads UTC (often a day ahead of local "today") — set the
|
||||
timestamp to the repo's today, not the host's UTC date.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: authentik-session-ttl-query
|
||||
description: "Query or change authentik login-session TTL on Dankclaw via ak shell — the string-typed UserLoginStage.session_duration trap, the apply logic, and why SESSION_SAVE_EVERY_REQUEST is unavailable."
|
||||
---
|
||||
|
||||
Change or audit how long an authentik login session lives (Dankclaw `authentik-server`, build 2026.x). The real knob is a DB stage row, not env — and the field format will bite you.
|
||||
|
||||
## Where the TTL actually lives
|
||||
- Run program through raw stdin: `ssh dankclaw 'podman exec -i authentik-server ak shell' <<'EOF' ... EOF 2>&1 | sed -n '/objects imported/,$p'`.
|
||||
- Get models by label, NOT import paths (`from authentik.core.models.setting import ...` fails; `authentik.core.models` is not a package; GlobalSettings is NOT a model in 2026.x):
|
||||
`from django.apps import apps; ULS = apps.get_model("authentik_stages_user_login", "UserLoginStage")`
|
||||
- The password sign-in path is flow `default-authentication-flow` → stage `default-authentication-login` (FlowStageBinding FK is `target`, not `flow`). Other stages `default-source-authentication-login` / `default-source-enrollment-login` govern external-IdP/enrollment logins only.
|
||||
|
||||
## THE FORMAT TRAP (will break login if wrong)
|
||||
`UserLoginStage.session_duration` is **string-typed** in authentik's own idiom: `"days=7"` / `"seconds=2"`.
|
||||
- Assigning a `datetime.timedelta` stores Python's `str(timedelta)` = `'7 days, 0:00:00'`, which authentik's parser `authentik.lib.utils.time.timedelta_from_string` **REJECTS** (`ValueError: not enough values to unpack`). Left like that, login crashes computing the expiry.
|
||||
- Correct write: `s.session_duration = "days=7"; s.save(update_fields=["session_duration"])`
|
||||
- Verify it parses: `from authentik.lib.utils.time import timedelta_from_string as P; P(s2.session_duration)` → `7 days, 0:00:00`. A single `key=value` token only; `"days=7 seconds=0"` also errors.
|
||||
|
||||
## How it applies (stage.py:77-87, verified)
|
||||
`delta = timedelta_from_string(session_duration)`; `if remember: delta += remember_me_offset`; then `set_expiry(0) if delta==0 else set_expiry(delta)`. So any positive duration applies to EVERY login (remember-me only adds on top), and `set_expiry(<positive>)` overrides `SESSION_EXPIRE_AT_BROWSER_CLOSE` → persistent cookie. Existing sessions keep their old expiry; new logins only.
|
||||
|
||||
## SESSION_SAVE_EVERY_REQUEST (sliding window) is NOT available
|
||||
`/authentik/root/settings.py` reads `SESSION_COOKIE_AGE` from `CONFIG.get("sessions.unauthenticated_age","days=1")`, sets `SESSION_EXPIRE_AT_BROWSER_CLOSE=True`, and references **`SESSION_SAVE_EVERY_REQUEST` nowhere** → no `AUTHENTIK_*` env or config.yml key can flip it. Enabling a sliding window means patching the vendor image; instead raise the login-stage duration (absolute, supported).
|
||||
|
||||
## Audit the rest of the expiry surface (live values)
|
||||
- framework: `settings.SESSION_COOKIE_AGE`, `SESSION_SAVE_EVERY_REQUEST`
|
||||
- OAuth2 RPs: `apps.get_model("authentik_providers_oauth2","OAuth2Provider")` → `access_token_validity` / `refresh_token_validity` (this fleet: access 1h [embedded-outpost 24h], refresh 30d)
|
||||
- proxy claim cache: `authentik_providers_proxy.ProxySession.expires` (observed ~1–24h; silent re-auth, not a logout)
|
||||
- tenant: `authentik_tenants.Tenant.default_token_duration` (hand API tokens, not sessions)
|
||||
|
||||
## Notes
|
||||
- These stage/provider rows are DB state, NOT repo-managed → a blueprint re-import or fresh rebuild reverts them; drop a setup script/blueprint in the repo if reproducibility matters.
|
||||
- Don't probe the guarded vhost for the agent API and don't confuse Anubis' 168h challenge cookie with the authentik login session.
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
name: bash-cwd-drift-multi-worktree
|
||||
description: "Diagnose \"files keep reverting / phantom edits\" in multi-worktree sessions: the persistent bash shell's cwd silently reverted to another checkout, so probes and git commands ran against different files."
|
||||
---
|
||||
|
||||
# Multi-worktree bash cwd drift
|
||||
|
||||
## Symptom (all of these at once, in one session)
|
||||
- A file "keeps reverting" to an older version between edits (esp. test files).
|
||||
- `grep -n` line numbers disagree with the tool/test output (e.g. bats reports `line 637`, grep says the match is at 519).
|
||||
- `wc -l`/`md5sum` stable when sampled back-to-back, but state "flaps" across the session.
|
||||
- `git status` clean in one call, surprising in the next.
|
||||
- Probes against `src/` show code that doesn't exist in the branch you think you're on.
|
||||
|
||||
## Root cause
|
||||
The persistent bash shell resets to the session's ORIGINAL cwd when a call omits `cwd` (e.g. after a shell respawn at a turn boundary). With several git worktrees of the same repo checked out, consecutive calls silently alternate between two full copies of the tree — half the reads land in the old branch's files. Nothing is rewriting anything; two files are being inspected.
|
||||
|
||||
## Decision
|
||||
- In multi-worktree work, pass an **explicit `cwd` on EVERY bash call**. Never rely on persistence.
|
||||
- Use **absolute paths** for `read`/`edit`/`write` (already required) — file tools don't drift; only bash does.
|
||||
|
||||
## Check
|
||||
When state seems to flap: `pwd` in the same call as the probe. If two calls to the same relative path report different content, `git worktree list` and census each checkout separately — you have two files, not a gremlin.
|
||||
|
||||
## Repair pattern (if corruption got committed via the wrong tree)
|
||||
1. `git log --oneline --all -- <file>`; per candidate commit print `wc -l` + counts of a few marker strings to classify clean vs corrupted blobs.
|
||||
2. Restore the good blob with `git show <commit>:<path> > <path>` (no checkout — safe under worktree policies).
|
||||
3. Re-apply the intended additions as surgical edits; verify with targeted tests before the full suite.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: bats-assertion-trust
|
||||
description: "When writing or reviewing bats shell tests, catch vacuously-passing assertions: grep -vq, sed range boundaries, unanchored section slices, and fixtures too small to trigger pipefail/SIGPIPE bugs."
|
||||
---
|
||||
|
||||
Prove a bats assertion can actually fail before trusting that it passes. These four traps each made a green suite lie in real work.
|
||||
|
||||
## 1. `grep -vq` proves nothing
|
||||
|
||||
`some_list | grep -vq 'x'` succeeds when ANY line lacks `x` — nearly always true. To assert absence, capture the text and test it:
|
||||
|
||||
```bash
|
||||
s="$(report_section)"
|
||||
[[ "$s" != *libfoo* ]] # absence
|
||||
[[ "$s" == *bun* ]] # presence
|
||||
```
|
||||
|
||||
Same trap in reverse: bash `[[ $s == *A*B* ]]` — `*` crosses newlines, so a substring check spanning two lines matches the wrong lines. Slice to one line first, then assert.
|
||||
|
||||
## 2. Section slicing helpers must have explicit boundary semantics
|
||||
|
||||
A helper used to scope assertions to one report section must define, and comment, both ends. The default you want: **start inclusive** (labels carry the count/names), **terminator exclusive** (a `sed -n "/A/,/B/p"` range includes B and leaks the next section's first line):
|
||||
|
||||
```bash
|
||||
# Call sites pass a bare anchored regex; both ends documented.
|
||||
section() { printf '%s\n' "$output" | awk -v s="$1" -v e="$2" '$0 ~ e {on=0} $0 ~ s {on=1} on'; }
|
||||
```
|
||||
|
||||
Two bugs this shape hides:
|
||||
- Helper prepends `^` while call sites already pass `^name` → pattern `/^^name/` → never matches → empty slice.
|
||||
- An **unanchored start address** matches the first line and the range runs to end-of-output, so every absence assertion becomes a vacuous pass. Symptom: tests that "verify X is not in section A" pass while X is printed later in the same output.
|
||||
|
||||
After fixing a helper like this, re-run the whole suite: tests that were passing vacuously suddenly fail, and that is the fix working.
|
||||
|
||||
## 3. Fixtures too small hide pipefail bugs
|
||||
|
||||
`set -euo pipefail` + `sort | head -1`: `head` exits, closes the pipe, `sort` dies of SIGPIPE, the command substitution returns `141`, `set -e` aborts the script. Small fixtures never overflow the 64 KiB pipe buffer, so the suite stays green and the first run against real data fails with no output.
|
||||
|
||||
- Take a minimum in one pass: `awk '{ if (m=="" || $1<m) m=$1 } END { print m }' file`
|
||||
- `awk 'NR==1{print; exit}'` is **not** a fix — exiting early closes the pipe exactly like `head`.
|
||||
- Pin it with a fixture larger than the pipe buffer: `seq 1 20000 | awk '{print "[2026-01-01T10:00:00-0700] [ALPM] installed pkgfiller (1-1)"}' > big.log` (use `seq`, not `yes | head`, or the fixture itself SIGPIPEs).
|
||||
|
||||
## 4. `tr` swallows the final newline and downstream tools preserve that
|
||||
|
||||
`tr '\n' ' ' < file` emits no trailing newline; `fold`, `sed`, and `cat` faithfully preserve the absence, so the *next* output line gets glued on. Breaks any tool that later slices output by line-start labels.
|
||||
|
||||
```bash
|
||||
{ tr '\n' ' ' < file; echo; } | fold -s -w 88 | sed 's/^/ /'
|
||||
```
|
||||
|
||||
Pin with: assert the following section's label does NOT appear inside the preceding section's slice.
|
||||
|
||||
## Debugging order when a bats assertion fails but the script output looks right
|
||||
|
||||
1. Reproduce the exact command outside bats (`bash -x` against the script under test) — if it works there, the bug is in the *test helper*, not the script.
|
||||
2. Dump the constructed argument: `printf '%s' "$sed_script" | od -An -c` inside the helper. That is how `/^^ repo /` (double anchor) got found.
|
||||
3. Keep stubs faithful to the real interface: if the stub answers `-Sqq` but the script calls `-Sl`, the test measures nothing. Make the stub exit `64` on unhandled args so a new real query fails loudly instead of silently returning empty.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
name: bats-output-glob-red-proof-traps
|
||||
description: "Write red bats tests over command output: [[ ]] glob adjacency (quoted * is literal), run-capture vs test stdout in failure dumps, and proving red against pre-fix code before trusting green."
|
||||
---
|
||||
|
||||
Three traps hit while writing red-then-green bats tests for `remote-job.sh` (Refs #60, `dank/command-center` PR #61). Each one cost a detour because it makes a test look like a bug report when it is an assertion bug.
|
||||
|
||||
## 1. In `[[ $s == pat ]]` a quoted `*` is LITERAL — and it eats the wildcard beside it
|
||||
|
||||
```bash
|
||||
s=' x* DONE rc=0 1 lines'
|
||||
[[ "$s" == *"x*"DONE*"rc=0"* ]] # NOMATCH — and it looks like the tool under test broke
|
||||
[[ "$s" == " x*"*"DONE rc=0"* ]] # MATCH — quoted literals, UNQUOTED separators
|
||||
```
|
||||
|
||||
`*"x*"DONE*` means: anything, then the literal substring `x*DONE`, then anything. Quoting only makes the `*` inside the quotes literal; it does **not** keep the next unquoted token from butting up against it. When the subject has whitespace between the two facts, the pattern must carry an **unquoted** `*` between the quoted literals:
|
||||
|
||||
```
|
||||
"literal-a"*"literal-b" # a, then anything, then b
|
||||
```
|
||||
|
||||
Existing suites in the repo write `*"a"*DONE*rc=0*` — unquoted separators throughout — precisely for this reason. Copy that shape; do not quote for "clarity".
|
||||
|
||||
Discriminate before believing the test: run the pattern against BOTH the expected line and the line the buggy code produced (`correct-red` / `NOMATCH` on the second is the pass condition). A pattern that matches the old output was never a regression test.
|
||||
|
||||
## 2. A bats failure dump prints the TEST's stdout, not `$output`
|
||||
|
||||
```
|
||||
not ok 1 status treats a job name as a name, not a pattern
|
||||
# `[[ "$output" == *"x*"DONE* ]]` failed
|
||||
# started x*
|
||||
```
|
||||
|
||||
`started x*` is the stdout the test body itself printed (from a `start` call several lines earlier), **not** what `run` captured. Reading it as the captured value sends you to debug the tool. To see the real capture, echo it:
|
||||
|
||||
```bash
|
||||
run some-command
|
||||
echo "CAP:[${output}]"
|
||||
```
|
||||
|
||||
in a scratch copy of the test (`/tmp/dbg.bats`, same `setup`/harness) rather than adding diagnostics to the committed test.
|
||||
|
||||
## 3. Reproduce outside bats when green refuses to arrive
|
||||
|
||||
When a test fails but the same command works by hand, replicate the *exact* bats call shape in `/tmp` with the same env exports (`REMOTE_JOBS_DIR` etc.) and the same `run bash -c '…' _ "$A" "$B" "$C"` argument mapping. If the scratch copy passes and the committed one fails, the difference is the assertion, not the system — go back to trap 1.
|
||||
|
||||
## Red-proof rule
|
||||
|
||||
Run the new test against the pre-fix code and read *why* it failed. Both traps above produced a failure that looked meaningful while proving nothing (one matched the buggy output; one printed the wrong stream). Keep the test only when the failure reason is the defect you set out to pin.
|
||||
|
||||
Related: `bats-assertion-trust` (vacuous assertions), `red-verify-without-stash` (proving red in a shared worktree).
|
||||
@@ -1,82 +1,32 @@
|
||||
---
|
||||
name: bounding-subagent-runtime
|
||||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, agentIdleTtlMs, per-agent model overrides, the cfg save-approval quirk, and the finished-work-but-never-exits nested-fanout deadlock."
|
||||
description: "Cap and diagnose runaway or hung omp subagents: task.maxRuntimeMs, softRequestBudget, maxConcurrency, maxRecursionDepth (the 0-trap), per-provider in-flight ceilings."
|
||||
---
|
||||
|
||||
# Bounding subagent runtime (omp harness)
|
||||
## When
|
||||
Subagent runs long, fan out too wide, or you must (re)declare the fleet's subagent limit policy. Also when asked "what limits can subagents have".
|
||||
|
||||
Use when a `task`/`eval` subagent runs absurdly long (tens of minutes on a small job),
|
||||
when it **finishes its deliverable and still never exits**, when you want a hard cap
|
||||
before spawning, or when you must tell a busy agent to stop.
|
||||
## Read first
|
||||
- Repo KB page `omp-docs/pi/subagent-limits.md` — the live policy of record (values + rationale + deliberately-not-set list). Do not re-derive; read it.
|
||||
- `cfg://task` and `cfg://providers/maxInFlightRequests` — effective values, not the stock defaults.
|
||||
|
||||
## The knobs (`cfg://task/...`)
|
||||
|
||||
| Key | Default | Meaning |
|
||||
## The knobs (cfg:// writes; `/save` suffix persists)
|
||||
| Knob | Effect | Gotcha |
|
||||
|---|---|---|
|
||||
| `maxRuntimeMs` | `0` (unlimited) | **Hard wall-clock limit per subagent (ms).** The only true time cap. |
|
||||
| `softRequestBudget` | `200` | Soft cap on assistant **requests** per run; force-stop at **1.5×**. |
|
||||
| `softRequestBudgetNotice` | `true` | Injects one "wrap up" steering message at the soft mark. |
|
||||
| `agentIdleTtlMs` | `420000` | Parks an **idle** agent after 7 min. Useless against a busy-but-slow agent. |
|
||||
| `completionProbe` | `true` | Asks a working subagent to self-report completeness at 2/5/10 requests. |
|
||||
| `maxConcurrency` | `32` (often lowered) | Concurrent subagents; sizes GPU/endpoint load. |
|
||||
| `agentModelOverrides` | `{}` | Per-agent model routing, e.g. `{"scout":"provider/model-id"}`. |
|
||||
| `tools.maxTimeout` | `0` | Caps the timeout the agent may set on a **tool**, not on the agent. |
|
||||
| `task.maxConcurrency` | session semaphore; excess queues | stock default 32 — "2" is not a default |
|
||||
| `task.maxRecursionDepth` | nesting cap | predicate is `depth < n`; **1 = children cannot spawn (correct for "no nesting"); 0 disables spawning for the MAIN session too — never reach for 0**; negative = unlimited |
|
||||
| `task.maxRuntimeMs` | hard wall clock, overrun = `aborted` | default 0 = off |
|
||||
| `task.softRequestBudget` | wrap-up notice at N, force-yield at 1.5N | scout/sonic may impose lower built-ins |
|
||||
| `task.agentIdleTtlMs` | park idle finished agents | `<= 0` disables parking |
|
||||
| `providers.maxInFlightRequests` | concurrent LLM requests per provider id | keys are the quoted block labels from `agent/models.yml` (e.g. `"llamacpp (dankclaw-3090)"`), flat map lookup; absent/≤0 = unlimited; scope = this machine's config root only — other workstations and non-omp clients are NOT counted; excess queues |
|
||||
| `tools.maxTimeout` | ceiling on any agent-requested tool timeout | kills `timeout: 0` runaways |
|
||||
| `task.isolation.enabled` | makes per-spawn `isolated` flag available | opt-in per spawn, not a blanket sandbox; needs git repo; off in plan mode |
|
||||
| `task.completionProbe` | periodic completeness side-requests | token sink on shared endpoints |
|
||||
|
||||
Reasonable fleet defaults: `maxRuntimeMs = 900000` (15 min), leave the request budget
|
||||
alone unless the failure mode is a looping search agent (then lower it — request count,
|
||||
not the clock, is what bounds a 200-call research loop).
|
||||
## Verify semantics from the bundle, not the label
|
||||
Setting descriptions lie by omission. `~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`:
|
||||
- depth gate: `grep -o '.\{130\}xC\b.\{130\}'` finds `U_(n,s)`; `grep -o 'function U_(e,t){.\{0,60\}'` → `e<0||t<e` (n=cap, s=taskDepth).
|
||||
- in-flight lookup: `grep -o '.\{150\}axInFlightRequests.\{200\}'` → flat `[providerId]` lookup, non-number/≤0 = unlimited, clamp ≥1.
|
||||
|
||||
**Verify the cap instead of assuming it.** `read cfg://task/maxRuntimeMs` at the start of
|
||||
any session that plans to rely on it. A session-only write is gone next session, and an
|
||||
unanswered `/save` approval means the value never persisted — so the fleet can run hours
|
||||
with the cap at `0` while everyone believes it is 15 minutes. Nothing else bounds a child.
|
||||
|
||||
## Three failure modes, three different fixes
|
||||
|
||||
- **Runaway loop** (hundreds of sequential tool calls): lower `softRequestBudget`; the
|
||||
clock cap only truncates it late.
|
||||
- **Slow model, legitimate work** (dozens of calls on a low-t/s endpoint = an hour of wall
|
||||
clock): fix routing, not the clock — check `cfg://task/agentModelOverrides` and move the
|
||||
agent class onto a faster/shared endpoint. Watch the slot math: subagent endpoints share
|
||||
KV slots with the interactive session, so `maxConcurrency + 1` sequences on an N-slot
|
||||
endpoint degrades everyone's decode.
|
||||
- **Finished work, no exit (nested fan-out deadlock).** A subagent that spawned **its own**
|
||||
children blocks on their barriers. If those children die out-of-band (killed, starved, or
|
||||
waiting on an answer whose subject changed mid-flight), the parent waits forever with its
|
||||
deliverable already written. No instruction text prevents this — not the repo AGENTS.md,
|
||||
not the task prompt; only the clock cap or a kill ends it.
|
||||
|
||||
Signature: parent and children report **the same elapsed wall clock**, and the parent's
|
||||
on-disk output is complete. Census (agent registry + `ps`): group by elapsed time, look
|
||||
for identical `etimes` across a parent/child set, then judge the **artifacts** instead of
|
||||
the process — balanced code fences, no truncated tables, trailing newline, front matter or
|
||||
the page's H1+status convention intact. Complete artifacts + still "running" = deadlock,
|
||||
not work.
|
||||
|
||||
Repair: kill the **subtree** and confirm the children leave (a killed parent does not
|
||||
cancel children that came from a separate `tasks[]` batch). Then **flatten**: the
|
||||
top-level session does the fan-out and children never spawn children. Re-run only the
|
||||
slices whose artifacts are missing — integrate from what is already on disk.
|
||||
|
||||
## cfg write mechanics (bit me once)
|
||||
|
||||
- `write cfg://task/maxRuntimeMs 900000` → applies **session-only**, replies "NOT saved".
|
||||
- Persist = `write cfg://task/maxRuntimeMs/save` with the same content, and that write
|
||||
needs its **own interactive approval**. If the approval prompt is not answered in time
|
||||
the value stays session-only and the tool says *NEVER retry unless the user asks* —
|
||||
report it in the reply instead.
|
||||
- Revert a session value by writing the old value back (e.g. `0`).
|
||||
|
||||
## Order of operations when one is already running late
|
||||
|
||||
1. `read proc://` → registry of running agents; `read proc://<id>` for status/output.
|
||||
2. `read agent://<id>` → live progress ("Latest assistant text") — often shows it is
|
||||
mid-batch, not hung. If the latest text is already a wrap-up/summary, treat it as
|
||||
**done** and collect its output instead of waiting for an exit event.
|
||||
3. `write agent://<id>` an explicit wrap-up order: stop searching, answer what is
|
||||
sourced, mark the rest `NOT VERIFIED — needs <what>`, cap the report length.
|
||||
4. Only then `write proc://<id>/kill` — the cap does **not** retroactively bind agents
|
||||
spawned before the change, so a long runner must be ordered or killed, never just
|
||||
re-configured. After any kill, re-read the registry: unreaped children and orphaned
|
||||
host-side processes (a scratch server holding GPU/RAM) are the expensive residue, and
|
||||
`pkill -f` patterns that also match production services are forbidden — kill by pid.
|
||||
## Landing a policy change
|
||||
cfg `/save` writes persist immediately; `agent/config.yml` is machine-live tier (heartbeat commits it). Human-facing policy changes land as an `omp-docs` page update via branch → PR (docs tier), `git worktree add` from `origin/main` if the current checkout is branch-locked by workspace policy.
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: browser-screenshot-disk-and-cookies
|
||||
description: Save omp managed-browser screenshots to real files (tab.screenshot path is silently ignored) and reset the shared Chromium cookie jar between persona captures.
|
||||
---
|
||||
|
||||
# Screenshots to disk with the omp `browser` eval helper
|
||||
|
||||
## Traps (both cost a full re-capture to discover)
|
||||
1. `tab.screenshot({ path })` **ignores `path` silently** — the option is not in
|
||||
its schema (`{selector, fullPage, silent, annotate, format, quality, ifChanged, threshold}`).
|
||||
The image attaches inline to the tool result and **nothing is written to disk**;
|
||||
`ls` shows an empty dir. Inline images may also be dropped from transcript
|
||||
archives later, so "attached to chat" is not "saved".
|
||||
2. The shared managed Chromium keeps **cookies across tabs and turns** (profile
|
||||
persists after `tab.close()`). A second capture script that expects the
|
||||
anonymous view lands signed-in and `waitForSelector("testid/…")` fails fast.
|
||||
|
||||
## Procedure
|
||||
Write files via `tab.run`, which exposes the raw Puppeteer `page` — its
|
||||
`screenshot({path})` does write:
|
||||
|
||||
```js
|
||||
const dir = "/tmp/<session-dir>";
|
||||
const tab = await browser.open({ name: "caps", url });
|
||||
const shot = (file) =>
|
||||
tab.run(async ({ page }, f) => { await page.screenshot({ path: f }); return f; },
|
||||
{ args: [`${dir}/${file}.png`] });
|
||||
|
||||
await tab.clearCookies(); // ALWAYS before an anonymous capture
|
||||
await tab.goto(url);
|
||||
await tab.waitForSelector("testid/first-state");
|
||||
await shot("1-anonymous");
|
||||
// ...drive states with direct helpers: click/fill("testid/<id>", ...) ...
|
||||
await shot("2-next-state");
|
||||
await tab.close();
|
||||
```
|
||||
|
||||
Verify with `ls -la <dir> && file <dir>/*.png` — do not trust silent success.
|
||||
|
||||
## Notes
|
||||
- Selectors accept `testid/<id>` handler; prefer it over CSS on `[data-testid]`.
|
||||
- `tab.run(fn, { args })`: functions receive `{tab, page, browser, wait, assert}`
|
||||
first, then each arg; a bare code string runs in an **isolated runtime, not the
|
||||
page** — `document` is undefined there. Use direct helpers or `tab.evaluate`
|
||||
(page-global expression; top-level `return` invalid) for page access.
|
||||
- Byte-identical PNG sizes between two frames = same view; fine as proof of a
|
||||
round-trip (state A → B → A).
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
name: command-center-bundle-lint-gate
|
||||
description: "Before committing docs in dank/command-center, lint the real omp-docs bundle directly — the bats suite only lints fixtures, so a green suite is not proof."
|
||||
---
|
||||
|
||||
# The bundle lint gate (`dank/command-center`)
|
||||
|
||||
Adding or editing pages under `omp-docs/`? `bats tests/` going green proves
|
||||
nothing about the pages you just wrote.
|
||||
|
||||
## Why this exists
|
||||
|
||||
`tests/kb-lint.bats` exercises `scripts/kb-lint.sh` against `mktemp -d` fixtures
|
||||
(`"conformant bundle passes"`, `"missing frontmatter is reported"`, …). For years
|
||||
nothing in the suite pointed the linter at the shipped tree, so a page with no
|
||||
frontmatter, an index that stopped listing a file, or a broken relative link
|
||||
committed under a green suite. (Two tests appended 2026-10-20 now run both
|
||||
checkers against `omp-docs/`; verify they are still present before assuming the
|
||||
suite covers it — `grep -n omp-docs tests/kb-lint.bats`.)
|
||||
|
||||
The general law: **a lint suite that tests only the linter does not lint your
|
||||
content.** Check which one a suite runs before calling green a gate.
|
||||
|
||||
## Gate a docs commit
|
||||
|
||||
```bash
|
||||
cd <worktree>
|
||||
bats tests/ ; grep -E '^1\.\.' /tmp/… # suite (count, failures)
|
||||
./scripts/kb-lint.sh omp-docs # want: kb-lint: 0 violation(s)
|
||||
python3 scripts/kb-coverage.py omp-docs # want: index-coverage: 0 issue(s)
|
||||
```
|
||||
|
||||
`kb-lint.sh <kb-dir>` (default `omp-docs`) = OKF v0.1 conformance **plus link
|
||||
integrity**, so it is the check that absolute bundle links like
|
||||
`/tools/gitea/client-access.md` resolve. `kb-coverage.py` = every page listed by
|
||||
its directory `index.md` (repo doctrine: nothing may appear in a directory its
|
||||
`index.md` does not list).
|
||||
|
||||
OKF shape that passes: every concept page carries frontmatter with a `type`
|
||||
(`type: reference` + `title`/`description`/`tags`/`timestamp`/`verified`);
|
||||
`index.md` files carry **no** frontmatter; new pages get an entry in their
|
||||
directory index in the same commit, plus a dated entry in `omp-docs/log.md`
|
||||
(one entry = what changed + the durable lesson, a few lines, never a narrative).
|
||||
|
||||
## Commit-message trap in this harness
|
||||
|
||||
A multi-line message written as a heredoc inside a chained `bash` command
|
||||
(`git commit -q -F - <<'MSG' … && …`) fails with
|
||||
`pi-natives:command: syntax error at end of input`. Write the message to
|
||||
`/tmp/msg-<topic>.txt` with the write tool and `git commit -q -F /tmp/msg-<topic>.txt`.
|
||||
Keep the `&&` chain so a failed gate stops the commit — hooks (secret gate) stay armed,
|
||||
never `--no-verify`.
|
||||
|
||||
## Land and reap
|
||||
|
||||
Merge is form-encoded `do=merge&merge_style=no-ff` and answers `200` with an
|
||||
**empty body**; never read that as ambiguous — verify by landing. Then
|
||||
`git -C ~/.omp merge --ff-only origin/main` (the checkout *is* the deployment;
|
||||
dirty `agent/memories/`, `agent/config.yml`, `APPEND_SYSTEM.md` are machine-tier
|
||||
live state and do not block an ff whose diff misses them), fast-forward the
|
||||
session's `wt/*` scratch branch the same way, then `git worktree remove <build-wt>`
|
||||
and `git branch -d <type>/<issue>-<slug>` (`-d` succeeding is itself proof of
|
||||
merge).
|
||||
|
||||
Stray files whose names start with `-` (e.g. `-c`, `-es` — vim takes a
|
||||
`--startuptime` flag name as the output filename) show as untracked dirt:
|
||||
`mv ./-c .AGENTTRASH/<date>-<meaning>.log`, never `rm`.
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
name: corpus-voice-skill-rewrite
|
||||
description: "Rewrite a skill to emulate an author corpus's voice: full read, style-DNA distillation from writing-for-agents levers, positive-phrasing pass, sharpened completion criteria"
|
||||
---
|
||||
|
||||
# Corpus-voice skill rewrite
|
||||
|
||||
Rewrite a skill so it reads as if a chosen author wrote it. Done for `grilling` against the `mattpocock/skills` corpus (2026-10-18, command-center PR #20); behaviour preserved, voice swapped.
|
||||
|
||||
## Where the corpus lives
|
||||
|
||||
Installed third-party skills: `~/.agents/skills/<name>/SKILL.md` (the skills-CLI global store; `~/.agents/.skill-lock.json` names sources). Whole mattpocock corpus ≈ 2,800 lines — small enough to read inline in ~4 `cat` batches via bash; do NOT delegate the reading, style lives in the exact prose.
|
||||
|
||||
Read the author's own meta-skill FIRST if one exists (`writing-for-agents` for mattpocock — it states his levers explicitly), then the corpus practicing them.
|
||||
|
||||
## Style DNA checklist (mattpocock; generalise per author)
|
||||
|
||||
- **Leading words**: compact pretrained concepts repeated as tokens, never sentences (_red_, _seam_, _fog of war_, _brief_, _frontier_). Define each once, reuse everywhere. Prefer words already in the author's other skills so the corpus shares one language.
|
||||
- **Vocabulary-first**: bolded glossary terms, then used exactly; explicit "_Avoid_: component, service".
|
||||
- **Positive phrasing**: prohibitions rewritten as target behaviours; keep a prohibition only when no positive form exists (hard guardrails).
|
||||
- **Sharpened completion criteria**: fuzzy bounds ("understanding reached") → checkable + exhaustive ("frontier empty and nothing sits blocked", "one artefact per question, counted before sending").
|
||||
- **Zero history**: no provenance, attribution, or absorbed-from lineage in the body.
|
||||
- **No-op hunting**: delete harness defaults and anything the environment already confesses (formats, rendering, caps). Gotchas the environment CANNOT confess (exclusive dialogs, Esc-abort semantics) stay inline — they're the cache worth keeping.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Read entire corpus; tabulate levers with per-skill evidence before writing.
|
||||
2. Rewrite preserving settled behaviour 1:1 — voice only; diff the two versions for semantic drift.
|
||||
3. Audit pass: grep the new text for history words (extracted/absorbed/from/upstream), machine-specific constants, repo-specific paths.
|
||||
4. Run repo checks, commit as `style(<scope>)`, note the lever→instance mapping in the PR comment so the reviewer can grade the emulation.
|
||||
@@ -1,49 +1,62 @@
|
||||
---
|
||||
name: dankclaw-pihole-lan-dns-record
|
||||
description: "Add/repair a LAN DNS record for a new *.franksdank.cloud service on Dankclaw's Pi-hole v6 (hosts/custom.list path), plus the public Porkbun A record step and client cache gotchas."
|
||||
description: "Add/repair a LAN DNS record for a new *.franksdank.cloud service on Dankclaw's Pi-hole v6 (hosts/custom.list + repo addn-hosts, plus the public Porkbun A record side) — and triage \"LAN names stopped resolving\" including the FTL boot-time record-load failure."
|
||||
---
|
||||
|
||||
Use when a new `*.franksdank.cloud` service resolves publicly/LAN-needs a local address, on Dankclaw (Pi-hole v6 container, quadlet `pihole.container`).
|
||||
# Dankclaw Pi-hole LAN DNS record
|
||||
|
||||
## Ground truth layout (verified 2026-10-11)
|
||||
## What this fixes
|
||||
|
||||
- Quadlet: `~/.config/containers/systemd/pihole.container`; volumes `/home/dank/Containers/pihole/{etc-pihole,etc-dockers,addn-hosts}`.
|
||||
- **LAN records live in `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`** — plain hosts format: `192.168.0.143 gitea.franksdank.cloud`. This is the boring, working path.
|
||||
- `pihole.toml` `[[dns.records]]` array is EMPTY on this host — do not expect v6 UI-style records there.
|
||||
- `gravity.db` exists but has NO `domain` table (v6 layout) — do not insert local records into SQLite.
|
||||
A new `*.franksdank.cloud` service resolves publicly (via Porkbun → `franksdank.cloud` A record → Caddy on Dankclaw `192.168.0.143`) but **does not resolve on the LAN**. Pi-hole is the LAN DNS server (`dankclaw_default` bridge, published port 53) and serves local records only from files it imports.
|
||||
|
||||
## Procedure (30 s, proven)
|
||||
## Live truth
|
||||
|
||||
```bash
|
||||
ssh dankclaw 'echo "192.168.0.143 <name>.franksdank.cloud" | sudo tee -a /home/dank/Containers/pihole/etc-pihole/hosts/custom.list >/dev/null \
|
||||
&& podman exec pihole pihole restartdns'
|
||||
ssh dankclaw 'getent hosts <name>.franksdank.cloud' # must resolve
|
||||
dig +short <name>.franksdank.cloud @192.168.0.143 # from any LAN box
|
||||
- **The repo owns the records.** `dankclaw/pihole/addn-hosts` (hosts-style lines: `192.168.0.143 <name>`) is copied by `deploy-remote.sh` to `/home/dank/Containers/pihole/addn-hosts` and bind-mounted **read-only** into the container at `/etc/pihole/hosts/addn-hosts` (`Volume=/home/dank/Containers/pihole/addn-hosts:/etc/pihole/hosts/addn-hosts:ro` in `dankclaw/pihole/quadlet/pihole.container`). Add the name there, in the repo, then deploy — that is the durable path. `tests/pihole.bats` validates every line (IPv4, LAN range, name suffix, Caddyfile cross-check).
|
||||
- `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list` also exists on the host (older mechanism, mode `640` owned by the container uid, root reads it with `sudo`). It is **not** the repo's owner file — do not hand-edit it as the fix.
|
||||
- `pihole.toml` carries a `dns.hosts = [ "192.168.0.143 …", … ]` array too (FTL's own in-config records). Both sources exist; if LAN names fail, neither is being applied.
|
||||
- Pi-hole runs as a rootless quadlet named `pihole` — query it with `podman exec pihole …`, never `sudo podman` (per-user storage law).
|
||||
- `pihole.franksdank.cloud` resolves through Pi-hole itself (loop/edge), so a vhost record pointing at `192.168.0.143` is correct for every Caddy-fronted service.
|
||||
|
||||
## Procedure (repo-first)
|
||||
|
||||
1. Append the line to `dankclaw/pihole/addn-hosts`:
|
||||
```
|
||||
192.168.0.143 <name>.franksdank.cloud
|
||||
```
|
||||
One record per vhost; keep it in sync with the Caddyfile (the bats seam cross-checks the two).
|
||||
2. `./tests/run.sh` (the `pihole` seam validates the file), commit, push.
|
||||
3. Deploy so the copy step runs: `./scripts/deploy.sh dankclaw --service pihole`.
|
||||
4. Verify from the box: `getent hosts <name>.franksdank.cloud` → `192.168.0.143`.
|
||||
5. Public edge record (optional; skip when every subscriber is LAN-only): Porkbun `POST /api/v3/json` with `apikey`/`secretapikey`/`command=add`/`type=A`/`hostname=<name>`/`data=76.49.118.136` (see `dankclaw-porkbun-edge`).
|
||||
|
||||
Client-side proof (avoid the resolver trap):
|
||||
|
||||
```
|
||||
sudo resolvectl flush-caches # systemd-resolved otherwise keeps the negative cache
|
||||
curl -sv --max-time 5 https://<name>.franksdank.cloud 2>&1 | grep 'Connected to'
|
||||
```
|
||||
|
||||
Hosts files are read-only-readable only via sudo (dir owned by uid 100999). `sudo -n` works on dankclaw.
|
||||
Expected: `Connected to <name>.franksdank.cloud (192.168.0.143)` — LAN route, not the public IP.
|
||||
|
||||
## API maze — DON'T re-explore
|
||||
## Triage: every LAN name broke at once (2026-10-19 law)
|
||||
|
||||
- FTL API base is `/api` at the web port (8083) **with `Host: pi.hole` header**; `/admin/api/*` → bad_request hint; bare 127.0.0.1 without the vhost → 404 `not_found`.
|
||||
- Login: `POST /api/login {"password": <WEBPASSWORD from /home/dank/Containers/pihole/env>}` → session in `X-Cors` header/cookie.
|
||||
- `POST /api/apps/domains/custom` → **404 on this build** (FTL version predates it); `/api/customdns/custom` also absent. There is no container sqlite3. Use custom.list.
|
||||
Signature: **all** `*.franksdank.cloud` return NXDOMAIN / Playwright `ERR_NAME_NOT_RESOLVED` while **external names still resolve**. The external-OK/local-fail split is the tell: FTL is answering external names from cache and holds **no local records at all** after a container/host restart — the mounts and files are intact and readable inside the container (`ls -l /etc/pihole/hosts/`, `grep -c . …/addn-hosts`), `dns.hosts` in `pihole.toml` is complete. The mechanism is unestablished; do not go hunting for a lost file.
|
||||
|
||||
## Public record (off-LAN reachability)
|
||||
|
||||
Porkbun API, keys in `/home/dank/Containers/caddy/env` (`PORKBUN_API_KEY`, `PORKBUN_API_SECRET_KEY`):
|
||||
|
||||
```bash
|
||||
curl -sL -X POST https://api.porkbun.com/api/json3/domain/api/create/franksdank.cloud \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"apikey\":\"$K\",\"secretapi\":\"$S\",\"name\":\"<name>\",\"type\":\"A\",\"content\":\"71.227.227.172\",\"ttl\":\"600\"}"
|
||||
```
|
||||
ssh dankclaw 'getent hosts grafana.franksdank.cloud' # empty => FTL, not the client
|
||||
ssh dankclaw 'podman exec pihole pihole restartdns reload' # fixes it in seconds
|
||||
```
|
||||
|
||||
- `json3` path 302-redirects → **`-L` required**; no-`-s` to see status.
|
||||
- Porkbun edge (openresty) intermittently 503s — transient; retry later, don't burn turns. Public record is OPTIONAL when LAN-only subscribers exist.
|
||||
- Wildcard `*.franksdank.cloud` TLS is already covered: Caddy + Porkbun DNS-01 (caddy-porkbun image), so a new subdomain needs NO new cert work — just the Caddyfile site block + reload.
|
||||
Distinguish before acting: a **single** client failing while others work is FTL rate-limiting that client — `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`; a `10.0.2.x` client address is a pasta container's DNS forwarder (SNAT), not a machine. Full causal page: `serverAppConfigs-docs/Infrastructure/Pi-hole.md` (**Records** pitfall + Failure modes table).
|
||||
|
||||
## Client-side cache gotchas
|
||||
## Gotchas
|
||||
|
||||
- Workstation (systemd-resolved): negative cache survives Pi-hole changes; `sudo resolvectl flush-caches` needs a TTY (faillock-sensitive — no `-S` guessing). Non-interactive proof: `curl --resolve <name>.franksdank.cloud:443:192.168.0.143 https://<name>.franksdank.cloud/...`; cache self-expires (SOA negative TTL).
|
||||
- Dankclaw/Dankserver resolve via Pi-hole directly — verify there first.
|
||||
- Do NOT insert local records into SQLite. v6's `gravity.db` `domain` table has NO `type=2` custom-DNS rows in this build.
|
||||
- `pihole restartdns reload` re-imports the hosts files **and** `dns.hosts`; that is the remedy for the boot-load failure above. Plain `pihole restartdns` also restarts the resolver — brief LAN-wide DNS blip.
|
||||
- `dig @<ip> <name>` from the workstation bypasses systemd-resolved — use it to check FTL's answer directly (`+short @192.168.0.143`).
|
||||
|
||||
## Cross-reference
|
||||
|
||||
- Fleet DNS flow: `dankclaw/Caddyfile` (Caddy binds `:443` on the host, serves `*.franksdank.cloud` vhosts)
|
||||
- Pi-hole service page: `serverAppConfigs-docs/Infrastructure/Pi-hole.md`, flow page `Infrastructure/LAN-DNS.md`
|
||||
- Public edge: managed skill `dankclaw-porkbun-edge`
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: dankvoice-bats-serialization
|
||||
description: "Run dankvoice bats suites without hangs or false failures: serialize dist rebuilds, never edit src mid-run, and rebuild before committing the tracked bundle."
|
||||
---
|
||||
|
||||
Use when running the dankvoice `bats tests/` suite on the shared box, when a suite run hangs or times out, or when the "committed dist is byte-identical to a fresh build" pin test fails after a commit.
|
||||
|
||||
## The two serialization hazards
|
||||
|
||||
### 1. `setup()` rebuilds `dist/` from `src/`
|
||||
Every test runs `cd "$REPO" && bun run build` in `setup()`. Consequences:
|
||||
|
||||
- **Never edit `src/` while a suite is running.** The next test's `setup()` compiles whatever is on disk at that instant, so an in-flight edit gets attributed to the run and produces a failure with no owner. Finish the gate, then write.
|
||||
- **Two suites on the same checkout race on `dist/`.** Serialize: run one, wait for its report, then start the next. Multiple worktrees of the repo each build their own `dist/` and do not collide.
|
||||
- Suite runtime is ~85 s for 55 tests; a backgrounded job with a 600 s deadline is the right shape (`timeout: 600`, `async: true`). Never poll it — `setup()` output lands in the conversation when the job ends.
|
||||
|
||||
### 2. `dist/` is tracked, and a pin test enforces it
|
||||
One test rebuilds and compares against the committed bundle. Therefore the commit order is:
|
||||
|
||||
```bash
|
||||
bun run build # BEFORE staging
|
||||
git add -A && git commit -m '…'
|
||||
bats tests/ # gate AFTER the commit
|
||||
```
|
||||
|
||||
Committing `src/` changes without rebuilding first fails the pin test (`not ok … byte-identical to a fresh build`) — the tracked bundle is the *previous* state. If that happens on an unpushed branch, fix it by amend rather than a follow-up commit:
|
||||
|
||||
```bash
|
||||
bun run build && git add dist && git commit --amend --no-edit
|
||||
```
|
||||
|
||||
Verify freshness any time the pin is in doubt:
|
||||
|
||||
```bash
|
||||
bun run build >/dev/null 2>&1 && git status --porcelain dist # empty = committed == fresh build
|
||||
```
|
||||
|
||||
## If a backgrounded gate job vanishes
|
||||
A crash can kill the job before its report; `read proc://bg_N` returning "not found" means it is gone, not that it failed. Reconstruct what landed before resuming — `git log --oneline -3`, `git status --porcelain`, then the freshness check above — and rerun the suite; the commit may well have succeeded while the gate never ran.
|
||||
|
||||
## Gotchas
|
||||
- `DV_HOLD` is seconds (`sleep "${DV_HOLD:-0.1}"`), so `DV_HOLD=3` really holds the stub for three seconds.
|
||||
- Isolation tests must not count shared state: any assertion that sweeps `$TMPDIR` or `~/.omp/cache` observes every other live session on the box (each narrating session creates `dankvoice-*` scratch dirs). Point `TMPDIR` at a private directory instead — see the `cutting off mid-utterance leaves no orphaned audio behind` test.
|
||||
- `bun run build` does not typecheck. Run `bunx tsc --noEmit` separately when moving code between modules; the bundler happily emits a bundle that references a deleted identifier.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: dankvoice-bundle-swap-red-proof
|
||||
description: "Prove a dankvoice behavior test is red against the pre-change bundle by pointing DV_EXT at git show HEAD:dist/index.js, with the driver extracted from the bats heredoc into a pinned scratch directory. Use before claiming a new bats assertion is red-capable."
|
||||
---
|
||||
|
||||
Prove a new `tests/behavior.bats` assertion is **red-capable** — that it fails against the *previous* build, not just passes against the new one — without stashing, branching, or editing `src/`.
|
||||
|
||||
## Why
|
||||
|
||||
`tests/behavior.bats` is a black-box suite: the plugin bundle is loaded through `DV_EXT`, so a *different bundle* is the red side. Stashing the fix would break the `dist` byte-identity pin test and the shared worktree.
|
||||
|
||||
## Working recipe
|
||||
|
||||
```bash
|
||||
cd ~/Projects/dankvoice # or the feature worktree: ~/Projects/wt-dv-*
|
||||
bun run build # current src = the green side
|
||||
git show HEAD:dist/index.js > /tmp/old-dist.js # the red side (pre-change bundle)
|
||||
|
||||
W=/tmp/dv-redproof # PINNED name; see traps below
|
||||
rm -rf "$W" && mkdir -p "$W/bin"
|
||||
cat > "$W/player.sh" <<'SH'
|
||||
#!/usr/bin/env bash
|
||||
printf '%s\n' "$*" >> "$DV_STUB_LOG"
|
||||
for arg in "$@"; do case "$arg" in *.wav) cp "$arg" "$DV_COPY" ;; esac; done
|
||||
sleep "${DV_HOLD:-0.1}"
|
||||
SH
|
||||
chmod +x "$W/player.sh"
|
||||
ln -s "$W/player.sh" "$W/bin/pw-play"
|
||||
printf '#!/usr/bin/env bash\nprintf "116\\talsa_output.usb-KT.0\\taudio/sink\\t*\\n"\n' > "$W/bin/wpctl"
|
||||
chmod +x "$W/bin/wpctl"
|
||||
# The driver lives inside the bats file as a heredoc: extract it verbatim.
|
||||
end=$(grep -n '^TS$' tests/behavior.bats | cut -d: -f1)
|
||||
sed -n "86,$((end - 1))p" tests/behavior.bats > "$W/driver.ts"
|
||||
|
||||
export DV_EXT="$PWD/dist/index.js" DV_STUB="$W/player.sh" DV_STUB_LOG="$W/player.log" \
|
||||
DV_COPY="$W/played.wav" DV_OMPCONFIG=/dev/null DANKVOICE_CONFIG="$W/settings.json" \
|
||||
DV_MODELS="$W/models.yml" DV_WPCTL=x DV_SELECT=""
|
||||
for ext in /tmp/old-dist.js "$PWD/dist/index.js"; do
|
||||
rm -f "$W/player.log" "$DANKVOICE_CONFIG"
|
||||
DV_EXT="$ext" bun "$W/driver.ts" <scenario> 2>&1 | grep -E '^PLAYS|^CALLS|^NOTIFY|^SETTINGS'
|
||||
done
|
||||
```
|
||||
|
||||
The scenario name is the driver's third argv (`speak`, `auto`, `repeat`, `humanizecycle`, `badurl`, …); the bats `-f` test body's env (`DV_PLAYER_BIN`, `DV_SETTINGS_JSON`, `DANKVOICE_VERBOSITY`, …) is passed the same way. `DV_SELECT` holds the menu answers, `|`-separated, consumed in order.
|
||||
|
||||
## Traps, each one cost a turn
|
||||
|
||||
- **The scratch directory must be pinned by name.** `ls -dt /tmp/tmp.*` picks up *other sessions'* temp dirs, and then `player.sh` and `driver.ts` live in different directories — the run silently does the wrong thing.
|
||||
- **The stub must append to `$DV_STUB_LOG`.** The driver counts player invocations from that file; a stub that only `sleep`s reports `PLAYS 0` for *both* bundles and the whole proof reads as "fix does nothing".
|
||||
- **`DV_EXT` is required.** Missing it yields `Cannot find package 'undefined'` from the driver, which looks like a broken driver, not a missing env var.
|
||||
- **`DV_SETTINGS_JSON` supports `__PORT__`** (the driver substitutes the fixture port), so a preseed can name a prefixed route — needed whenever the test is about *which* endpoint got resolved.
|
||||
- **Never run this while a `bats` run is in flight** in the same repo: `setup()` runs `bun run build`, so two runs clobber `dist/` and the pin test.
|
||||
|
||||
## Reading the result
|
||||
|
||||
Red side must show the *defect* (`PLAYS 0`, `644`, a chat `CALL` with the wrong prefix, `Invalid URL`), green side the fix — assert on printed markers only (`PLAYS`, `CALLS`, `CALL <url>`, `NOTIFY`, `SETTINGS`). `CALLS` counts `/audio/speech` calls only; the rewrite call appears as its own `CALL /…/chat/completions … sys=<full system prompt>` line, which is how prompt-level knobs (verbosity, heckle) are observed.
|
||||
|
||||
Prefer this over a synthetic leak: if the check already caught a real leftover/failure once, that event *is* the sensitivity proof — say so in the commit message instead of faking one.
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
name: dankvoice-epithet-pool-audit
|
||||
description: "Grow or audit the dankvoice 1000-name epithet pool: gender classes, individual eviction, typo sweep, safe regeneration into framing.ts."
|
||||
---
|
||||
|
||||
# Dankvoice epithet pool audit
|
||||
|
||||
Use when adding names to, reclassifying, or cleaning `src/framing.ts` `EPITHETS_*` in `dank/dankvoice`. The pool is law: exactly 1000 distinct names, curated at build time, never user-editable, "the commons, never the characters".
|
||||
|
||||
## Classification rules (proven traps in parentheses)
|
||||
- **man**: gendered lexicon (king, lord, priest, uncle...) + tokens ending `man|men|master` — but never `human|inhuman|nonhuman`. Compounds like `quartermaster`, `hangwoman` are single tokens: token-equality checks miss them, use suffix rules.
|
||||
- **woman**: lexicon (queen, sister, daughter...) + tokens ending `woman` + an EXPLICIT `-ess` title list (baroness, countess, prioress, canoness, chartress, heiress, priestess...). Never a bare `-ess` suffix match: `endless`, `harmless`, `fortress` all end in "ess".
|
||||
- **Possessives follow the head noun**: `lieutenant of the king's hull` is a lieutenant → neutral commons; `count's armsman` is an armsman → man. Do not gender by words inside `'s` phrases.
|
||||
- Modern-neutral offices stay commons: hero, governor, wizard. `witch/lass/bride` are female-coded; `warlock` male-coded.
|
||||
- Regex trap for absence checks: `he\b` matches inside `the`; token-split before matching.
|
||||
|
||||
## Individual eviction law
|
||||
Evict: named fictional/historical individuals (duke caladan, uncle toby, pyrrhus, usul, endymion, skynet, moya-born, medusa), named protagonist ships (rocinante), and real authors in universe clothing (strugatsky, niven).
|
||||
Keep: species, orders, houses, orgs/companies (sardaukar, jedi, cyberdyne), places (kessel; `falconer of the caladan eyries` is fine — the person problem was only `duke caladan`), institutions (zekma, karass).
|
||||
Replace each eviction with the generic office beside it, class-stable, and unique against the pool.
|
||||
|
||||
## Invariants tests pin (breaking any of these goes red)
|
||||
- 1000 names, all distinct.
|
||||
- Sentinel literals must survive: `hangman of the holiday` (man), `high priestess of the ear moon` (woman), `comrade operator` (other).
|
||||
- Pool firsts feed pool[0] probe tests — union order is MAN-first / WOMAN-first / OTHER; keep source order within classes so firsts stay: `chairman of the presidium of processors` / `hangwoman of the public convenience` / `comrade operator`.
|
||||
- Class counts (86/15/899 as of 2026-10) may churn but must be reported; tests never pin them.
|
||||
|
||||
## Regeneration
|
||||
Pool lives as three generated arrays between markers `/** Male-coded` and `/** The unions` in `src/framing.ts`. Splice new arrays between those markers (python splice of a generated `const NAME = [ "x", ... ] as const;` block is fine; do not hand-edit 1000 lines). Then `bun run build` (dist pin test byte-compares), `bun run typecheck`, and the full `bats tests/` — the class sweeps are exhaustive `Math.random`-cycled walks (each ~200 s; suite ~17 min; never two bats processes at once).
|
||||
|
||||
## Fleet cutover after merge
|
||||
main checkout ff-only → `omp plugin upgrade omp-dankvoice` → `cmp` `~/.omp/plugins/node_modules/omp-dankvoice/dist/index.js` against merged `dist/index.js`.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: dankvoice-menu-knob-workflow
|
||||
description: "Add a persisted /say knob to dankvoice and test its menu row through the bats driver (DV_SELECT picks, INPUT stub, dist-pin rebuild gate)."
|
||||
---
|
||||
|
||||
Adding a setting to `dankvoice` (`~/Projects/dankvoice`, omp speech extension) means five touchpoints plus one behavioral test. Skip one and TypeScript or a silent drop catches it; skip the test and the row ships dead.
|
||||
|
||||
## The five touchpoints (order matters)
|
||||
|
||||
1. `src/settings.ts` — field on `Settings` (optional) **and** a type-guarded line in `readSettings`. A field without a guard line is written to disk and never read back; tsc stays clean, so nothing else catches it.
|
||||
2. `src/config.ts` — `Config` field (required, defaulted), resolution line, entry in the returned object, and the knob documented in the file header comment (that header is the README's pointer for "full knob list").
|
||||
- Convention: explicit env wins over settings (`env.X === "1" ? true : env.X === "0" ? false : (settings.x ?? false)`); booleans use `1`/`0`, enumerations use the literal value.
|
||||
3. `src/index.ts` — `persistSettings()` key list (a knob missing there never survives the session) **and** the getter in the `new Speaker({...})` deps object.
|
||||
4. `src/utterance.ts` — getter on `SpeakerSettings` if the Speaker reads it at speak time.
|
||||
5. `src/menu.ts` — the row. Rows carry their own `run`; never dispatch on `choice === options[N]`.
|
||||
|
||||
Gate at the call site, not the prompt, when a mode must not apply to operator-typed text: `Speaker.speak(ctx, text, thinking, auto)` — `auto=true` only from `message_end`/`agent_end`; `/say <text>` passes nothing. Prompt-only gating leaks.
|
||||
|
||||
## Test the row behaviorally, never as a config assertion
|
||||
|
||||
`tests/behavior.bats` drives the built `dist/index.js` against a mock harness. Selects and typed text both come from one pipe-separated queue: `DV_SELECT="row label|sub choice|typed text"`, consumed in call order; a consumed-exhausted `select` returns `options[0]`, so always name the row by its **rendered label** (`"Verbosity: normal (change)"`, `"Intro: none (change)"`).
|
||||
|
||||
- `ui.input(title, placeholder)` in the driver mock logs `INPUT <title> <placeholder>` and takes the next queue entry. If the mock lacks `input`, no typed value was ever testable — add the stub before writing the test.
|
||||
- Scenario pattern that proves a pick works (`case "knobs"`): `say.handler("menu")` → `say.handler("on")` → two `agentEnd(answer(...))` with `settle()` between. Then assert on observable output, not on the settings file:
|
||||
- prompt-shaping knob: `[[ "$output" == *"ONE spoken sentence"* ]]` (directive reached the rewrite request)
|
||||
- framing: `[[ "$output" == *"input=Update, chief!"* ]]`
|
||||
- throttle/framing counting: `[[ "$output" == *"CALLS 1"* ]]`
|
||||
- A guard test whose negative case would pass trivially must also assert its premise, e.g. truncation: assert `CALLS 1` so "no outro" cannot be a green caused by nothing ever being cut.
|
||||
|
||||
Red first, in a batch: write all the new `@test` blocks, run `bats tests/behavior.bats -f "<pattern>"`, confirm the new ones fail, then implement.
|
||||
|
||||
## Gates and gotchas
|
||||
|
||||
- **Two `bats` runs at once corrupt each other**: `setup()` runs `bun run build`, so a concurrent run rebuilds `dist/` mid-suite and the byte-identical pin test fails spuriously. Serialize.
|
||||
- **Edit `src/` → rebuild before commit**: the pin test compares committed `dist/` to a fresh build. Sequence is `bun run build && bunx tsc --noEmit && bats -f <pattern>`, then `git add -A` (dist included). A failure there is normally a stale bundle, not a bad assertion.
|
||||
- `VAR=value some_function args` in bats does reach the child process, so knob tests need no `export`.
|
||||
- `edit` tool: inserting after a line whose content you know but that the current tag never displayed gets refused — the refusal echoes the real line; if it matches intent, re-issue the identical edit instead of re-reading.
|
||||
- Ship: branch `feat/<issue#>-<slug>` off fresh `origin/main`, one commit per knob, PR targets `main`, merge only on Frank's explicit OK (`do=merge`, `merge_style=no-ff`), then `omp plugin upgrade dankvoice` and `cmp` installed `dist/index.js` against the merged one.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: dankvoice-silence-triage
|
||||
description: "Triage silent or late dankvoice auto-speak on the fleet: arming key, /say grammar, endpoint/sink, event granularity, humanizer latency, plus the black-box bats driver."
|
||||
---
|
||||
|
||||
# Triage "dankvoice is silent" on the fleet
|
||||
|
||||
Symptom that recurs: auto-speak only speaks when `/say <text>` is typed
|
||||
explicitly. Work the gates in this order; each has an objective check.
|
||||
|
||||
1. **Armed?** `cat ~/.omp/cache/dankvoice.json | jq .auto` — must be `true`
|
||||
(or `DANKVOICE_AUTO=1`). Arming is read at **startup only** and `/say on`
|
||||
never persists, so a session started before the key was added is silent even
|
||||
though fresh sessions work. `/say start` is *not* a subcommand: unknown
|
||||
arguments are spoken verbatim (it read out the word "start"). `/say on` is
|
||||
the verb; `/say status` must print `auto-speak on`.
|
||||
2. **Endpoint + sink?** `/say status` must show a real endpoint (not
|
||||
`NO ENDPOINT`) and the saved sink must exist:
|
||||
`pw-dump | grep -o 'alsa_output\.[^"]*' | sort -u`.
|
||||
3. **Did it synthesize?** `pgrep -a pw-play` during a turn: one process per
|
||||
segment, `--target <sink>`, wav under `/tmp/dankvoice-*/say-<epochms>.wav`.
|
||||
One temp **directory per utterance chain** — count dirs to count utterances,
|
||||
pids to count segments.
|
||||
4. **Event granularity (the structural cause).** `agent_end` fires once per
|
||||
*prompt* when the loop settles and is skipped when the harness maps a
|
||||
non-terminal settle (`willContinue`) — long runs with background jobs or
|
||||
subagents therefore narrate nothing. Mid-run narration needs `turn_end`
|
||||
(fires per assistant message + tool results). Verify delivery, don't infer
|
||||
it: skill `omp-extension-event-probe`.
|
||||
5. **Latency is not silence.** Narration passes through the humanizer
|
||||
(frank-zealot) before synthesis, so an utterance lands several seconds after
|
||||
its turn; under endpoint load that stretches to tens of seconds.
|
||||
`/say humanize off` = mechanical voice, near-instant. Judge cadence from
|
||||
probe-log timestamps, never from when audio starts.
|
||||
|
||||
## Design rules the plugin now encodes (dank/dankvoice#2→#3)
|
||||
|
||||
- Narrate `turn_end` per turn **plus** `agent_end` for the settled answer;
|
||||
dedupe identical text per run (reset on `agent_start`) so the answer speaks once.
|
||||
- A turn landing while the player is alive is **dropped** — no queue, no
|
||||
interrupting; a tool loop would cut every sentence off after a word.
|
||||
Busy test: `state.child !== undefined && state.child.exitCode === null`.
|
||||
- Tool-only turns (no prose, no narration-flagged thinking) stay silent.
|
||||
- `print`/`json` modes stay silent, so benches never chatter.
|
||||
- Settings writes must **round-trip the file** (`{...onDisk, ...owned keys}`).
|
||||
Rebuilding from owned keys deleted `"auto"` on any menu choice and silently
|
||||
un-armed every later session — the classic cause of "it worked yesterday".
|
||||
|
||||
## Black-box test harness (repo `dank/dankvoice`)
|
||||
|
||||
`tests/behavior.bats` drives the built artifact through an in-process driver:
|
||||
`const say = DV_EXT(ctx, cfg)[0]`, handlers via `const events = new Map();
|
||||
const pi = { on: (n,h)=>events.set(n,h) }`, then `events.get("turn_end")?.({
|
||||
turnIndex:0, message:{role:"assistant",content:[{type:"text",text:"…"}]},
|
||||
toolResults:[]}, ctx)`. Assert on `CALL`/`PLAYS`/`NOTIFY` lines and
|
||||
`SETTINGS <file contents>`; stubs are `DV_PLAYER_CMD` (logs voice+input, honours
|
||||
`DV_HOLD`) and `DV_REWRITE_URL` (404 → falls back to mechanical text). `setup()`
|
||||
runs `bun run build` per test → serialize suites on the shared `dist/`.
|
||||
`tests/packaging.bats` requires `dist/` **committed** (compares to HEAD), so
|
||||
build → commit → the drift pin goes green.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: deployment-checkout-trunk-sync
|
||||
description: Use when the live deployment checkout (~/.omp main) must merge a moved origin/main while dirty tracked files block the merge and workspace policy blocks git checkout.
|
||||
---
|
||||
|
||||
---
|
||||
name: deployment-checkout-trunk-sync
|
||||
description: Use when the live deployment checkout (~/.omp main) must merge a moved origin/main while dirty tracked files block the merge and workspace policy blocks git checkout.
|
||||
---
|
||||
|
||||
# Sync a dirty live checkout to a deployment (checkout blocked)
|
||||
|
||||
Situation: the checkout IS the deployment; months of machine/hand writes sit
|
||||
uncommitted; `origin/main` has moved; `git checkout`/discard is BLOCKED by
|
||||
workspace policy ("edit forward"; exemption only during active conflict).
|
||||
|
||||
## Procedure
|
||||
1. `git fetch`; trial `git merge --no-commit --no-ff origin/main` to enumerate
|
||||
exact blockers (dirty files the merge would overwrite + untracked collisions).
|
||||
Untracked blockers: `cp -a` to `.AGENTTRASH/pre-merge-<date>/`, then `mv` aside.
|
||||
2. **Preservation commit:** `git add` ONLY the blocking paths (not the whole
|
||||
tree) and commit them as-is: `chore: adopt pre-existing working-tree state
|
||||
before trunk merge`. Nothing is discarded; the secret gate still runs.
|
||||
Include EVERY dirty deployment-state path (dotfiles/live tiers too) — a
|
||||
dirty file that is neither committed nor trashed is unprotected from step 3.
|
||||
3. `git merge --no-ff origin/main`. Conflicts are now resolvable — the policy
|
||||
exemption is active during a merge.
|
||||
4. Resolution law:
|
||||
- **live machine config** (`agent/config.yml`, `models.yml`, dotfiles live
|
||||
state like noctalia `settings.toml`) → `--ours`: the running box is its
|
||||
own authority; the remote copy is a snapshot.
|
||||
- **prose/law/docs** (`AGENTS.md`, KB) → `--theirs` (upstream canonical;
|
||||
local variants survive in the preservation commit).
|
||||
- **append-mostly logs** → union: cut only the three marker lines
|
||||
(`<<<<<<<`/`=======` range + `>>>>>>>`), keep both sides' entries.
|
||||
5. `git commit --no-edit`, push trunk, verify `main == origin/main`.
|
||||
6. Pre-merge byte-compare every dirty file vs `origin/main`
|
||||
(`git show origin/main:$f | diff -q - $f`): SUPERSEDED (identical → discardable),
|
||||
DIVERGED (resolve), untracked-in-upstream (collision).
|
||||
|
||||
## Gotchas
|
||||
- **NEVER `git stash` to unblock a deployment sync, NEVER `git stash clear`/
|
||||
`drop` afterward.** Observed 2026-10-18: a sync stashed the dirty tree, the
|
||||
stash entry was later destroyed, and months of GUI-built bar layout
|
||||
(noctalia settings.toml, never committed) vanished with it — recovered only
|
||||
via dangling-object archaeology (below). Step 2's preservation commit exists
|
||||
precisely so stashing is never needed.
|
||||
- **Stash-and-retry loops are a red flag**: three `WIP on main` stash commits
|
||||
within ~60s of each other means the operator was fighting the dirty tree,
|
||||
not resolving it.
|
||||
- **Recovery from destroyed stashes** (objects survive entry deletion until gc):
|
||||
`git fsck --lost-found` → `git log -1 --format='%ci %s' <dangling>` to date
|
||||
them → `git show <c>:<path>` to dump candidates; newest `WIP on` commit is
|
||||
usually the last good state. Validate before deploying (e.g.
|
||||
`noctalia config validate`), back up the clobbered file to `.AGENTTRASH/`,
|
||||
then restore through the symlink target.
|
||||
- Never wrap `git checkout` in `2>/dev/null` here: the policy-block error was
|
||||
swallowed and the loop silently no-op'd; the merge then failed identically.
|
||||
- `git add` a directory pathspec stages its untracked children (fine for
|
||||
allowlist dirs; never for prose dirs).
|
||||
- The live-sync heartbeat (#11) commits machine tiers hourly, so this sync is
|
||||
a rare operation post-deployment — mostly memories/dotfiles stragglers.
|
||||
Remember the heartbeat can lag up to an hour behind GUI edits: the tree
|
||||
between heartbeats is the ONLY copy of fresh GUI work.
|
||||
- **Frank's accepted risk posture (2026-10-18):** hourly sweep + the
|
||||
preservation-commit law + fsck recovery is defense-in-depth ENOUGH. Do not
|
||||
re-propose faster heartbeats or path/event triggers to shrink the
|
||||
GUI-edit-to-commit window; he chose status quo after measuring (heartbeat
|
||||
itself is 57 ms CPU — cost was never the objection; un-debounced
|
||||
DirectoryNotEmpty triggering was, and path-unit proposals are vetoed).
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: docker-pin-latest-census
|
||||
description: "Prove a Docker Hub / lscr.io image pin is the latest version before bumping it: registry-API tag enumeration, the rolling-latest trap, digest pinning, and rollback proof."
|
||||
---
|
||||
|
||||
Use when asked whether a Docker Hub / lshr.io image pin is the latest version before bumping it, or when a manifest claims an image is stale. Registry API is the answer; the `latest` tag is not.
|
||||
|
||||
## The API
|
||||
|
||||
```bash
|
||||
# hub: token then paginate (the list is 100 per page)
|
||||
tok=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" | jq -r .token)
|
||||
for off in 0 100 200 300 400 500; do
|
||||
curl -s -H "Authorization: Bearer $tok" \
|
||||
"https://hub.docker.com/v2/repositories/${repo}/tags?page_size=100&ordering=last_updated&page=${page}" \
|
||||
| jq -r '.results[] | "\(.name)\t\(.last_updated)"'
|
||||
done | sort
|
||||
```
|
||||
|
||||
```bash
|
||||
# lscr.io (linuxserver): tags are semver, `latest` is a rolling alias
|
||||
curl -s https://images-api.github.com/v2/linuxserver/${repo}/tags/list # wrong host for lscr
|
||||
curl -s "https://hub.docker.com/v2/repositories/linuxserver/${repo}/tags?page_size=100" | jq -r '.results[].name'
|
||||
```
|
||||
|
||||
Docker Hub's own tag list (`hub.docker.com/v2/repositories/<repo>/tags`, ordered by `last_updated`) is the practical source. The registry API (`/v2/<repo>/tags/list`) needs the bearer token flow above; both return tags, neither sorts by semver.
|
||||
|
||||
## The `latest` trap
|
||||
|
||||
- `latest` is a **rolling alias**, not a version. "Pin is older than `latest`" says nothing about whether a *versioned* tag exists.
|
||||
- `latest` can lag or be rebuilt without a version bump; comparing a digest pin against it proves nothing.
|
||||
- For linuxserver images: `stable` is the rolling branch and version tags (`2026-09-30`) are what you pin; multi-arch manifest lists mean the same tag has per-arch digests — the pin must be the manifest-list digest (`RepoDigests[0]`) you actually pull, or a re-pull on another arch silently changes the image.
|
||||
|
||||
## Verification recipe (before bumping a pin)
|
||||
|
||||
1. Read the pinned tag and digest from the manifest/quadlet.
|
||||
2. Enumerate tags from the registry API (paginate; `last_updated` is not semver order).
|
||||
3. Compare **versions**, not recency: does a version greater than the pinned one exist?
|
||||
4. If bumping: pull the new tag and record `docker/podman inspect --format '{{index .RepoDigests 0}}'` as the new digest — pin the digest, not the tag.
|
||||
5. Prove the service: it starts, version endpoint answers the new version, no crash-loop, and the rollback is the previous digest.
|
||||
|
||||
## Report
|
||||
|
||||
"pinned: X (digest …), latest available: Y, upstream changelog: …" — or "pin is current: no version tag newer than X exists". Never claim staleness from the `latest` tag alone.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: docs-dedupe-verify-campaign
|
||||
description: "Run an exhaustive docs dedupe + fact-verification campaign (command-center or any OKF vault, e.g. franksign): scout cluster ledgers, override scouts with evidence, gate every commit, and sweep scaffolding before PR."
|
||||
---
|
||||
|
||||
# Exhaustive docs dedupe + verification campaign (dank/command-center; transplants to any OKF vault — proven on franksign 2026-10-21, #64 2026-10-21)
|
||||
|
||||
## Frame
|
||||
- Read `omp-docs/references/okf-spec.md` (or the vault's vendored OKF contract, e.g. `franksign-docs/read-only/okf-v0.2-spec.md`) + root `index.md` first; inventory with `find <vault> -name '*.md' | wc -l`.
|
||||
- Run `scripts/kb-lint.sh omp-docs` baseline; index-coverage gate is `scripts/kb-coverage.py omp-docs` (each dir's `index.md` must link every sibling file/subdir). On franksign the whole mechanical gate is one seam: `bun tests/lib/okf-lint.mjs` (frontmatter, links, index coverage, version-once) — run it before and after.
|
||||
- Branch off fresh `origin/main` in a NEW worktree (`git -C ~/.omp worktree add ~/.omp/wt/<name> -b docs/<issue>-slug origin/main`) — workspace policy blocks `git checkout -b` in session worktrees. In a session worktree, `git checkout -b` is intercepted anyway: commit on the session branch, then named-ref push `HEAD:refs/heads/docs/<n>-slug` (verify `git log origin/main..HEAD` shows ONLY your commit first).
|
||||
- Open the Gitea issue first (multi-moving-parts law); branch name `docs/<issue#>-<slug>`.
|
||||
|
||||
## Scout fleet (overlap detection)
|
||||
- One `task` call, `tasks[]` batch, agent `scout`, read-only; one cluster per task. **Fan out 2-3 clusters at a time, never seven**: `task.maxConcurrency` is 2 on this fleet, so the excess only queues, and `proc://` reports queued tasks as `running` — identical uptimes with zero deliveries means thrash, not progress (#64: seven scouts, nothing in 17 min; killing the five redundant ones let the two survivors land within minutes).
|
||||
- Do NOT scout a cluster you have already audited inline. Kill the redundant jobs early with `write proc://<id>/kill` — the endpoint is the scarce resource, and your own session shares it.
|
||||
- Ledger format per cluster: inventory rows (file | verdict KEEP|MERGE|DELETE|VENDOR|UPDATE | evidence line/command). **Demand the ledger as the yield TEXT, never a `local://` file reference**: #64 produced one 0-byte ledger plus three files whose names contained the scout's own serialized parameter block (`gitea-caddy-inventory.md` + newline + a parameter tag), recoverable only by `ls`-ing `~/.omp/agent/sessions/<session>/local/<dir>/`.
|
||||
- **Scouts lie.** Verified misses in the 2026-10-19 campaign: fabricated ledger entries ("peer-reviewed BrowseComp" claim that exists in no file), a DELETE verdict for a generated artifact that is checked in by design AND under test (`tools/niri/keybindings.md`, see `scripts/gen-keybinds.sh` + `tests/gen-keybinds.bats`), and a KEEP for a leftover the scout misread its own changelog about. Franksign 2026-10-21: 2 of 3 cluster scouts fabricated — "7 missing component stories" refuted by the pairing-law seam that runs green on every push, invented entry/page counts, DELETE verdicts on live-box census numbers the purpose test KEEPS; a third hit the 30-min runtime cap producing zero output. #64 same day: three pages "missing an H1" that have one on their first heading line; a 215-line `omp-docs/AGENTS.md` that does not exist; a "wikilinks + 700-char wall" defect in `core/settings.md` that a commit from that same morning had already removed; `core/subagent-limits.md` called superseded while `agent/config.yml` carries the cited `maxRuntimeMs` / `softRequestBudget` / `maxRecursionDepth` values verbatim.
|
||||
- **Four pre-checks before a scout row enters the diff**: (1) the file exists; (2) the named heading/marker is actually absent (`grep -c '^# '`, `grep -c '^# Citations'`); (3) `git log --oneline -3 -- <file>` — a same-day commit may already have fixed it; (4) if the claim is "config supersedes the doc", grep the config for the cited keys.
|
||||
- Cluster sizing: for a vault under ~20 pages, scout fleets are net-negative — audit inline yourself (reading 5 pages beats babysitting a scout). Reserve scouts for multi-hundred-page trees.
|
||||
|
||||
## Mechanical sweeps that pay (run on every vault)
|
||||
- Empty-text links: `grep -rnE '(^|[ ,])\]\(' <vault>` — a generated "See Also" pass left five `- ](index.md)` links (#64); rendered, an empty bullet reads as style, never as a broken reference.
|
||||
- Citation convention census BEFORE adding anything: count `grep -rl '^resource:'` against `grep -rl '^# Citations'`. command-center's convention is `resource:` frontmatter (12 vs 3) — "add Citations to six pages" is a convention change, not a fix; only the pages missing `resource:` were real hits.
|
||||
- Long-line walls: `awk 'length>300 {print FILENAME":"NR"("length")"}'` — reflow prose to ~100 columns, wording intact; leave tables (they cannot wrap) and the text of verbatim blockquotes alone.
|
||||
- Dated log headings: three `## YYYY-MM-DD` sections for one day is three logs wearing one date. Collapse to one `##` plus one `###` per campaign, and write the shape into the spec page so nobody has to infer it.
|
||||
|
||||
## Fact verification
|
||||
- Fleet facts: read-only probes over `ssh -o BatchMode=yes dankclaw|dankserver` (podman ps/inspect/top, stat, /etc/os-release, curl localhost APIs, `podman exec <c> <tool> --version`). Live state beats docs even when docs claim recent verification — the 2026-10-18 "audit" still drifted (zealot 27B→9B, build 10754→10777, container census 14→32; #64 then found `8C/16T` where SMT is off, a "masked" unit that is simply absent, Pi-hole missing from a DNS list, and a weekly-drifting engine count that became a size class with `/config` named as the only authority).
|
||||
- Local box facts too: `pacman -Qs <pkg>` / `command -v` before trusting a page's "installed tool" claim (#64: `wlr-randr` documented as available, never installed).
|
||||
- Unit health is not endpoint health: an `:7770` that answers `/v1/models` while its systemd unit sits inactive `Result=exit-code` means a container escaped its unit. Report the drift to the operator; never record transient state as docs, and never touch a production unit without a per-run OK.
|
||||
- Code-bound facts (franksign-style): every doc claim about routes/columns/shipped status gets grepped against `apps/*/src`, `packages/db/migrations/` — docs narrating features that never shipped ("page ships in the slice" while only the API landed) get truthed with a status pointer to the owning issue.
|
||||
- External claims: web_search/web read yourself (no subagents per Frank's rule). Kill fabrications outright (invented distro `FreeToken`, nonexistent `ornith-llama-cpp-setup` tree, unattributable "44.5% peer-reviewed CUA") — delete/drop with an explicit note, never keep "plausible" numbers.
|
||||
- Policy drift (live ≠ tuned docs): flag in the doc + PR body as an operator decision; NEVER silently re-apply policy.
|
||||
- Deletion proposals from scouts die by default: archived trees carrying an honest archive banner (`pi/**`, banner 2026-10-18) stay, a directory `index.md` cannot be deleted (index-coverage gate), and incident logs are laws, not bookkeeping. Deletion is a pruning session with Frank present.
|
||||
|
||||
## OKF §9 / frontmatter audit (cheap wins, check every vault)
|
||||
- Log entries filed under the wrong ISO-date heading (crossed midnight): cut the entry (`CUT N.=M @reg`), insert the new date heading above the old one (newest-first headings), paste the entry under it.
|
||||
- Missing trust family (`generated` on machine-written pages) — lint may not catch it; grep every frontmatter block.
|
||||
- Laws living only in `log.md`: promote each to its owner page (mechanism law → architecture page owning the thing), leave the log entry as the incident record; the HANDOFF/README boundary then points at the vault instead of restating.
|
||||
- A law restated in two tiers (vault spec page + always-loaded `AGENTS.md`) is not a duplicate to cut: name the co-ownership in the spec page ("change one, change the other") instead of silently dropping one copy.
|
||||
|
||||
## Log compaction (log.md)
|
||||
- Entry law: dated entry = what changed + durable lesson + concept-page pointer; narratives recoverable via `git log --follow`. Record the law at the top of the log. Tag historical-era entries.
|
||||
- One `## YYYY-MM-DD` per day, `###` per campaign. Put the fan-out and ledger-verification lesson in the entry — that is what stops the next campaign from trusting a scout.
|
||||
|
||||
## Gates + landing
|
||||
- Gate every commit: `bash scripts/kb-lint.sh omp-docs && python3 scripts/kb-coverage.py omp-docs && bats tests/`. Pre-commit hook prints `error: command not found: caddy` on Caddyfile-touching commits — warning only, commit still lands. On franksign: kill the :3000 demo stack first (a live API hangs the gate ~10 min; law lives in `franksign-docs/architecture/testing.md`), then `bun tests/lib/okf-lint.mjs && tests/run.sh docs`.
|
||||
- Before opening the PR: `git diff --stat origin/main..HEAD | grep -i 'tmp\|scratch\|\.bak'` — scaffolding merged under a `.tmp-` name in the 2026-10-19 campaign and needed a follow-up PR to promote.
|
||||
- PR via API (skills `gitea-api-pr-operations`, `gitea-merge-form-encoded`): token through a 600-perm `-K` curl config, never `-H` (`/proc/<pid>/cmdline` is world-readable); create/PATCH = JSON WITH Content-Type, merge = form body with NO Content-Type, `do=merge&merge_style=no-ff`, success = HTTP 200 empty body verified by landed state, not by re-reading the PR. `shred -u` the cfg and payload afterwards. `xdg-open` the PR the moment it opens; comment the campaign ledger on the issue — ownership-move table, re-read facts with the command that read them, and rejected findings with their evidence.
|
||||
- After Frank's merge OK: propagate the deployment with `git -C ~/.omp fetch && git merge --ff-only origin/main`, but first prove the FF cannot touch live machine-tier state: `comm -12 <(git -C ~/.omp status --porcelain | grep -v '^??' | awk '{print $2}' | sort) <(git diff --name-only HEAD..origin/main | sort)` must print nothing, and re-check the dirty count after the merge.
|
||||
- Worktree reaping is the live-sync heartbeat's job (`wt/*` older than 7 days) unless Frank says otherwise — no `rm`, no deleting a session branch you did not create.
|
||||
|
||||
## Line-edit op syntax trap (bit five times)
|
||||
- Valid ops: `PUT N.=M:` (body `+text` lines), `CUT N.=M` / `CUT N.=M @reg`, `PUT <N @reg`, `PUT >N:` / `PUT <N:`. There is NO `PUT N+=:` form — it lands in the file as a literal garbage line and shifts the insert to the wrong position. For gap-insert use a lone `PUT >N:` or `PUT <N:` per call; pre-compute line numbers against the file map (ops reference original lines within one call, but a failed op poisons positions).
|
||||
- **PUT with an empty body DELETES the range** (franksign 2026-10-21: a typo-fix `PUT 7-7:` with empty body deleted the whole entry). Bodyless deletes: use `CUT N.=M`.
|
||||
- A `CUT`/`PUT` range outside the file errors cleanly (`out of range (file has N lines)`) — that is the cheapest way to learn a body was shorter than assumed; re-read the tail rather than guessing offsets.
|
||||
- Bare body rows get auto-prefixed with `+` and a warning — harmless, but never rely on it: prefix every body row yourself.
|
||||
- Editing a PUT body that ends adjacent to the next original line can silently consume the closing frontmatter `---` — after any frontmatter-range PUT, re-read lines 1–10 of the result.
|
||||
- Recovery recipe when a literal op line lands: read exact bounds, `CUT N.=N` (junk), `CUT A.=B @block` (misplaced section), then `PUT <M @block` at the true anchor, re-read tail, add spacing blank if a `---` now abuts a list line.
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
name: docs-point-of-use-destruction
|
||||
description: "Documentation exists for point of use, not archival. Use before any dedupe/compact/verify-docs job, and whenever a census or \"only copy\" argument is being used to KEEP documentation: apply the purpose test and destroy obsolete measurements. Supersedes the preservation conclusions of vault-one-fact-one-owner-dedupe §3/§12."
|
||||
---
|
||||
|
||||
# Documentation exists for point of use
|
||||
|
||||
**Supersedes** `vault-one-fact-one-owner-dedupe` §3 ("port before cut") and §12
|
||||
("expect docs flat") wherever those conclude a page must be *kept*. Sole custody of a
|
||||
useless fact is still useless.
|
||||
|
||||
## The test, before you write and again before you keep
|
||||
|
||||
- Would the reader, doing a real task tomorrow, **act differently** because of this
|
||||
line? No → delete it.
|
||||
- Is it a measured rate (t/s, tok/s, ms, MiB-of-pool, acceptance %, a bench ladder) for
|
||||
a model / runtime / layout that is **not running now and not being compared now**?
|
||||
Destroy it. Not "file it in an Eras row", not "preserve it for history". AI moves
|
||||
faster than the vault: the number is wrong before anyone reads it, and a wrong number
|
||||
in a runbook gets believed.
|
||||
- Keep the **law** the measurement taught; drop the measurement. Qualitative causal
|
||||
stories survive; numbers do not.
|
||||
- What survives: live config and its exact numbers, procedures, failure signatures,
|
||||
security posture, interface contracts, rollback pointers.
|
||||
- **Losing information is an acceptable, usually correct outcome.** Frank said so
|
||||
explicitly on 2026-10-19 after rejecting a PR that had preserved 53 ledgers.
|
||||
|
||||
## Traps that lead agents to preserve junk
|
||||
|
||||
- **A census cannot decide what to keep.** `port-census.py`-style tools answer "where
|
||||
does this fact live", never "does anyone need it". "14–63 sole-record lines per page"
|
||||
was read as *irreplaceable* when the fact was *obsolete*. Ask usefulness, not custody.
|
||||
- **A census that includes the file under test proves the page is held by itself.**
|
||||
Exclude it, print the corpus size, assert absence; treat `0 unheld` on a long page as
|
||||
a tool bug until proven otherwise.
|
||||
- **"One fact, one owner" is a placement rule, not a preservation rule.** It says where
|
||||
a surviving fact lives; it never says the fact must survive.
|
||||
- **A change diary (`log.md`) is the largest archive in most vaults** (this one: 2,380
|
||||
lines, ~8 % of the vault). Git holds the chronology; a causal law belongs on the page
|
||||
that owns the thing it governs. Retire the diary.
|
||||
|
||||
## Execution that worked (2026-10-19, −13.7k lines)
|
||||
|
||||
1. Write the kill list as a reviewed artifact, one path per line, grouped by *why it is
|
||||
dead* — that list becomes the PR's argument and the trash manifest.
|
||||
2. Destroy whole pages first (`scripts/destroy-vault.py <list>`): move to
|
||||
`.AGENTTRASH/<date>-destruction/`, then delete every reference from survivors, whole
|
||||
line included when the dead link was its only content. Do not compare link targets
|
||||
with `pathlib.Path.stem` — it strips real suffixes (`sd.cpp`).
|
||||
3. Then strip numbers from what remains (`scripts/axe-measurements.py`): delete ledger
|
||||
sections (noun-LEDS the heading, nothing instructional after it), blank any table cell
|
||||
reporting a rate, fold Eras older than the current one into a single row pointing at
|
||||
git. Never touch a shape number or a Laws/Wire/Rollback section.
|
||||
4. Sweep indices (generator must exclude the trash dir, drop rows whose page cell became
|
||||
empty), re-lint, re-check links.
|
||||
5. Rewrite the governing policy in the same PR, or the next agent re-grows the vault:
|
||||
project `AGENTS.md` *Documentation Policy* + the global agent settings.
|
||||
|
||||
## Report honestly, per area
|
||||
|
||||
docs / scripts / operator-doc separately, measured against the merge-base. The tools are
|
||||
a cost that must be named, not hidden inside a headline "net" number — that is what got
|
||||
the previous pass rejected.
|
||||
|
||||
## Verify the repair tools themselves
|
||||
|
||||
One repair pass matched "line ends with a comma" and deleted the trailing comma from 660
|
||||
wrapped prose lines. Restore by matching lines that are byte-identical to `HEAD` except
|
||||
for trailing punctuation (`restore-punctuation.py`), keep the destructive rule out of the
|
||||
tool, and record the incident in its docstring.
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
name: docs-refresh-scout-summaries
|
||||
description: "Refresh a stale documentation cluster without delegating the writing: scout subagents produce per-line verdict tables (holder verified by comma-insensitive grep), the parent makes every edit, and a port-before-cut census proves nothing was lost. Use for dedupe-our-docs, fix-the-stale-runbooks, pointers-instead-of-copied-numbers jobs."
|
||||
---
|
||||
|
||||
# Refresh stale docs with scout summaries, parent-side writes
|
||||
|
||||
Use when a cluster of documentation pages is stale and/or restates facts another page owns, and
|
||||
the fix touches many files. **Scouts read and classify; the parent writes.** Sibling skill
|
||||
`vault-one-fact-one-owner-dedupe` covers collapsing era pages into owner pages - its section 3
|
||||
(orphan census), section 7 (edit-tool hazard) and section 9 (verify/PR) apply unchanged.
|
||||
|
||||
## Why not writer subagents (measured cost)
|
||||
|
||||
Writer agents produce plausible output and cannot be trusted on the one claim that matters - the
|
||||
owner page already holds this fact. In one run they asserted every cut was grep-confirmed while
|
||||
FOUR cuts left pointers to pages that did not contain the fact (118->238 GiB, 16,095 tokens,
|
||||
0.739/0.554/0.427/0.340, PR #23897). Each became a broken promise the next reader trusts.
|
||||
Classification is the only part that parallelizes; writing is where taste (ownership, era
|
||||
naming, what counts as a law) lives. So fan out read-only scouts and keep the pen.
|
||||
|
||||
If writer agents were already spawned and are no longer wanted: `write agent://all` to stop
|
||||
them, then `git status` the target tree - `proc://` does not list agent jobs, and a finished
|
||||
worker may already have edited files.
|
||||
|
||||
## Pipeline
|
||||
|
||||
1. **Cluster** targets into groups of 3-4 pages sharing subject matter (one cluster per scout).
|
||||
Know the owner map first (endpoint/service/host pages, plus the repo documentation law).
|
||||
2. **Wave of 2 scouts** (harness cap is `task.maxConcurrency`, usually 2). Scout contract:
|
||||
|
||||
> READ-ONLY, no writes of any kind. Per file, one table row per candidate fact:
|
||||
> `| lines | verbatim figure + 6 words | holder elsewhere (file:line) or none | VERDICT |`
|
||||
> Verdicts: `CUT -> [[Owner]]` (holder verified), `KEEP(owned here)` (no holder, this page is
|
||||
> the record), `PORT(parent)` (rightful owner lacks it - name page and section), `STALE`
|
||||
> (contradicted by current state; quote the contradicting line), `INDEX` (locator page).
|
||||
> Verify every holder claim with a COMMA-INSENSITIVE grep: a holder writing 1,350,182
|
||||
> satisfies a page writing 1350182 and vice versa; 16,095 does NOT satisfy a check for 16095
|
||||
> unless commas are stripped. Never name a holder line you have not read.
|
||||
> End with a per-file tally: kept / cut / port / stale.
|
||||
|
||||
Put the law, the owner map and the verification rule in the batch-level `context`, never per
|
||||
task.
|
||||
3. **Write while the next wave runs**: edit cluster N from wave N, launch wave N+1 in the same
|
||||
turn. Never launch a wave you have nothing to write.
|
||||
4. **Verify each CUT yourself** before cutting - read the holder line. A scout row is a
|
||||
hypothesis, not evidence.
|
||||
5. **PORT items are your edits**: add the fact to the owner page section (Eras row / Laws / spec
|
||||
table), then cut it from the technique page in the same commit.
|
||||
6. **STALE items: check live state**, not another document - `ssh <host> systemctl is-active
|
||||
<svc>`, the endpoint `/health`, the env file. Then rewrite with a date, or drop transient
|
||||
live state an owner page already records.
|
||||
|
||||
## The ownership test that stops wasted ports
|
||||
|
||||
Two pages carrying one figure is the defect - a figure with exactly one holder is fine wherever
|
||||
it lives. A technique page that is the sole record of a campaign owns that campaign's arm
|
||||
tables; do not port such ledgers into an endpoint Eras table. Write the clause into the repo
|
||||
documentation law so the next pass stops porting single-owner facts. Corollary: a worked example
|
||||
may repeat a live number if it declares itself dated and names the owner (a snapshot, not fleet
|
||||
state - recompute from the live files).
|
||||
|
||||
## Census after each cluster (catches your own slips too)
|
||||
|
||||
Run the sibling skill's section 3 census over `git diff -U0` for the cluster (deleted lines x
|
||||
distinctive numeric keys vs the whole vault **plus AGENTS.md**), comma-insensitively; drive the
|
||||
count to zero or list each accepted loss in the commit message.
|
||||
|
||||
```python
|
||||
flat = corpus.replace(',', '')
|
||||
pres = lambda n: n in corpus or n.replace(',', '') in flat
|
||||
toks = lambda l: [n for n in re.findall(r'\d[\d,]*(?:\.\d+)?', l)
|
||||
if len(n.replace(',', '').split('.')[0]) >= 4
|
||||
or (len(n.split('.')[-1]) >= 2 and len(n.replace(',', '')) >= 5)]
|
||||
orphan = [l for l in deleted if toks(l) and not any(pres(n) for n in toks(l))]
|
||||
```
|
||||
|
||||
## Verify and land
|
||||
|
||||
- Vault link seam must print `0 unresolved links`; hard-wrapped `[[Two Word Page]]` links split
|
||||
across lines are the usual cause - unwrap them.
|
||||
- Full suite once at the end, not per cluster; a docs seam running while scouts are live may
|
||||
read half-written files, so re-run after the last edit.
|
||||
- Commit per cluster (docs-only means **no deploy**); push, open the PR through the API with a
|
||||
600-perm curl config, `xdg-open` the PR URL immediately.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: dotfiles-adoption-audit
|
||||
description: "Use when deciding which userspace configs on a Dank box belong in the command-center dotfiles tier — separates tracked intent from generated expansion, app state, secrets, and config owned by another repo. Not for deploying one known file."
|
||||
---
|
||||
|
||||
# Dotfiles adoption audit
|
||||
|
||||
Decides, with evidence, which files under `~/.config` (and top-level dotfiles) belong in
|
||||
`dank/command-center`'s `dotfiles/` tier. The repo law being applied: **config in, state
|
||||
out**, and **one fact, one owner**.
|
||||
|
||||
## The four gates (all four must pass)
|
||||
|
||||
1. **Installed?** `command -v <prog>` — config for software absent from the box is a prune
|
||||
candidate, never an adoption target. This eliminates most of the clutter.
|
||||
2. **Authored, or the expansion of something else?** Generated files must stay out even
|
||||
though they look like config.
|
||||
3. **Does another repo or installer already write it?** If a script or package owns the
|
||||
file, tracking it here creates a second owner that loses every argument.
|
||||
4. **Could a secret ever live in it?** If yes: template + quarantine, or leave it out.
|
||||
|
||||
## Gate 2 is the one people get wrong — prove generation from mtime
|
||||
|
||||
A generator walking the app list writes dozens of files in the same sub-second window.
|
||||
Compare timestamps across unrelated programs:
|
||||
|
||||
```bash
|
||||
cd ~/.config && stat -c '%y %n' */themes/noctalia* */noctalia.* */colors/noctalia* 2>/dev/null | sort
|
||||
```
|
||||
|
||||
Identical seconds + microsecond progression across ten different programs = generated.
|
||||
Re-check one file days later: a new mtime with no human edit proves regeneration, so the
|
||||
tracked file must be the *source* (palette/config the generator reads), never its output.
|
||||
The correct pattern is a tracked config that *imports* the generated path.
|
||||
|
||||
## Classify the whole tree per file, not per directory
|
||||
|
||||
The tracked unit is the **file symlink**, so a directory can be "real" while containing only
|
||||
tracked symlinks. Never classify by directory:
|
||||
|
||||
```bash
|
||||
cd ~/.config && for d in */; do d=${d%/}; tr=0; rl=0; rlfiles=""
|
||||
while IFS= read -r f; do
|
||||
case "$(readlink -f "$f" 2>/dev/null)" in
|
||||
*/.omp/dotfiles/*) tr=$((tr+1));;
|
||||
*) rl=$((rl+1)); rlfiles="$rlfiles ${f#./}";;
|
||||
esac
|
||||
done < <(find "$d" -maxdepth 2 \( -type f -o -type l \) 2>/dev/null \
|
||||
| grep -vE 'Cache|cache|Cookies|storage|sessions|Crashpad|Default|thumbnails|extensions')
|
||||
[ $((tr+rl)) -eq 0 ] && continue
|
||||
printf '%-22s tracked=%-3s box-only=%-3s %s\n' "$d" "$tr" "$rl" "$(echo $rlfiles | cut -c1-90)"
|
||||
done
|
||||
```
|
||||
|
||||
Then read the headers of suspicious files — installers often confess ("Installed by
|
||||
`infra/setup-dev.sh` into `~/.config/containers/systemd/`") and unit files name another
|
||||
project's `WorkingDirectory`/`ExecStart`, which means that project owns them.
|
||||
|
||||
## Ownership before adoption
|
||||
|
||||
```bash
|
||||
git -C ~/.omp ls-files dotfiles # packages deploy.sh sees
|
||||
grep -rl "<unit-or-file-name>" ~/Projects/<proj> # does a project repo own it?
|
||||
systemctl --user is-active <unit> # dead config is not load-bearing
|
||||
```
|
||||
|
||||
## Secret-gap check (do this before adopting anything env-shaped)
|
||||
|
||||
The ignore list may cover only `.env`, so `frank-job-hunt.env` / `postgres.env` fall
|
||||
through, and a token-shape secret gate (`sk-`, `ghp_`, PEM, JWT) does not match a
|
||||
low-entropy `POSTGRES_PASSWORD=…`. Test both explicitly:
|
||||
|
||||
```bash
|
||||
cd ~/.omp && git check-ignore -v --no-index 'dotfiles/x/postgres.env' || echo "NOT IGNORED"
|
||||
sed -n 's/^patterns=//p' .githooks/pre-commit
|
||||
```
|
||||
|
||||
Fix = add `*.env` to the quarantine block **and** a password-assignment pattern to the gate,
|
||||
with a red test that stages a fake `POSTGRES_PASSWORD=` file.
|
||||
|
||||
## Land it
|
||||
|
||||
Packages are home-relative trees under `dotfiles/<pkg>/`; per-host overrides go in
|
||||
`zz-$(hostname)` (stowed last). `dconf` is loaded via `dconf load /`, `systemd-user` gets
|
||||
`daemon-reload` + conditional `enable --now`. Sequence: `scripts/deploy.sh --check` (dry run,
|
||||
reports divergents) → adopt → `deploy.sh` → confirm live paths are symlinks into the repo →
|
||||
`bats tests/` (includes the real-bundle `kb-lint`/coverage gate) → branch off `origin/main`,
|
||||
PR to `main`, never merge without Frank's OK.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
name: dotfiles-deployment-truth-audit
|
||||
description: "Answer \"is my X config actually tracked and deployed?\" per package: classify every file by readlink, expose tracked-then-ignored debt, and re-derive census numbers from the command printed on the page."
|
||||
---
|
||||
|
||||
Answer to "does this repo still track my <pkg> settings?" must be a measurement, never a remembered count. The tracked unit is the **file**, not the directory: a live directory can be a real directory and still be fully tracked, because every file inside it is a symlink into the repo.
|
||||
|
||||
Run in the deployment checkout (the checkout IS the deployment).
|
||||
|
||||
## 1. Tracked set, per package, both tiers
|
||||
|
||||
```bash
|
||||
cd ~/.omp # or the repo that owns dotfiles/
|
||||
git ls-files dotfiles/<pkg> # base tier
|
||||
git ls-files dotfiles/zz-$(hostname) | grep <pkg> # host override tier — counted separately
|
||||
```
|
||||
|
||||
The `zz-*` tier overrides base files by the same relative path, so a per-package count
|
||||
that ignores it under-reports what a machine actually runs.
|
||||
|
||||
## 2. Live wiring — classify every tracked file (never by directory)
|
||||
|
||||
```bash
|
||||
for f in $(git ls-files dotfiles/<pkg>); do
|
||||
rel=${f#dotfiles/<pkg>/}; live="$HOME/$rel"
|
||||
if [ -L "$live" ]; then printf 'link %-50s -> %s\n' "$rel" "$(readlink "$live")"
|
||||
elif [ -e "$live" ]; then printf 'REAL %-50s (NOT a link)\n' "$rel"
|
||||
else printf 'absent %s\n' "$rel"; fi
|
||||
done
|
||||
```
|
||||
|
||||
- `REAL` = the repo copy is a fiction: live state diverged, deployment never reached it.
|
||||
- `absent` = tracked relic, nothing consumes it (candidate for `git rm --cached`).
|
||||
- Spaces in filenames break the `for` loop (`palettes/Gruber Red.json` splits into two
|
||||
rows). Use `git ls-files -z | tr '\0' '\n'` + `while read -r` when names can hold spaces.
|
||||
- Reverse direction finds untracked-but-load-bearing files:
|
||||
`find ~/.config/<pkg> -maxdepth 2 \( -type f -o -type l \) | while read -r p; do case "$(realpath "$p")" in $PWD/dotfiles/*) ;; *) echo "not from repo: $p";; esac; done`
|
||||
|
||||
## 3. Live state that must stay untracked
|
||||
|
||||
Generated palette output and app state are expansion, not authorship. Distinguish by
|
||||
mtime cluster, not by name: `find ~/.local/state/<app> -type f -printf '%TY-%Tm-%Td %p\n'` —
|
||||
a same-second mass write is a generator; single later re-stamps are re-runs. Pointers a
|
||||
switcher script writes (e.g. `cfg/display.kdl -> desk-1-home.kdl`) are machine state too.
|
||||
|
||||
Then check the hazard this creates: grep the entrypoint for includes of those generated
|
||||
files. A **non-optional** include of a correctly-untracked generated file bricks a fresh
|
||||
box (`include "noctalia.kdl"` with no `optional=true` = niri cannot start until the theme
|
||||
pipeline has run once). Report it as a fresh-box failure, not a style note.
|
||||
|
||||
## 4. Tracked-then-ignored debt (invisible to every normal check)
|
||||
|
||||
```bash
|
||||
git ls-files dotfiles | grep AGENTTRASH # trash committed inside the tree
|
||||
git check-ignore --no-index -v <that path> # does an ignore rule match it?
|
||||
```
|
||||
|
||||
`.gitignore` patterns without an interior slash (`.AGENTTRASH/`) match at **any depth**, so
|
||||
the rule was fine all along — the file simply entered history *before* the rule existed, and
|
||||
an ignore rule never reaches an already-tracked path. Only repair: `git rm --cached <path>`
|
||||
plus moving the working copy to the root `.AGENTTRASH/` (never `rm`). `git check-ignore`
|
||||
without `--no-index` prints nothing for tracked paths and reads as "not ignored" — that
|
||||
false negative is the trap.
|
||||
|
||||
## 5. A census number is only valid with its command
|
||||
|
||||
If a page prints a classifier command, every number on it must be reproducible by running
|
||||
that command. A row that the command does not reproduce was written from memory — wrong, not
|
||||
merely stale. Fix the row, and state the correction in the commit message rather than
|
||||
silently editing the number.
|
||||
|
||||
## Verification before landing
|
||||
|
||||
`bats tests/`, `scripts/kb-lint.sh omp-docs`, `scripts/kb-coverage.py omp-docs`, and
|
||||
`scripts/deploy.sh --check` **from the worktree** — `--check` is read-only; never run the
|
||||
real deploy from a worktree, it repoints live symlinks into the worktree the heartbeat reaps.
|
||||
@@ -0,0 +1,80 @@
|
||||
---
|
||||
name: dotfiles-tier-live-audit
|
||||
description: "Audit the command-center dotfiles tier before running deploy.sh: prove per-file tracked/deployed state, refuse generated theme output, catch stale zz overrides, wrong-box dconf keys, and unresolvable systemd ExecStart specifiers."
|
||||
---
|
||||
|
||||
# Deploy-tier audit (`~/.omp` command center)
|
||||
|
||||
Run **before** any real `scripts/deploy.sh` on a box with a live session, and whenever a
|
||||
symlink, palette, wallpaper or unit behaves oddly. Facts and laws live in
|
||||
`omp-docs/tools/dotfiles-deploy.md` + `omp-docs/inventory/home-config-census.md` — this page
|
||||
is only the probe order. Do not restate the laws in new files; the vault owns them.
|
||||
|
||||
## 0. Classify per file, never per directory
|
||||
|
||||
```bash
|
||||
cd ~/.omp
|
||||
for f in $(git ls-files dotfiles/<pkg>); do t=$(readlink -f "$HOME/${f#dotfiles/<pkg>/}")
|
||||
printf '%-50s %s\n' "${f#dotfiles/<pkg>/}" "$([ -L "$HOME/${f#dotfiles/<pkg>/}" ] && echo LINK || echo real)"
|
||||
done
|
||||
```
|
||||
A live directory can be real and still fully tracked (every file a symlink). `niri` 8/2,
|
||||
`gtk-3.0` 3/0. `zz-$HOST` files count in the `zz` tier's row, not the base package's.
|
||||
|
||||
## 1. The plan, and how to read it
|
||||
|
||||
```bash
|
||||
./scripts/deploy.sh --check # writes nothing; read the last line, not the RELINK lines
|
||||
```
|
||||
`--check` prints `RELINK` for **every** path the `zz` tier carries even when the live link
|
||||
already points at the override (base is walked first, intermediate owner change reported).
|
||||
Only `actions=0` is a real summary; confirm a pending change with `readlink`.
|
||||
|
||||
## 2. Generated theme output must never be tracked or deployed
|
||||
|
||||
```bash
|
||||
git ls-files dotfiles | grep -Ei 'noctalia\.(kdl|toml|json|conf|css)|themes/noctalia'
|
||||
```
|
||||
Hit = generated palette output (same-millisecond mtime cluster proves it). Untrack
|
||||
(`git rm --cached` + `.gitignore` patterns); `deploy.sh` refuses the class (`SKIP … generated
|
||||
theme output`). Tracking one means the next sync quarantines the user's live palette and
|
||||
restores the committed colours.
|
||||
|
||||
## 3. Stale `zz` overrides
|
||||
|
||||
For each path present in both base and `zz-$HOST`: `diff` them, then compare each file's
|
||||
`git log -1 --date=short`. A `zz` copy older than the base change is a time bomb — sync
|
||||
relinks cross-package owners **silently** (`nothing to quarantine`). Delete the `zz` copy in
|
||||
the same commit that base absorbed.
|
||||
|
||||
## 4. dconf: compare key by key, never with `diff`
|
||||
|
||||
`dconf load /` writes only keys present, so a text diff misleads both ways. Parse both sides
|
||||
into `[section]/key → value` and report: value conflicts, keys the dump would newly write,
|
||||
and any value containing `/home/` (foreign-box state). File-chooser `last-folder-path`,
|
||||
`window-state`, `picture-uri*` are state, not preferences — delete them from the dump and
|
||||
clear the live keys if they were already deployed here.
|
||||
|
||||
## 5. Units: state, then resolvability
|
||||
|
||||
```bash
|
||||
for u in <units in dotfiles/systemd-user>; do printf '%s %s %s\n' "$u" \
|
||||
"$(systemctl --user is-enabled $u)" "$(systemctl --user is-active $u)"; done
|
||||
```
|
||||
`is-enabled` exits **1** for `linked` exactly as for `disabled` — read the printed word.
|
||||
Never start a unit that is already installed: `swayidle-lock` sits `enabled`/`inactive` and
|
||||
`--now` arms a screen lock mid-session. Specifiers are systemd's, not the shell's: probe
|
||||
`command -v "${bin//%h/$HOME}"` before believing `ExecStart … absent on this host` — the
|
||||
literal `%h/…` string made this repo's own heartbeat look foreign on every box.
|
||||
|
||||
## 6. Landing it
|
||||
|
||||
`bats tests/` (deploy contract runs `scripts/deploy.sh` in a scratch repo with `DEPLOY_REPO`,
|
||||
`DEPLOY_HOST`, stub `HOME`, stub `systemctl` on `PATH`) + `scripts/kb-lint.sh omp-docs` +
|
||||
`scripts/kb-coverage.py omp-docs`. Branch off `origin/main`, PR, never merge without Frank's
|
||||
OK (`gitea-merge-form-encoded`: form-encoded `do=merge`, `merge_style=no-ff`).
|
||||
|
||||
Then deploy from the **deployment root** `~/.omp` only — never from a worktree (live symlinks
|
||||
would point into a checkout the heartbeat reaps in 7 days). Acceptance: `actions=N
|
||||
quarantined=0`, two consecutive `--check` runs both `actions=0`, no `ENABLE … started` for a
|
||||
unit that was already installed.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: e2e-port-squat-triage
|
||||
description: "When an e2e seam on a fixed port fails weirdly (404s on routes that exist, stale behavior, health probe passes but app seems ancient), audit for leftover dev/debug servers squatting the port before believing any red or green."
|
||||
---
|
||||
|
||||
# E2E fixed-port squat triage
|
||||
|
||||
## Symptom classes (all seen with a stale server squatting the seam's fixed port)
|
||||
- Routes that exist in current `dist`/source return 404 ("Cannot POST /api/...") while unit tests through the same app factory pass.
|
||||
- A seam that is *green* but exercises old code: its setup curls `http://host:PORT/health` and gets a response from **whoever owns the port** — a leftover server, not the one it just spawned (the spawned one dies or idles with EADDRINUSE, sometimes still showing in `pgrep`).
|
||||
- Behavior matches an older build (missing mounts/guards) despite a fresh build step.
|
||||
|
||||
## Triage recipe (before touching code)
|
||||
1. Who owns the port:
|
||||
```bash
|
||||
ss -tlnp 2>/dev/null | grep :PORT
|
||||
```
|
||||
2. List every server process with **start times**:
|
||||
```bash
|
||||
for p in $(pgrep -f "path/to/server-entry"); do ps -o pid,lstart,cmd -p $p --no-headers; done
|
||||
```
|
||||
3. Compare start times against the build artifact timestamps (`ls -la dist/...`). A server older than the dist you just built is a liar — its in-memory modules predate the code under test.
|
||||
4. Kill all leftovers, then re-check: `pgrep -f "server-entry" || echo all-dead`.
|
||||
5. Rerun the seam. Only now is red/red and green/green meaningful.
|
||||
|
||||
## Why it happens
|
||||
- `kill %1` in **non-interactive** bash fails silently (no job control) — use explicit PIDs or `pkill -f <pattern>`.
|
||||
- Seam setups that "wait for health" cannot distinguish their own server from a squatter on the same port; a pre-flight "port must be free" assertion is the permanent fix (assert *before* spawn).
|
||||
|
||||
## Hardening (if you own the seam)
|
||||
Add to setup, before spawning: fail fast if the port is already listening — turns the silent liar into a loud error:
|
||||
```bash
|
||||
ss -tln "sport = :PORT" | grep -q LISTEN && { echo "port PORT busy — kill leftovers first" >&2; return 1; }
|
||||
```
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: fixture-delay-window-race-test
|
||||
description: Pin a race/abort bug in a fixture-server test by delaying only the Nth request so the interruption lands inside the in-flight window; assert via a driver-installed unhandledRejection reporter.
|
||||
---
|
||||
|
||||
# Reproducing a timing-window bug against a fixture server
|
||||
|
||||
Use when a defect only appears in a window — a promise created before it is awaited, a kill/abort landing mid-flight, a prefetch interrupted — and a fixture HTTP server answers instantly, so the window never opens. Earned in `dankvoice` issue #7 (`[Unhandled Rejection] AbortError` from `/say stop` during a pipelined synthesis).
|
||||
|
||||
## Why naive attempts pass vacuously
|
||||
|
||||
1. **Uniform delay on the fixture route** (`DV_LATENCY_MS` for every request) makes *all* requests resolve at the same time, so by the time the interruption fires nothing is in flight. The test goes green against the broken code — worse than no test.
|
||||
2. Asserting on a log line count that also counts interruption noise: `dankvoice`'s driver prints `PLAYS <n>` as the number of **player log lines**, and the stub appends `KILLED` on SIGTERM — so a single playback with one interrupt reports `PLAYS 2`. Count the payload lines instead (`grep -c 'PLAYER .*\.wav$'`).
|
||||
|
||||
## The shape that works
|
||||
|
||||
- **Delay only requests after the first.** The first response gets the pipeline into playback; the second stays in flight, which is the window:
|
||||
```ts
|
||||
const spoken = received.filter((c) => c.url?.endsWith("/audio/speech")).length; // already pushed
|
||||
const delay = spoken > 1 ? Number(process.env.DV_SYNTH_MS ?? 0) : 0;
|
||||
if (delay > 0) setTimeout(answer, delay); else answer();
|
||||
```
|
||||
- **Make the interruption land inside it.** Player holds long (`DV_HOLD=2`), interruption at a fixed offset (`DV_STOP_AT=400`) with the delayed response far later (`DV_SYNTH_MS=1200`): margins wide enough that the abort is neither before the request nor after the response.
|
||||
- **Turn unhandled rejections into observable output** in the test driver instead of letting the runtime crash on them — that is also exactly what the real harness does to the user:
|
||||
```ts
|
||||
process.on("unhandledRejection", (reason) => {
|
||||
console.log(`UNHANDLED ${reason instanceof Error ? `${reason.name}: ${reason.message}` : String(reason)}`);
|
||||
});
|
||||
```
|
||||
Then assert `[[ "$output" != *"UNHANDLED"* ]]` plus the concrete behavior (segments actually played, player killed).
|
||||
- **Leak checks measure a delta, not an absolute:** count `dankvoice-*` temp dirs before and after the run and compare; an absolute `== 0` fails on another test's leftovers.
|
||||
|
||||
## Order
|
||||
|
||||
Write the scenario first and watch it fail **on the rejection assertion specifically** — if it fails on some other assertion (a count, a kill line), the window is wrong, not the bug. Then fix, then confirm red→green on that one assertion before running the full suite.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: fleet-host-process-service-wiring
|
||||
description: "Add a host-process (non-quadlet) systemd USER service/exporter to serverAppConfigs — every file that must change together, with the signal-relay precedent and the Prometheus gateway/LAN reach rules."
|
||||
---
|
||||
|
||||
A host-process service (runs `python3`/a binary on the host, NOT a container) is the right shape when the thing must read host-only state (DRM/cgroup sysfs, `shells to nvidia-smi`, signal-cli) that a container cannot reach without passthrough. Precedent to mirror: `scripts/signal-relay.{py,service}` + `install_signal_relay` in `deploy-remote.sh`. A new one landed 2026-10-20: `scripts/gpu-exporter.{py,service}` + `deploy_gpu_exporter`.
|
||||
|
||||
## The unit is a USER unit, not a quadlet
|
||||
- Hand-written `scripts/<svc>.service`: `[Service]` `ExecStart=/usr/bin/python3 /home/dank/serverAppConfigs/scripts/<svc>.py` (runs straight from the git checkout — a pull updates it, no copy step), `Restart=always`, `[Install] WantedBy=default.target`.
|
||||
- It CAN be `systemctl --user enable`-ed (generated quadlet units cannot on this host's podman generator). With `Linger=yes` for `dank`, `[Install]`+`enable` is the whole boot story.
|
||||
|
||||
## Every file that MUST change together (a lone change on one side fails silently)
|
||||
1. `scripts/<svc>.service` — the unit. Base binds **loopback only**; the LAN address is added by a deploy **drop-in**, so the checked-in unit is never a wildcard.
|
||||
2. `scripts/deploy-remote.sh`:
|
||||
- a `deploy_<svc>()` function: `cp` the unit to `~/.config/systemd/user/`, write a `<svc>.service.d/bind.conf` drop-in with the per-host `Environment=..._BINDS=127.0.0.1,<lan>` (map LAN from `$TARGET`: dankclaw=192.168.0.143, dankserver=192.168.0.235 — do NOT hardcode one host if it runs on both), `daemon-reload`, `systemctl --user enable`+`restart`, then a **loop health gate** (`for i in $(seq 1 15); curl -fsS -m5 http://127.0.0.1:<port>/metrics && return; sleep 1`) — a one-shot curl races the bind and once failed a healthy deploy.
|
||||
- a dispatcher branch `<svc>) deploy_<svc> ;;` in the central `case "$service"` (near the observability branch).
|
||||
- an **allowlist entry on EVERY target** it runs on: `dankclaw:<svc>` and/or `dankserver:<svc>` (the allowlist `case` around the top of the file).
|
||||
3. `scripts/deploy-plan.py`: add `"<svc>"` to `SERVICE_ORDER`, to `DANKCLAW_SERVICES`/`DANKSERVER_SERVICES` (whichever it runs on), and a `scripts/<svc>.*` path-map in the `parts[0] == "scripts"` block: `if name in {"<svc>.py","<svc>.service"}: _add(services, "<svc>")`. `target_services` then filters per target.
|
||||
4. `scripts/test-changed.sh`: a `case` routing `scripts/<svc>.py|scripts/<svc>.service` → its seam + `deployment_scoping.bats` (place before the generic fallbacks). `tests/run.sh` runs all `*.bats` regardless, so the mandatory gate is covered even if this route is missing — but add it for focused runs.
|
||||
5. `shared/prometheus/prometheus.yml`: one scrape job per host, using the **reach rules** (below).
|
||||
6. Dashboard JSON + the owning page under `serverAppConfigs-docs/Infrastructure/Observability.md` (one owner per fact).
|
||||
|
||||
## Prometheus reach rules (the whole cross-file contract)
|
||||
Prometheus is a **bridge container**. On this netavark host a bridge container CANNOT dial the host's own LAN IP at a host-bind (only at a *published* port). Therefore:
|
||||
- **Same host (Dankclaw):** scrape `host.containers.internal:<port>` (netavark delivers it to the host LAN bind). Scraping `192.168.0.143:<port>` reads DOWN forever even though the endpoint is healthy.
|
||||
- **Different host (Dankserver):** plain LAN `192.168.0.235:<port>` reaches it.
|
||||
- Never target a `*.franksdank.cloud` vhost — Prometheus cannot solve an Anubis challenge; it is red by construction.
|
||||
- The exporter must bind BOTH loopback and the host LAN (like `node-exporter`/`cadvisor`), so `host.containers.internal` and the LAN both land.
|
||||
|
||||
## Firewall
|
||||
Leave it LAN-open like node-exporter/cadvisor (unauthenticated metrics on the LAN is the repo's exporter posture). Only signal-relay gets an iptables allowlist (loopback+host+`10.89.0.0/16`+DROP) because it is an unauthenticated message-SENDING webhook — do not copy that lock onto a read-only exporter.
|
||||
|
||||
## Seam test (tests/<svc>.bats) pattern
|
||||
- Black-box the binary with injectable inputs (fixture dirs / stub commands), assert only consumer-visible output; assert **absent means absent** (anchor guards to a sample line `^metric{`, not the always-emitted `# TYPE` header — grepping the bare family name matches the header and false-fails).
|
||||
- Cross-file contracts (grep the other files): the unit's `ExecStart`/bind posture, the deploy function + allowlist + dispatch + health gate, the planner selection (run `deploy-plan.py --target T --paths scripts/<svc>.py`, assert output == `<svc>` per target), the prometheus targets, the dashboard `json.load` + that its `targets[].expr` reference the emitted families.
|
||||
|
||||
## Gotchas hit while building gpu-exporter
|
||||
- Values are float-normalised (`397.40`→`397.4`); a test pinning the literal fails — assert the normalised value. Derive expected byte counts from real units (MiB×1048576, sysfs µW/1e6, temp1_input/1000), don't guess the multiply.
|
||||
- `xdg-open <PR-URL>` after opening the PR (repo law) — don't make Frank hunt in Gitea.
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: fleet-service-runtime-census
|
||||
description: "Answer \"what is service running right now?\" for a dank-fleet service: unit state, container image/digest, endpoint contract, loaded env — machine over repo, in that order."
|
||||
---
|
||||
|
||||
Answer "what is service running right now?" for a dank-fleet service without trusting the repo. Repo state is intent; the machine is truth, and they differ by however long since the last deploy.
|
||||
|
||||
## Order of operations (cheap → expensive)
|
||||
|
||||
1. **Unit state on the host** (rootless quadlet ⇒ `--user`; some units are root):
|
||||
`ssh -o BatchMode=yes host 'systemctl --user show S -p ActiveState -p SubState -p ExecMainStartTimestamp -p NRestarts'`
|
||||
Boot-loop tell: `NRestarts` climbing with a short `ExecMainStartTimestamp`.
|
||||
2. **The container's own view**: `podman ps --filter name=S --format '{{.Image}} {{.Status}} {{.Names}}'`, then `podman inspect` for `Image` (tag **and** `RepoDigests`) and `Env`.
|
||||
3. **The endpoint contract, from the service itself** — this is the only answer that matters to a consumer:
|
||||
`curl -s host:port/v1/models` (served model id), `curl -s host:port/props` (`n_ctx`, `n_ctx_train`, caps), `curl -s -o /dev/null -w '%{http_code}' host:port/health`.
|
||||
4. **Effective flags**: the live process is the truth. `systemctl --user cat S` shows `EnvironmentFile=`; read that file. A repo `scripts/S.env` can be ahead of, or behind, what the running unit loaded — and a unit restart is what re-reads it.
|
||||
5. **Logs last**: `journalctl --user -u S -n 80 --no-pager` for the boot ledger / refusal messages, after you know whether it is even up.
|
||||
|
||||
## Facts that bite
|
||||
|
||||
- **A service is up ≠ configured how you think.** llama.cpp's `/v1/models` reports `max_model_len`, not the engine's per-slot ceiling; halogen reports `n_ctx` per slot. Read `/props` before quoting any window number.
|
||||
- **`systemctl is-enabled` answers enablement, not activity.** A timerless service reports `linked`/`static`; `deploy.sh` that prints `ENABLE` for it is lying. Read `systemctl is-enabled --quiet` exit status, not the text.
|
||||
- **`NRestarts: 0` with `ActiveState=activating` is a crash-loop only if the timestamps move**; check `ExecMainStartTimestamp` twice, 5 s apart.
|
||||
- **Read-only is always allowed** from the dev box (`ssh -o BatchMode=yes`); state changes on remote boxes are not, without a per-run OK. Production services (`frank-archon`, `frank-dragoon`, `frank-zealot`) are never stopped/restarted to "see what happens".
|
||||
- **Digest-pinned images**: "when was this image updated?" is answered by `podman inspect` `RepoDigests` + the registry API, not by the unit's start time (a restart re-uses the local digest).
|
||||
|
||||
## Report
|
||||
|
||||
One line per layer with the evidence: unit (`active (running)`, since, restarts), container (image + digest), endpoint (served id, `n_ctx`, health code), loaded env (file path + the knobs that matter), and explicitly which of those came from the machine versus the repo — that gap is the answer to most "why is it behaving like that" questions.
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
name: fleet-status-daily-report
|
||||
description: "Answer \"status update / what happened yesterday\" for the dank fleet: git-window record on origin/main plus a live per-host census (endpoints, units, RAM, DNS), with the reboot-attribution check that stops false incident causes."
|
||||
---
|
||||
|
||||
# Fleet status report (Dankclaw + Dankserver)
|
||||
|
||||
Use for "what's the state of the fleet", "what happened yesterday", or a post-incident sweep. Read-only.
|
||||
For one service's live state, prefer `fleet-service-runtime-census`; this is the fleet-wide sweep.
|
||||
|
||||
## 1. The day's record (repo is the ledger)
|
||||
|
||||
```bash
|
||||
git fetch origin --quiet
|
||||
git log --no-merges --since="YYYY-MM-DD 00:00" --until="YYYY-MM-DD+1 00:00" \
|
||||
--date=format:'%H:%M' --pretty='%ad %h %s' origin/main
|
||||
git log --merges --since=... --pretty='%ad %h %s' --date=format:'%H:%M' origin/main # PR numbers
|
||||
```
|
||||
Merge lines carry the PR number — that is the review trail; the non-merge log is what actually changed.
|
||||
Group commits into streams (endpoint names lead the subject) rather than listing 20 subjects.
|
||||
|
||||
## 2. Live census (parallel: one call per host + one endpoint sweep)
|
||||
|
||||
Endpoint sweep from any LAN box — `/health` is honest on all four (archon binds only after full load):
|
||||
|
||||
```bash
|
||||
for u in 192.168.0.143:7770 192.168.0.143:7771 192.168.0.235:7770 192.168.0.143:7772; do
|
||||
printf "%s %s\n" "$u" "$(curl -s -m 12 -w ' [%{http_code}]' http://$u/health | head -c 160)"; done
|
||||
curl -s http://192.168.0.143:7770/v1/models # dragoon lists EVERY engine it serves, incl. NPU-lane models
|
||||
```
|
||||
|
||||
Per host: `uptime -s; uptime -p`; `systemctl show <llm units> -p ActiveEnterTimestamp -p NRestarts -p DropInPaths`
|
||||
(`sudo` for the root LLM units, `--user` for the quadlets); `list-units --failed`; `free -h`;
|
||||
`journalctl -k --since "<evening>" -p err`; the host's own engine log tail.
|
||||
|
||||
## 3. Attribution rules (where reports go wrong)
|
||||
|
||||
- **Never call a host reboot from a container's start timestamp.** Check `uptime -s` first; if uptime spans the
|
||||
event, the trigger was a **container restart** — find it with
|
||||
`journalctl --user -u <name>.service --since … --until …` + `podman inspect <c> --format '{{.State.StartedAt}}'`.
|
||||
(This exact mistake mislabelled a 2026-10-19 Pi-hole incident as boot-caused; the law's remedy belongs after
|
||||
*any* restart, which is a different action item.)
|
||||
- `podman stats` is a 28-container wall — sort/limit it; RSS of the LLM tenants is what matters.
|
||||
- This podman's inspect template has **no `.Config.PodName`, no `.Config.GroupAdd`, and `.HostConfig.Devices`
|
||||
can read `[]` even when a quadlet `AddDevice=` worked**. Prove GPU use from the engine's own log
|
||||
(`ggml_vulkan: Found 1 Vulkan devices: … Radeon 8060S (RADV GFX1151)`), not from inspect.
|
||||
- Kernel lines: **count** occurrences (`journalctl -k --since '7 days' | grep -c "NETDEV WATCHDOG"`) before
|
||||
calling a one-off a trend.
|
||||
- Failed units: `systemctl show <u> -p ExecMainExitTimestamp -p ExecMainStatus --value` separates
|
||||
"broken since boot" from "broke yesterday" — and a nightly failure (e.g. logrotate) is a live risk, not noise.
|
||||
|
||||
## 4. Report shape
|
||||
|
||||
Stream table (what landed) → live-state table (what is true now) → **corrections to my own earlier record** →
|
||||
open items that need Frank's decision, each with the evidence line and the one action. Flag anything that would
|
||||
touch host config outside the repo as a decision, never a silent fix.
|
||||
@@ -1,35 +1,37 @@
|
||||
---
|
||||
name: frank-job-hunt-maintenance
|
||||
description: "Maintain frank-job-hunt agents: live-census gate hardening loop, registry locator verification, and the workstation/Dankserver/Dankclaw deploy-sync ritual."
|
||||
description: "Maintaining frank-job-hunt: the census gate's fail-closed rules, ATS locator token derivation, and the two-host deploy-sync verification."
|
||||
---
|
||||
|
||||
# frank-job-hunt maintenance
|
||||
Use when maintaining `frank-job-hunt`: the live-census gate, the registry locator, or Dankserver/Dankclaw deploy-sync. The repo is built and deployed; agents live on Dankserver, ntfy on Dankclaw.
|
||||
|
||||
Repo `~/Projects/frank-job-hunt` (Gitea `dank/frank-job-hunt` on Dankclaw 192.168.0.143:3000). Production: Dankserver user timers `frank-job-sweep.timer` 07:30, `frank-job-digest.timer` 08:15 (`Linger=yes`). ntfy relay: Dankclaw quadlet `ntfy.service`, route `ntfy.franksdank.cloud` (no anubis), LAN record in `hosts/custom.list` (Pi-hole v6 file at `/home/dank/Containers/pihole/etc-pihole/hosts/custom.list`; v6 API paths are hostile — edit the file, then `podman exec pihole pihole restartdns`). Secrets: `~/.config/frank-job-hunt.env` (600, workstation + Dankserver).
|
||||
## The two rules that caused every bug here
|
||||
|
||||
## Law (validated 2026-10-11)
|
||||
- The fleet judge (`frank-zealot`, dankclaw:7771) is a **ranker, never a gate** — it inflates anything reaching it (non-technical roles scored 88-91). All discrimination lives in deterministic gates (`agents/lib/gates.ts`); zealot only ranks survivors.
|
||||
- Thinking-model fault: verdict lands in `reasoning_content` with `content:""` → judge sends `chat_template_kwargs:{enable_thinking:false}` (0.7s clean JSON).
|
||||
- **Family beats topic**: role family (recruiter/sales/finance/PM/designer/people-ops/revops/…) gates `gate:scope` unconditionally — no infra-token override ("DevOps Recruiter" is recruiting). Compound technical titles keep the `INFRA_TITLE` override via `NEG_TITLE` branch.
|
||||
- **gate:region**: remote is US-only (visa reality). "Remote - Bangalore" = onsite abroad → gate; `US` regex pass wins over `NONUS` match ("U.S. & Canada" passes).
|
||||
1. **A URL is data.** The census gate and the registry locator treat portal URLs as opaque strings; nothing may template a URL from a company name. `jobs.dankclaw.example` is a fixture host (RFC 2606) — never derive a real domain from a company name, and never click a link the pipeline constructed.
|
||||
2. **A stale row is worse than a missing row.** Every gate must fail closed: an unverifiable posting is reported as unverifiable, not kept as live. Deleting is allowed; asserting is not.
|
||||
|
||||
## Gate hardening loop (live-census, TDD)
|
||||
1. `bun agents/sweep.ts --dry` (full lane ≈ 30–90 s; 852 postings ≈ 62 pass / 790 filed).
|
||||
2. Classify leaks in the top-10: family gap / region gap / scoring tail. Accept long tail (Director/Founder edge cases) — digest + human curation is the design.
|
||||
3. Token fix in `gates.ts` regexes (`NEG_FAMILY` = no override; `METRO`/`US`/`NONUS` for arrangement). NEVER dedupe-check by memory — read file first; the file has bitten line-offset edits repeatedly.
|
||||
4. Pin every leak with a named regression test in `agents/test/gates.test.ts` before/with the fix.
|
||||
5. `bun test agents/test/ && bun x tsc --noEmit` — hooks re-run these at commit.
|
||||
6. Ship: `git commit` (pre-commit = tests + tsc + docs-link + secret scan) → `git push` → `ssh dankserver 'cd ~/Projects/frank-job-hunt && git pull'` (timers pull from master; Dankserver drift = tomorrow's run is stale).
|
||||
## Census gate
|
||||
|
||||
## Registry verification (monthly, `agents/verify-registry.ts`)
|
||||
Scouts' `ats`/`ats_identifier` are guesses (14/84 were wrong). Ritual:
|
||||
1. `bun agents/verify-registry.ts` (dry, ~70 s serial) → REPAIR census.
|
||||
2. `--write` applies; repairs with live jobs preferred over empty boards (0-job boards 404 later).
|
||||
3. Unresolved = honest manual lane (vanity Workdays, self-built ATS incl. majors, neogov) — NOT a code bug; per-entry error census in sweep output is the same signal.
|
||||
4. Update `targets/README.md` census lines + `docs/log.md`; commit/push/pull (same ship step).
|
||||
`scripts/census-gate.sh` decides keep / stale / unverifiable per row.
|
||||
|
||||
## Pitfalls
|
||||
- `--ats=<x>` smoke filters the census line — "N API-sweepable" reflects the filter, not the registry; don't diagnose a phantom bug.
|
||||
- Workday vanity careers URLs (`careers.uw.edu`) are NOT CXS-API-resolvable; tenant-only identifiers are (site path required).
|
||||
- Root `README.md` vs `targets/README.md` name collision in edit anchors: use full path headers.
|
||||
- Zealot endpoint probe: `curl http://192.168.0.143:7771/v1/models`.
|
||||
- Live proof = HTTP GET on the posting URL against the ATS API or the page itself; `404`/`410` ⇒ stale, non-200 with no body ⇒ unverifiable. Never infer from age alone.
|
||||
- The gate runs against the *live* registry, so a row that passes here and then fails in production means the registry drifted — re-run before blaming the gate.
|
||||
- Test seams: fixture ATS server + `…_BASE_URL` env. Tests assert the row's fate (kept / dropped / flagged), never internal state.
|
||||
|
||||
## Registry locator
|
||||
|
||||
Resolves which ATS a company uses (Greenhouse `boards-api.greenhouse.io/v1/boards/<token>/jobs`, Lever `api.lever.co/v0/postings/<token>`, Ashby `api.ashbyhq.com/posting-api/job-board/<token>`, Workable, SmartRecruiters).
|
||||
|
||||
- Token ≠ company name: it comes from the discovered posting URL, so a locator change must keep the derivation (URL → token), not substitute a slug.
|
||||
- Unknown ATS ⇒ unverifiable, not "no jobs". "Zero jobs" is a strong claim; only a successful API answer with an empty list may say it.
|
||||
- Rate limits: 429 ⇒ back off and mark unverifiable for this pass, never stale.
|
||||
|
||||
## Deploy-sync (Dankserver agents, Dankclaw ntfy)
|
||||
|
||||
- Two hosts, two roles: inference/agents on Dankserver, notification sink on Dankclaw. A deploy that only touches one leaves the pair inconsistent — the agent can post to a sink that no longer accepts its token, silently.
|
||||
- Deploy is `scripts/deploy.sh` from the correct checkout (dev side, clean tree); it pushes the branch first, so a stale local `main` makes the push fail — `git merge --ff-only origin/main` before deploying after any externally merged PR.
|
||||
- Verify after deploy on the *served* surface: the agent answers a probe, and a test ntfy message arrives on the phone. "The unit is active" is not the proof.
|
||||
|
||||
## Tests
|
||||
|
||||
Black-box, fixtures over network: fake ATS servers on localhost, fake ntfy sink, temp registry file. Assert output rows and exit status. Never assert that an env var is set or that a helper was called.
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
name: frank-tts-add-cloned-voice
|
||||
description: "Add or repair a cloned voice on the dank fleet: pack builder, the reference.wav contract, the boot warm hook (argv-cap and env-quote laws), and how to prove a voice really answers."
|
||||
---
|
||||
|
||||
# Adding (or repairing) a cloned voice on the dank fleet
|
||||
|
||||
Repo law lives in `serverAppConfigs-docs/Dankclaw/Voice Packs.md` and
|
||||
`Frank TTS Engine.md` — read those for the pack format. This skill is the
|
||||
**operating sequence plus the traps that cost real time on 2026-10-18**.
|
||||
|
||||
## Stack, fast
|
||||
|
||||
- `frank-tts-engine` — arm A `qwentts.cpp tts-server`, container-internal `:8880`,
|
||||
publishes **no port**. Reach it with `podman exec frank-tts-engine curl …`.
|
||||
- `frank-tts` — OpenAI façade on host `:7772`; resolves voice **by name**.
|
||||
- Packs: `/home/dank/voices/<name>/` with `reference.wav` (+ manifest, `.spk`,
|
||||
`.rvq`). Builder: `scripts/build-voice-pack.sh` (crop 19.84–25 s, first 10 s of
|
||||
any source is forbidden).
|
||||
- Weights: `/home/dank/models/Qwen3-TTS-GGUF/` —
|
||||
`scripts/download-frank-tts-engine-qwentts.sh [--verify-only]` (1.4 GiB, ~18 s
|
||||
on the LAN; verifies byte count + Hub LFS sha256).
|
||||
|
||||
## Sequence
|
||||
|
||||
1. Build the pack (dry-run first — sizes are arithmetic and asserted).
|
||||
2. Deploy the engine only if its files changed:
|
||||
`./scripts/deploy.sh dankclaw --service frank-tts-engine`
|
||||
(first run builds the image: ~10 min recursive source clone).
|
||||
3. Register/warm: `podman exec frank-tts-engine /app/warm-voices.sh`
|
||||
— registers **every** pack each run (the engine replaces a name it already
|
||||
holds; `tools/tts-server.cpp:148`), then synthesizes twice per voice.
|
||||
4. Prove from the consumer side, not the config:
|
||||
`curl -s localhost:7772/v1/audio/voices` then a real
|
||||
`POST /v1/audio/speech` and listen to the bytes.
|
||||
|
||||
## Trap 1 — the kernel's argument cap (this broke the first deploy)
|
||||
|
||||
`/usr/bin/curl: Argument list too long` in the warm log ⇒ a request body was
|
||||
passed as **one argv string**. Linux caps a single argument at
|
||||
`MAX_ARG_STRLEN = 131 072 bytes`; a ~1 MB `reference.wav` base64's to ~1.27 MB.
|
||||
curl never runs, and hooks report it as `HTTP no response` while `/health` is
|
||||
green. Fix = write the body to a file and use `--data-binary @file`; feed the
|
||||
payload through a redirection, not `$(base64 …)`.
|
||||
Regression-test shape: **real** curl + real listener + a fixture >128 KB, so the
|
||||
kernel (not a stub) produces the failure. Any pack over ~96 KB trips it.
|
||||
|
||||
## Trap 2 — two readers of the env file disagree about quotes
|
||||
|
||||
`bash set -a; source` strips `"…"`. podman's `--env-file` (how the unit really
|
||||
reads it) does **not**. A value interpolated into a JSON body therefore arrives
|
||||
quoted and invalidates it. Test both readers, not just `source`.
|
||||
|
||||
## Trap 3 — locally built images skip rebuilds on a stale hash
|
||||
|
||||
`build_compose_image` compares a `cfhash` label; it must cover every file the
|
||||
`Containerfile` `COPY`s, or an edited `app.py`/hook deploys "successfully" while
|
||||
the container runs old code.
|
||||
|
||||
## Measured numbers (2026-10-18, deployed build — retire the trial's ladder)
|
||||
|
||||
| thing | value |
|
||||
|---|---|
|
||||
| first synthesis of a freshly registered voice | 1 883 ms |
|
||||
| later short requests | 1 226–1 283 ms |
|
||||
| ~110-word request (repeat) | 9.4 s / 9.8 s — tracks generated audio, not state |
|
||||
| whole boot warm, 3 packs | 15.2 s |
|
||||
| engine resident | ~4.0 GiB |
|
||||
|
||||
The trial's `6.2–8.0 s cold → 0.5 ms warm` claim does **not** reproduce; warming
|
||||
is for registration (registry is memory-only, re-register ≠ warm) plus ~0.6 s.
|
||||
|
||||
## Ops notes
|
||||
|
||||
- `systemctl --user stop` **removes** the quadlet container — `podman start` then
|
||||
finds nothing. To run the engine without its `ExecStartPost` hook, use a
|
||||
separate container, not stop+start.
|
||||
- `podman cp` of a fixed script into a running container is a **trial only**: the
|
||||
layer dies with the container. Merge + redeploy to make it durable.
|
||||
- Nothing on the voice pair exposes `/metrics`; Prometheus can't scrape it (a
|
||||
blackbox HTTP probe on 7772 is the only honest option).
|
||||
@@ -0,0 +1,81 @@
|
||||
---
|
||||
name: frank-tts-consumer-voice-triage
|
||||
description: "Triage a consumer (OpenWebUI, omp, open-notebook) rejecting or ignoring a frank-tts voice name: prove whether the server or the browser-side Kokoro.js engine produced the error."
|
||||
---
|
||||
|
||||
# frank-tts consumer voice triage
|
||||
|
||||
A consumer says a voice name is invalid, speaks in the wrong voice, or is
|
||||
silent. `frank-tts` (`192.168.0.143:7772`, shim
|
||||
`dankclaw/voiceservice/app.py`) resolves `voice=<name>` against
|
||||
`/home/dank/voices/<name>/reference.wav` and answers **400 + the list of
|
||||
voices that exist** for a name it cannot resolve — never a silent fallback.
|
||||
Any error naming a *different* voice list did not come from it.
|
||||
|
||||
**Signature that decides the case immediately:** an enum of ~29 names
|
||||
`af_heart, af_alloy, … am_*, bf_*, bm_* … bm_fable` is **Kokoro's fixed pack**,
|
||||
the voice list of **Kokoro.js running in the browser** (OpenWebUI ships it
|
||||
in its own front end as TTS engine value `browser-kokoro`, with a personal
|
||||
"Kokoro.js dtype" setting; model `onnx-community/Kokoro-82M-v1.0-ONNX`).
|
||||
`narrator1/2/3` are frank-tts cloned voices. Kokoro cannot clone.
|
||||
|
||||
## Instrument ladder (stop at the first that answers)
|
||||
|
||||
1. **What the endpoint actually accepts** (never infer from a consumer's text):
|
||||
`curl -s http://127.0.0.1:7772/v1/audio/voices` → JSON array of pack names.
|
||||
2. **Speak with the consumer's exact body, from inside the consumer's
|
||||
container** — same network, same shape. Write the JSON to a file and
|
||||
`--data-binary @file` (nested `sh -c` quoting silently sends an empty
|
||||
body and yields a bogus `400 malformed request body`):
|
||||
```
|
||||
printf '%s' '{"model":"frank-tts","input":"probe","voice":"narrator1","response_format":"mp3","speed":1.0}' > /tmp/p.json
|
||||
podman cp /tmp/p.json <consumer>:/tmp/p.json
|
||||
podman exec <consumer> curl -sS -o /tmp/o.bin -w 'http=%{http_code} type=%{content_type} bytes=%{size_download}\n' \
|
||||
-X POST http://host.containers.internal:7772/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' --data-binary @/tmp/p.json
|
||||
```
|
||||
`200 audio/mpeg|audio/wav` ⇒ the server path is healthy; the failure is elsewhere.
|
||||
3. **Did the request arrive at all?** `podman logs --tail 400 <consumer> | grep -c audio/speech`.
|
||||
A server-path failure leaves a log line **and** a frank-tts error body; a
|
||||
client-side engine leaves **neither**. This contrast is the discriminator.
|
||||
4. **Where the rejection string lives** (proves the emitter without guessing):
|
||||
```
|
||||
podman exec <consumer> sh -c 'grep -ho ".\{200\}Should be one of.\{80\}" /app/build/_app/immutable/chunks/*.js | head'
|
||||
podman exec <consumer> sh -c 'grep -ho ".\{60\}af_heart.\{200\}" /app/build/_app/immutable/chunks/*.js | head -2'
|
||||
```
|
||||
Matched inside the front-end bundle ⇒ browser-side validator, not the API.
|
||||
5. **Config authority sweep** (OpenWebUI): admin config is the `config`
|
||||
table (one key per row, JSON `value`; expected
|
||||
`audio.tts.engine=openai`, `audio.tts.model`, `audio.tts.voice`,
|
||||
`audio.tts.openai.api_base_url`); per-user overrides are the `settings`
|
||||
JSON column of table `user` (singular, no `is_active`; some rows hold the
|
||||
string `null` → guard it). `audio.stt.*` and `audio.tts.*` are separate.
|
||||
No `sqlite3` CLI in the image or on the host: `podman exec -i <c> python -`
|
||||
with the stdlib module. **`podman exec` needs `-i` to accept a heredoc
|
||||
script** — without it the script is silently dropped and you get no output.
|
||||
|
||||
## Known traps
|
||||
|
||||
- OpenWebUI is a **rootless** quadlet: `sudo podman …` reports *no such
|
||||
container*. Read it rootless.
|
||||
- `podman ps` in this podman build has no `.ImageName` field — use `{{.Image}}`.
|
||||
- The in-browser engine choice is **per-browser storage**: empty
|
||||
`user.settings` + correct admin config + zero backend hits is the normal
|
||||
shape of this bug. Fix is UI: Settings → Audio → TTS Engine → **Default**.
|
||||
- `GET /api/config` publishes no `audio` block, and nonexistent `/api/...`
|
||||
paths return the SPA HTML with `200` — never conclude "config missing" from
|
||||
an HTML body.
|
||||
- TTS is memory-registry sensitive: the engine's voice registry is
|
||||
memory-only, so "every name returns 400" after a restart points at the warm
|
||||
pass, not at the caller (see [[frank-tts-add-cloned-voice]]).
|
||||
|
||||
## Local playback side (omp `/say`, manual probes)
|
||||
|
||||
If synthesis succeeds (`200`, sane WAV bytes) but nothing is audible, the
|
||||
fault is the PipeWire graph, not the endpoint:
|
||||
`wpctl status` (default sink marked `*`, and *Default Configured Devices* —
|
||||
a remembered-but-offline `bluez_output.*` there is the classic silent
|
||||
route), `wpctl get-volume @DEFAULT_AUDIO_SINK@`, `pw-play -v <file>` (expect
|
||||
`stream drained` + exit 0), and target a device explicitly with
|
||||
`pw-play --target <node-id>` to identify which hardware is live. A one-word
|
||||
clip (~0.7 s) is easy to miss — probe with a multi-second sentence.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: frank-tts-repoint-voice-consumers
|
||||
description: "Rotate/rename a frank-tts voice across all consumers (repo DEFAULT_VOICE, Open Notebook speaker profiles over REST, docs) after packs change."
|
||||
---
|
||||
|
||||
## When
|
||||
frank-tts voice packs on `/home/dank/voices` (Dankclaw) changed — packs added, removed, or renamed — and old voice names (e.g. `narrator1/2/3`) no longer resolve. Consumers still sending a dead name now 400 with `unknown voice 'x'; available voices: ...`. This is a maintenance sweep, NOT a bug; confirm with the operator whether the change is intentional before "fixing".
|
||||
|
||||
## Facts that make this non-obvious
|
||||
- frank-tts (:7772) resolves a voice as a **directory**: `<name>/reference.wav` must exist. `model` field is ignored; the voice comes solely from `voice`/`voice_id`.
|
||||
- Live census: `curl -s http://127.0.0.1:7772/v1/audio/voices` (and `/health` returns the same list). This is the truth; the vault docs may be stale.
|
||||
- Two consumers carry voice names, in **different stores**:
|
||||
- **repo**: `scripts/frank-tts.env` `DEFAULT_VOICE=` — answers a voice-less request (the omp harness's normal shape). Deploy via git.
|
||||
- **app-side**: Open Notebook speaker profiles (SurrealDB) and OpenWebUI `audio.tts` voice — NOT repo files.
|
||||
|
||||
## Step 1 — repo: DEFAULT_VOICE
|
||||
Edit `scripts/frank-tts.env` `DEFAULT_VOICE=<new real voice>`. Trunk-based: this worktree can't `git checkout -b` (shared-workspace policy); commit on HEAD and `git push origin HEAD:refs/heads/<branch>`, open PR vs `main`. No bats test pins the DEFAULT_VOICE value (the tts bats use `narrator1` as a self-contained fixture that builds its own `reference.wav` — mechanism test, leave it).
|
||||
|
||||
## Step 2 — Open Notebook speaker profiles (REST, host-side, takes effect immediately)
|
||||
Voice ID is **free text passed to the TTS request untouched** (no enum/whitelist). The active fleet profile is `dank-duo` (episode profile `dank-daily` → speaker_config). PW is the host `OPEN_NOTEBOOK_PASSWORD`.
|
||||
|
||||
```bash
|
||||
ssh dankclaw bash -s <<'REMOTE'
|
||||
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
|
||||
base=http://127.0.0.1:5056/api/speaker-profiles
|
||||
# census: name -> speaker/voice_id
|
||||
curl -s -K "$cfg" "$base" | jq -c '.[]|{name,speakers:[.speakers[]|{name,voice_id}]}'
|
||||
# repoint by NAME (GET-by-name returns full record incl id; PUT whole transformed object to /{id})
|
||||
curl -s -K "$cfg" "$base/dank-duo" > /tmp/s.json
|
||||
id=$(jq -r .id /tmp/s.json)
|
||||
jq '(.speakers[].voice_id) |= ({"OLD1":"NEW1","OLD2":"NEW2"}[.] // .)' /tmp/s.json > /tmp/s2.json
|
||||
curl -s -K "$cfg" -X PUT -H 'Content-Type: application/json' -d @/tmp/s2.json "$base/$id" -w ' HTTP %{http_code}\n'
|
||||
curl -s -K "$cfg" "$base/dank-duo" | jq -c '[.speakers[]|{name,voice_id}]' # verify
|
||||
rm -f "$cfg" /tmp/s.json /tmp/s2.json
|
||||
REMOTE
|
||||
```
|
||||
- A duo has only 2 speakers — an extra voice is available-but-unassigned, not missing.
|
||||
- Upstream's seeded profiles (`business_panel`, `solo_expert`, `tech_experts`) use OpenAI names + `voice_model:null`; they aren't wired to frank-tts — leave them.
|
||||
|
||||
## Step 3 — OpenWebUI (check before touching)
|
||||
`curl -s http://127.0.0.1:<webui-port>/api/config | jq '.audio.tts'` (port from `podman ps`). If `engine`/`openai.*` are null, the active path is `browser-kokoro` (synthesizes in-browser) and the stored voice is **dormant, not broken** — flag, don't edit a live UI unasked.
|
||||
|
||||
## Step 4 — docs (point of use, one owner)
|
||||
Update the census + speaker-mapping lines in `serverAppConfigs-docs/Dankclaw/Frank TTS.md` and `Open Notebook.md` to the live packs. Leave DATED measurement logs (e.g. a `2026-10-19` `/health` probe) as history. Do NOT chase `scripts/voice-packs.env` / `Voice Packs.md` narrator builder defs — ask the operator whether those packs are being rebuilt or retired.
|
||||
|
||||
## Verify (behavior, not config text)
|
||||
Synthesize each new voice and expect 200 + distinct bytes:
|
||||
```bash
|
||||
curl -s -o /dev/null -w '%{http_code} %{size_download}B\n' -X POST http://127.0.0.1:7772/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' -d '{"model":"voxcpm2","input":"test","voice":"<name>"}'
|
||||
```
|
||||
`tests/run.sh` EXIT=0 before commit. `DEFAULT_VOICE` only lands after merge + `./scripts/deploy.sh dankclaw --service frank-tts`; run that from the dev `main` checkout (`/home/blank/Projects/serverAppConfig`), NOT the agent worktree (deploy.sh refuses a non-`main` branch and pushes `main` forward — it must run on the dev box, not the target).
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: frank-tts-voice-locate
|
||||
description: "Answer fleet-voice questions (where voices are stored, add a voice) by jumping straight to the serverAppConfig vault pages and the live 7772 roster probe, instead of re-searching Gitea for a frank-tts repo that does not exist."
|
||||
---
|
||||
|
||||
# Frank-TTS voice facts: skip the search, go to the owners
|
||||
|
||||
Questions like "how do I add a voice?", "where are voices stored?", "why is narratorN missing?" are already answered elsewhere. The frank-tts code does NOT live in a Gitea repo named frank-tts (repo search finds nothing; it is `dank/serverAppConfigs`), and the engine repo is upstream qwentts.cpp pinned at `5916bf7c`.
|
||||
|
||||
## Resolution order (one grep, no repo search)
|
||||
|
||||
1. **Vault pages (canonical, all of it lives here)** in the local checkout `~/Projects/serverAppConfig`:
|
||||
- `serverAppConfigs-docs/Dankclaw/Voice Packs.md` — pack anatomy, the builder, the 10-second source invariant, "Adding a fourth voice" steps, engine-bump arithmetic.
|
||||
- `serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md` — registry law (memory-only; warm hook), routes, Vulkan device law, troubleshooting table.
|
||||
- `serverAppConfigs-docs/Dankclaw/Frank TTS.md` — the 7772 facade contract.
|
||||
- Code/seam: `dankclaw/voiceservice/`, `scripts/build-voice-pack.sh`, `scripts/voice-packs.env`, `tests/voice_pack_unit.bats`.
|
||||
2. **Live roster** (authoritative over any doc census): `curl -s http://192.168.0.143:7772/v1/audio/voices` (bare JSON array; `/health` also echoes `voices`). A missing name = its dir lacks `reference.wav` or the name is not lowercase — check `/home/dank/voices/<name>/` via `ssh dankclaw`.
|
||||
|
||||
## Facts that cost turns to re-derive
|
||||
|
||||
- Voice storage: `/home/dank/voices/<name>/` on Dankclaw (host state, never git; RO mount at `/voices`). Files: `reference.wav` (24 kHz mono, 18–25 s, existence gate), `reference.spk` (8192 B), `reference.rvq` (5456–6864 B), `manifest.json`.
|
||||
- Roster = facade directory scan; engine registry is in-memory, filled by `podman exec frank-tts-engine /app/warm-voices.sh` (boot `ExecStartPost`; re-register replaces, restart costs 6–8 s/voice).
|
||||
- Clients (omp-dankvoice, OpenWebUI) store NO voices — they GET `/v1/audio/voices`; only the chosen name persists client-side (`~/.omp/cache/dankvoice.json` `voice`).
|
||||
- Crop law: past first 10 s (hard invariant), 20–25 s preferred, floor 19.84 s for `.rvq`; sources pinned by sha256 in `scripts/voice-packs.env`; sample media live on workstations (`/home/blank/audiosample*`) and must be copied to `/home/dank/voice-sources` before a build.
|
||||
- Engine `GET /v1/audio/voices` returns `{}` at the pinned commit — always ask the 7772 facade, never 8880, for the list.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: frank-tts-voice-pack-audit
|
||||
description: Census live frank-tts voice packs against scripts/voice-packs.env to catch name-drift (renamed/retired packs) and untracked CLI-built packs; use after any voice add/rename when reconciling the builder recipe with what is actually on disk.
|
||||
---
|
||||
|
||||
# frank-tts voice-pack audit (builder ↔ live census)
|
||||
|
||||
Read-only census answering: *does `scripts/voice-packs.env` still describe the packs that actually exist, and is every live pack reproducible from the repo?* Run it after anyone adds or renames a voice. The runtime being healthy (all voices synthesize) does NOT mean the builder is honest — the gap is reproducibility, not serving.
|
||||
|
||||
## Why this exists
|
||||
Packs are host state under `/home/dank/voices` (never in git). A voice can be:
|
||||
- **built by the env** (a `VOICE_PACK_<NAME>_*` block exists), or
|
||||
- **built by CLI override** (`build-voice-pack.sh <name> <dir> --source --offset --duration --sha256` with NO env block) — then its only provenance is `/home/dank/voices/<name>/manifest.json`, and `--all` silently skips it.
|
||||
|
||||
Renaming a pack directory (e.g. `narrator1` → `wilwheaton`) preserves the crop bytes and its source sha256, so the env recipe stays *correct but keyed under a dead name*. `build-voice-pack.sh --all` then rebuilds the dead names and skips the live ones.
|
||||
|
||||
## Procedure (on Dankclaw)
|
||||
|
||||
1. **Live set + which builder made each** — the manifest `kind` field tells them apart:
|
||||
- `kind: "voice-pack"` → built by `scripts/build-voice-pack.sh` (env or CLI override).
|
||||
- `kind: "voice-reference-pack"` → the older trial `pipeline.sh`.
|
||||
```
|
||||
for d in /home/dank/voices/*/; do n=$(basename "$d"); [ -f "$d/reference.wav" ] \
|
||||
&& printf '%-16s %8sB %s\n' "$n" "$(stat -c%s "$d/reference.wav")" "$(stat -c %y "$d/reference.wav" | cut -d. -f1)"; done
|
||||
```
|
||||
2. **Per-voice provenance** — `manifest.json` is the sole record for CLI-built packs. Read `name/voice`, `kind`, `source.path`, `source.sha256`, `crop.offset_seconds`, `crop.duration_seconds`. Use `python3 -c 'import json,...'`, NOT a `jq '{a,b,c}'` object — the brace form gets brace-expanded by the remote zsh and errors.
|
||||
3. **Builder tracked list**:
|
||||
```
|
||||
grep -E '^VOICE_PACKS=|^VOICE_PACK_[A-Z0-9_]+_(SOURCE|OFFSET|DURATION|SHA256)=' /home/dank/serverAppConfigs/scripts/voice-packs.env
|
||||
```
|
||||
Diff the names in `VOICE_PACKS` against the live dirs. Three findings to report:
|
||||
- env keys that have **no live dir** (renamed/retired),
|
||||
- live dirs with **no env block** (CLI-built, unreproducible from repo),
|
||||
- matching source **sha256** between a dead env key and a live dir = that live pack is the renamed crop (safe to re-key the env, don't recompute).
|
||||
4. **Distinguish CLI-built from uncommitted env drift** — check the deployed env matches the repo:
|
||||
```
|
||||
cd /home/dank/serverAppConfigs && git diff --stat HEAD -- scripts/voice-packs.env # empty = no drift
|
||||
```
|
||||
Empty diff + a `kind: "voice-pack"` manifest with no env block ⇒ built via CLI overrides (correct, just untracked), NOT a rogue remote edit.
|
||||
|
||||
## Repoint the consumers (separate concern)
|
||||
If voices were renamed/retired, the runtime consumers still point at dead names — see the `frank-tts-repoint-voice-consumers` skill. Consumers that name a voice: `scripts/frank-tts.env` `DEFAULT_VOICE`, Open Notebook `dank-duo` speaker profile (`voice_id`, app-side REST), OpenWebUI `audio.tts.voice` (config DB). A voice-less request to frank-tts 400s naming `DEFAULT_VOICE`, so a dead default is a live break, not just cosmetic.
|
||||
|
||||
## Gotchas
|
||||
- **A published port is reachable at `host.containers.internal` too.** OpenWebUI's docs say use the LAN IP because that name "doesn't resolve on the bridge" — true only for *non-published* ports. frank-tts 7772 IS published, so `host.containers.internal:7772` answers 200 from the open-webui container. Verify before calling a URL the cause; the real break is usually the dead voice name.
|
||||
- **Source custody is split**: `audiosample*.mp3/m4b` live on the workstation (`/home/blank`), newer sources on Dankclaw (`/home/dank/voice-sources`). `--all` is only reproducible once each env SOURCE path points at where the file actually is.
|
||||
- Do NOT edit `voice-packs.env` to add CLI-built voices without confirming each source path + offset + sha against its manifest, then run `./tests/run.sh` (the pack-list invariant in `voice_pack_unit.bats` walks `VOICE_PACKS`).
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
name: franksign-demo-inspection-stack
|
||||
description: "Boot franksign's prod-shaped stack, seed all assignment lifecycle states through the real API, and open a headed inspection tour (platform + storybook) without squatting the e2e port."
|
||||
---
|
||||
|
||||
Visual-inspection recipe for the franksign repo (Dank boxes). Proven 2026-10-20; use when asked to "show/open the platform" or tour a new slice.
|
||||
|
||||
## Boot (do NOT squat :3100 — the e2e seam curls whoever owns that port)
|
||||
```bash
|
||||
node --env-file=.env apps/api/dist/cli.js # binds :3000 by default; use named service
|
||||
cd apps/web/storybook-static && python3 -m http.server 6007 # gate seam 12 already built it
|
||||
curl -s http://127.0.0.1:3000/health # {"status":"ok","db":"up"}
|
||||
```
|
||||
|
||||
## Seed lifecycle states through the real API (cookie jar, NOT bearer)
|
||||
better-auth sign-up returns a bearer `token`, but the org routes resolve sessions
|
||||
from **cookies only** (`Authorization: Bearer …` → get-session null → 401). Use
|
||||
`-c /tmp/jar` at sign-up and `-b /tmp/jar` after; state-changing calls need an
|
||||
`origin:` header. Keep auth headers out of argv via `curl -K <file>`.
|
||||
|
||||
1. `POST /api/auth/sign-up/email` `{"email","password"(>=8 chars),"name"}` → save cookies (`-c`)
|
||||
2. `POST /api/orgs` `{"name"}` → `org.id`
|
||||
3. `POST /api/orgs/:org/templates` `{"name","baseTemplateId":"sample"}` → `template.id`
|
||||
4. `POST /api/orgs/:org/templates/:tpl/assignments` `{"email"}` → `{token, signUrl}` (repeat per demo signer)
|
||||
5. signed state: `POST /api/releases/:token` `{"values":{"name":…,"date":…,"agreement":true},"signatureName":…}` — no auth needed (token seam)
|
||||
6. voided state: `POST /api/orgs/:org/assignments/:assignmentId/void` (cookie session)
|
||||
Remaining pending assignment → three inspectable pages: `/app/sign/:token` renders form / already-signed / cancelled.
|
||||
|
||||
## Byte-identity spot check (reconstruction contract, issue #18)
|
||||
Fetch `GET /api/releases/:token/pdf` twice with `sleep 1.5` **between** fetches
|
||||
and sha256-compare, plus the org door `GET /api/orgs/:org/assignments/:aid/pdf`
|
||||
(cookie jar). Same-second-only comparisons are blind to clock leaks
|
||||
(pdf-lib's `PDFDocument.load` restamps `/ModDate` unless `updateMetadata:false`).
|
||||
To see printed text: inflate the content streams and unescape `\(`, `\)` —
|
||||
pdf-lib writes hex-string `Tj` operands.
|
||||
|
||||
## Inspection browser (managed Chromium via browser eval)
|
||||
- `viewport` must be `{width, height}` — string fails
|
||||
- `tab.waitFor(ms)` does NOT exist (ms is parsed as selector) → `await Bun.sleep(900)`
|
||||
- open with `headed: true, persist: true`; window lands on the desktop — confirm with `niri msg windows | grep -i chrome`
|
||||
|
||||
## Teardown (hard gate hazard)
|
||||
A live demo API on :3000 **hangs the full `tests/run.sh` gate for ~10 minutes**
|
||||
(observed 2026-10-21: >600s kill with the service up; 14/14 in ~15s without
|
||||
it). Always `proc://franksign-demo-api/kill` BEFORE running the gate; verify
|
||||
`pgrep -f apps/api/dist/cli.js` is empty so the next `tests/run.sh` e2e seam
|
||||
owns :3100 honestly.
|
||||
|
||||
## Adjacent Gitea quirk (same fleet)
|
||||
Closing an issue/PR: `PATCH {"state":"closed"}` → 201; `POST` → 405. Merging needs form-encoded lowercase `do=merge` (see gitea-merge-form-encoded). Label assignment: `PATCH /issues/N` with `labels:[id]` silently clears labels — use `POST /issues/N/labels` `{"labels":[id]}`.
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
name: franksign-worktree-db-cred-trap
|
||||
description: "Diagnose franksign e2e seam 5 / health 503 db:down in a fresh worktree: setup-dev.sh minted a password that mismatches the shared franksign-db container."
|
||||
---
|
||||
|
||||
# franksign fresh-worktree DB credential trap
|
||||
|
||||
## Symptom
|
||||
`tests/run.sh` in a fresh franksign worktree fails seam 5 ("e2e server did not
|
||||
become healthy") while the server log says `franksign-api listening on :3100`.
|
||||
Manual probe shows the server IS up:
|
||||
|
||||
```bash
|
||||
PORT=3100 node --env-file=.env apps/api/dist/cli.js >/tmp/manual-e2e.log 2>&1 &
|
||||
sleep 2; curl -s http://127.0.0.1:3100/health
|
||||
# -> 503 {"status":"degraded","db":"down",...}
|
||||
```
|
||||
|
||||
Not a regression, not a :3100 port collision (check `ss -tlnp | grep 3100` to rule that out).
|
||||
|
||||
## Root cause
|
||||
`infra/setup-dev.sh` mints a fresh `POSTGRES_PASSWORD` per checkout
|
||||
(`if [ ! -f infra/postgres.env ]`), but `franksign-db` + `franksign-pgdata`
|
||||
are shared box state owned by the main checkout
|
||||
(`~/Projects/franksign`). The new worktree's `.env` DATABASE_URL then fails
|
||||
TCP auth against the initialized volume.
|
||||
|
||||
The setup script masks it: its readiness gate is
|
||||
`podman exec franksign-db pg_isready`, which uses the in-container local
|
||||
socket and passes regardless of password — so setup prints "postgres ready"
|
||||
while the checkout is dead on arrival.
|
||||
|
||||
## Fix (git-ignored files, no commit needed)
|
||||
Copy the shared credentials from the owning checkout:
|
||||
|
||||
```bash
|
||||
grep DATABASE_URL /home/blank/Projects/franksign/.env # source of truth
|
||||
# PUT that exact DATABASE_URL line into this worktree's .env
|
||||
# PUT the matching POSTGRES_PASSWORD into this worktree's infra/postgres.env
|
||||
```
|
||||
|
||||
Gotcha: the shared password may contain literal `$$`. Keep it byte-for-byte —
|
||||
systemd `EnvironmentFile` and `node --env-file` both read values literally,
|
||||
so `$$` in the file IS `$$` in the password. No escaping, no de-escaping.
|
||||
|
||||
## Verify
|
||||
```bash
|
||||
PORT=3100 node --env-file=.env apps/api/dist/cli.js & # then
|
||||
curl -fsS http://127.0.0.1:3100/health # -> {"status":"ok","db":"up"}
|
||||
tests/run.sh # 14/14
|
||||
```
|
||||
|
||||
## Permanent fix (proposed, not yet landed)
|
||||
`setup-dev.sh` should reuse `~/.config/containers/systemd/postgres.env`
|
||||
(symlink to the owning checkout) when present instead of minting new creds.
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
name: gguf-lora-adapter-fit-census
|
||||
description: "Read-only census before applying a GGUF LoRA/adapter to a llama.cpp (or fork) endpoint: Hub metadata, declared base provenance, runtime flag support, and the behavioural probe that proves the effect."
|
||||
---
|
||||
|
||||
# GGUF LoRA / adapter fit census (read-only, before you promise a file to a payload)
|
||||
|
||||
A 3 MB adapter is a payload change: it edits the compute graph at load. Do the census
|
||||
before editing an endpoint's trio, and prove the EFFECT behaviourally — no metadata,
|
||||
card table, or health gate can see it.
|
||||
|
||||
## 1. Census the Hub repo first (no auth for public repos)
|
||||
|
||||
```bash
|
||||
curl -sS "https://huggingface.co/api/models/<org>/<repo>?blobs=true" | python3 -c "
|
||||
import json,sys; d=json.load(sys.stdin)
|
||||
print(d.get('pipeline_tag'), d.get('library_name'))
|
||||
print([t for t in d.get('tags',[])][:20]) # base_model / base_model:adapter rows matter
|
||||
for s in d['siblings']: print(s['rfilename'], s.get('size'), (s.get('lfs') or {}).get('oid'))"
|
||||
curl -sS "https://huggingface.co/<org>/<repo>/raw/main/README.md" > /tmp/card.md # grep for --lora flags
|
||||
```
|
||||
|
||||
- `base_model` / `base_model_relation: adapter` name the pack the adapter was built for.
|
||||
- **A file with `lfs.oid = None` is a plain git object**: pin the sha256 of the Hub's bytes,
|
||||
not an LFS oid. Fetcher law still applies: `curl -fsSL --fail-with-body`, byte count,
|
||||
sha256, `--verify-only`, no resume.
|
||||
|
||||
## 2. Read the adapter's own GGUF KV section (settles provenance in one read)
|
||||
|
||||
Small python read of the KV pairs — no torch, no download beyond the adapter itself:
|
||||
magic `GGUF`, `u32 ver`, `u64 n_tensors`, `u64 n_kv`; key = `u64 len` + bytes; type `u32`
|
||||
(0 u8,1 i8,2 u16,3 i16,4 u32,5 i32,6 f32,7 bool,8 str,9 array,10 u64,11 i64,12 f64);
|
||||
array = `u32 elem_type` + `u64 count`.
|
||||
|
||||
Keys worth printing: `general.type` (`adapter`), `adapter.type` (`lora`),
|
||||
`adapter.lora_alpha`, and whatever provenance the publisher wrote — e.g. PrismML/abliteration
|
||||
packs carry `prism.abliterate.base_gguf`, `base_sha256`, `direction_row`, `target_layers`,
|
||||
`target_embedding`, `sites`, `basis`.
|
||||
|
||||
**Provenance is metadata, not a gate.** The loader matches **tensor names, not hashes**, so an
|
||||
adapter fit against a sibling quantization of the same model still applies — which is a
|
||||
decision to surface to the operator, not a failure to hide. Compare `base_sha256` against the
|
||||
sha the endpoint's own fetcher pins; if they differ, say so in the PR body, the env header and
|
||||
the endpoint page, and prove the effect behaviourally instead.
|
||||
|
||||
Two publisher facts that decide whether a fork can carry the file at all: the build it was
|
||||
verified on, and what it deliberately does NOT touch (an embedding rank is the usual crash —
|
||||
`token_embd` left alone is a graph constraint, not an omission).
|
||||
|
||||
## 3. Ask the runtime binary, not your memory
|
||||
|
||||
```bash
|
||||
<llama-server> --help | grep -iE "lora|adapter"
|
||||
```
|
||||
Expect `--lora FNAME`, `--lora-scaled FNAME:SCALE,...`, `--lora-init-without-apply`
|
||||
(+ `POST /lora-adapters` for runtime scale). Prefer `--lora-scaled <file>:<scale>` as ONE
|
||||
argv token so a scale can never reach the engine without its file, and follow the endpoint's
|
||||
existing opt-out convention (the `MMPROJ_FILE=` shape: `:`-less default so an explicitly empty
|
||||
value means "off, on purpose"). Gate the deployer on the file's existence with the adapter's
|
||||
OWN fetcher named in the error text.
|
||||
|
||||
## 4. Prove the effect: refusal/behaviour probe, before AND after
|
||||
|
||||
The probe is not a bench, so the no-bench law does not block it — and the **before** half must
|
||||
be captured while the old shape is still live.
|
||||
|
||||
- Same prompts, `temperature` low, both before and after; classify with a refusal-marker regex
|
||||
and print only the opening characters, never the generated body.
|
||||
- **Budget ≥ 2 048 tokens on a thinking model.** A 600-token request can be swallowed whole by
|
||||
the reasoning channel: `content` comes back empty with `finish_reason: length` and
|
||||
`reasoning_content` full — which reads exactly like a refusal that never happened.
|
||||
- A leading `⚠️ Disclaimer`/hedge after the adapter is style, not the circuit: judge the marker
|
||||
sentence, not the tone.
|
||||
|
||||
## 5. Post-deploy reads (the ledger is the instrument)
|
||||
|
||||
- Boot log shows `llama_adapter_lora_init_impl: loading lora adapter from '<file>'` and then
|
||||
**dumps the adapter's KV block** — the provenance mismatch is printed at every start, so the
|
||||
runbook should point at the journal instead of restating it.
|
||||
- Confirm the flag on argv: `tr '\0' ' ' < /proc/$(systemctl show <svc> -p MainPID --value)/cmdline`;
|
||||
confirm `systemctl show <svc> -p DropInPaths --value` is empty (a leftover bench drop-in outranks
|
||||
everything you just deployed).
|
||||
- Shape must be untouched by an adapter: `n_slots` / `n_ctx_slot` / KV lines unchanged, and
|
||||
`/metrics` still answering (a llama.cpp endpoint is invisible to Prometheus without `--metrics`).
|
||||
- Root units log to the journal; rootless podman storage is read WITHOUT sudo (`sudo podman
|
||||
inspect` sees a different storage namespace and reports "no such object").
|
||||
|
||||
## Yields
|
||||
|
||||
One sentence per question: does the file exist and at what size/sha; which pack it declares as
|
||||
its base and whether that matches the served payload; which flag the runtime actually offers;
|
||||
what the before/after probe classified. Anything else is a decision for the operator, and the
|
||||
rollback should be one env line on a file the endpoint already loads.
|
||||
@@ -19,6 +19,8 @@ for p in (home/".config/secrets").iterdir():
|
||||
if m: vals.add(m.group(1))
|
||||
known[p.name] = {v for v in vals if len(v) >= 12}
|
||||
```
|
||||
Also add every secret the repo itself embeds (e.g. credentials living in a *test helper*)
|
||||
by concatenating its quoted fragments, then search for the joined value.
|
||||
Report ONLY `value[:4] + len + sha256[:10]`. Never print a value, and never quote a
|
||||
secret-shaped literal in a doc/commit message — the pre-commit gate will (correctly) block it.
|
||||
|
||||
@@ -41,6 +43,19 @@ git status --porcelain -uall --ignored=matching # !! lines: on-disk ignored fi
|
||||
git config --list --show-origin # local/global/system credentials
|
||||
git ls-files | grep -E '(auth\.json|trust\.json|\.env|\.pem|\.key|identity)' # quarantine
|
||||
```
|
||||
**Invocation forms that actually work** (each of these cost a wasted round-trip):
|
||||
- `git grep --all -F '<fragment>'` — the options MUST precede the pattern; `git grep -l -F '<frag>' --all`
|
||||
treats `--all` as a pathspec and silently returns 0 lines. Grepping a *fragment* (not the full
|
||||
value) is what catches split-concatenation disguises like `"Assf.""1234"`.
|
||||
- `git ls-tree -r --format=... <rev>` works for ONE rev; pass several revs and it emits nothing,
|
||||
no error. For sha→path across all refs use `git rev-list --objects --all`, which yields
|
||||
`"<sha> <path>"` lines directly.
|
||||
- `git cat-file --batch` fed with `rev-list --objects` lines echoes `sha path` headers, so naive
|
||||
header parsing misaligns and content is lost. Feed bare SHAs in chunks, or use
|
||||
`git cat-file --batch-all-objects --batch` and parse `"<sha> blob <size>\n<content>\n"` records.
|
||||
- Check every worktree/checkout: `git -C <other-checkout> rev-parse HEAD` often differs from the
|
||||
audited worktree, so the leak may exist on `main` while the audited tree looks clean.
|
||||
|
||||
LFS: scan `.git/lfs/objects/**` as BYTES (`decode("latin-1")`) — a screenshot is a classic
|
||||
secret carrier and the tracked tree holds only pointers.
|
||||
|
||||
@@ -53,6 +68,9 @@ Flag every one whose value is in `known` as a LIVE leak; list the rest for triag
|
||||
- prose: `Set password: user.set_password(...)`, `authenticate(user, password)`
|
||||
- 40-hex hits that are git object ids / packfile names (`pack-<40hex>.pack`)
|
||||
- memory/KB records that *describe* a secret operation
|
||||
- length-threshold gates that miss short passwords: a check like
|
||||
`len(val) >= 24 and re.fullmatch(...)` clears a real 12-char password, so a passing
|
||||
secret-scan test proves nothing about short credentials
|
||||
A hit is a leak only when the value matches a live secret, decodes to a real credential, or
|
||||
sits in a field a program reads.
|
||||
|
||||
@@ -61,8 +79,12 @@ Check file modes of the secret dir (`0600`, dir `0700`) — world-readable secre
|
||||
most common real finding. Then grep the live values across shell rc files, `~/.gitconfig`,
|
||||
`~/.npmrc`, `~/.netrc`, `~/.curlrc`, `~/.docker/config.json`, `~/.cache/huggingface/token`,
|
||||
`~/.aws/credentials`, `~/.ssh/config` and report filenames only.
|
||||
Also check files the deployer *generates* on hosts: a script that writes a credential file
|
||||
without `chmod 600` (while sibling writes in the same script do chmod) is the same class.
|
||||
|
||||
## 6. Report
|
||||
Per layer: scope numbers + verdict. Every hit: path:line, redacted value, verdict
|
||||
(leak / benign + why). Name the gate's coverage gap if the project's own hook misses shapes
|
||||
you matched. Fix modes/chmod on the spot; changes to a security hook are the operator's call.
|
||||
A secret in git history is only removable by rotation or history rewrite — say so, don't
|
||||
imply an edit fixes it.
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
name: git-secret-gate-hook
|
||||
description: "Write a fail-closed pre-commit secret-gate hook (staged-content scan) with the grep/ERE traps that kill naive versions, plus its bats fixture test."
|
||||
---
|
||||
|
||||
# Git secret-gate pre-commit hook
|
||||
|
||||
Use when a repo promises a "pre-commit secret gate" but none exists, or when porting the gate to another dank repo. Ships with three traps already paid for.
|
||||
|
||||
## Shape
|
||||
|
||||
- Track the hook at `hooks/pre-commit` (chmod +x). Git never self-installs hooks — wire per clone:
|
||||
`git config core.hooksPath hooks` — run it in the MAIN checkout once; `core.hooksPath` lives in the
|
||||
**common** git dir config, so every linked worktree inherits it.
|
||||
- Do NOT put `hooks` in `package.json` `files` — an installed plugin under node_modules never runs git.
|
||||
- Scan **staged** content (`git show ":$f"`), not the working tree: that is what lands in history.
|
||||
- Denylist by path (README, docs/, tests/, `.env.example`) — files whose job is naming secrets or holding
|
||||
fixture dummies produce theatre, not safety.
|
||||
- Never echo the matched secret; print file + line number only.
|
||||
- Red-first test: fixture repo + `cp` the real hook in + `git config core.hooksPath hooks`, then drive
|
||||
REAL `git commit` attempts: credential-shaped file must exit non-zero AND `git log` must show no new
|
||||
commit (block, not warn); clean commit passes; hook must survive committing itself.
|
||||
|
||||
## The three traps (each cost a red test)
|
||||
|
||||
1. **`--` is a ONE-SHOT end-of-options marker.** `grep -nE -- 'p1' -- 'p2'` treats everything after the
|
||||
first `--` as FILE NAMES: `grep: --: No such file or directory`, rc=2. Multi-pattern = repeat `-e`:
|
||||
`grep -nE -e 'p1' -e 'p2'`.
|
||||
2. **`if hits=$(grep ...)` is fail-OPEN on rc>=2** (rc=2 is truthy, hits empty). Capture `rc=$?`
|
||||
explicitly: `rc>=2` must BLOCK with "pattern error, refusing to pass unchecked" — a gate whose regex
|
||||
silently dies is worse than none.
|
||||
3. **POSIX ERE (`grep -E`) gotchas:** no `(?:...)` groups (use plain `(...)`); a `[:-]` inside a bracket
|
||||
starts a collating element `[:` and dies with "repetition-operator operand invalid" — put `-` FIRST
|
||||
in every class (`[-A-Za-z0-9/+=_.:+]`). Also `-----BEGIN` literals: prefer printf-safe fixtures in tests.
|
||||
|
||||
## Pattern set (shapes, not entropy — say so in the header comment)
|
||||
|
||||
```
|
||||
-----BEGIN[ A-Z0-9]*(OPENSSH|RSA|DSA|EC|PGP)? ?PRIVATE KEY-----
|
||||
AKIA[0-9A-Z]{16}
|
||||
gh[pousr]_[A-Za-z0-9]{36,}|github_pat_[A-Za-z0-9_]{50,}
|
||||
xox[baprs]-[A-Za-z0-9-]{10,}
|
||||
https://hooks\.slack\.com/services/[A-Za-z0-9/]+
|
||||
(aws_secret_access_key|secret(_access)?_key|(api|access|auth)[_-]?key|[a-z_]*token|passwd|password|credential)[[:space:]"']?[[:space:]]*[:=][[:space:]]*["']?[-A-Za-z0-9/+=_.:+]{16,}
|
||||
(Bearer|token|authorization)[[:space:]]*[:=][[:space:]]*["']?[-A-Za-z0-9._+/]{20,}
|
||||
```
|
||||
|
||||
## Test fixture hygiene
|
||||
|
||||
Fake secrets must be inert (RFC 2606 domains, all-zero ids, `AKIA0000000000000EXAMPLE`) AND must not
|
||||
match the hook's own self-scan. Single quotes in ERE patterns: write `'"'"'`.
|
||||
|
||||
## Wiring gotcha (dank worktree policy)
|
||||
|
||||
Set `core.hooksPath` AFTER the merge that lands `hooks/` into main — with the config set but the path
|
||||
missing, git fails every commit in worktrees lacking `hooks/` ("cannot run hooks/pre-commit: No such
|
||||
file or directory"). Order: merge -> land -> `git config core.hooksPath hooks` from main checkout.
|
||||
|
||||
Reference implementation: `dank/dankvoice` `hooks/pre-commit` (PR #19), pinned by `tests/gate.bats`.
|
||||
@@ -1,105 +1,85 @@
|
||||
---
|
||||
name: gitea-api-pr-operations
|
||||
description: "Open, list, merge, and verify pull requests (and diagnose issue-API scope failures) on the Dankclaw Gitea via its REST API from a workstation, keeping the token out of argv (curl -K config file, mode 600) — includes the credential-store extraction one-liner, the merge endpoint's Do-field law and Content-Type-header trap, the suggested-URL-is-not-a-PR gotcha, and the insufficient-token-scope trap."
|
||||
description: "\"Open, list, merge, and diagnose issues/PRs and manage labels on the Dankclaw Gitea API (192.168.0.143:3000) with the token stored in ~/.config/secrets/gitea — incl. the token-only write auth (Basic reads but 422s on POST/PATCH), the required merge payload, where review comments actually live on 1.27.3, and how to recover the session id for the mandatory body footer.\""
|
||||
---
|
||||
|
||||
# Gitea API PR operations (Dankclaw)
|
||||
# Gitea API operations (Dankclaw `dank/command-center`, `dank/serverAppConfigs`)
|
||||
|
||||
`git push` on this fleet prints `Create a new pull request for ...` — that is a
|
||||
**suggestion, not a PR**. Verify (`GET /pulls?state=open` → expect `[]`) before
|
||||
claiming a PR "should exist"; create it via the API when asked.
|
||||
## Where
|
||||
- Token: `~/.config/secrets/gitea` (`http://user:TOKEN@192.168.0.143` single line — git-credential URL form, NOT `username=`/`password=` lines).
|
||||
Extract WITHOUT echoing:
|
||||
`tok=$(sed -nE 's#^http://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)`
|
||||
- Build a 600-perm temp curl config and use `curl -K`:
|
||||
`cfg=$(mktemp) && chmod 600 "$cfg" && printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"`
|
||||
Delete `$cfg` after use. NEVER pass `-H "Authorization: token $tok"` (the literal
|
||||
header lands in transcript + logs). NEVER echo the token.
|
||||
- Base URL: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>`.
|
||||
|
||||
## Endpoint + auth
|
||||
- API base: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>`
|
||||
- Token lives in the git credential store configured by
|
||||
`git config --get credential.helper` → `store --file ~/.config/secrets/gitea`,
|
||||
line format `http://user:TOKEN@192.168.0.143:3000`.
|
||||
- The repo-local `.env` `GITEA_TOKEN` (main checkout, gitignored) carries
|
||||
**`read:issue` too** (measured 2026-10-18): filing issues needs THAT token,
|
||||
the credential-store one 403s on issues.
|
||||
## Auth: reads tolerate Basic, writes DO NOT (measured 2026-10-20)
|
||||
GETs succeed with an HTTP Basic header built from the same credential, but every
|
||||
POST/PATCH/DELETE with Basic returns `407/422 {"message":"token is required"}`.
|
||||
Always use the `Authorization: token <tok>` header for everything — never debug
|
||||
"permissions" when the header shape is the fault.
|
||||
|
||||
Extract the token (the `http://` scheme contains a colon — scheme-stripping
|
||||
awk regexes silently grab the port instead; use this anchored sed):
|
||||
## Recipes (all verified live)
|
||||
- **Open PR** (token only needed for private repos; this one reads publicly but
|
||||
POST needs auth): `POST /pulls` JSON `{title, head, base:"main", body}`.
|
||||
- **PATCH PR title/body**: `PATCH /pulls/<n>` with JSON body `{title}` and/or
|
||||
`{body}`. Returns 201 with the full PR. Use `jq -n --arg` to build JSON; for
|
||||
multi-line bodies write the text to a temp file and `--arg b "$(cat b.txt)"`.
|
||||
(jq CLI: a bare `---` argument after the program is parsed as a flag → error;
|
||||
fold footers into the --arg string instead.)
|
||||
- **List open PRs**: `GET /pulls?state=open` → `jq -r '.[] | "#\(.number) \(.head.ref) \(.title)"'`.
|
||||
- **Read PR**: `GET /pulls/<n>` (no token; public repo) → `jq -c '{state,merged,mergeable,head:.head.ref}'`.
|
||||
`mergeable:false` is often Gitea's *not-yet-checked* state, not a conflict — prove a real
|
||||
conflict locally with `git merge-tree --write-tree HEAD origin/main` before believing it.
|
||||
- **MERGE**: `POST /pulls/<n>/merge` with `-H 'Content-Type: application/json'`
|
||||
and body **`{"Do":"merge"}`** — this build REQUIRES the `Do` field:
|
||||
`{}` or `{"DoNotMerge":false,...}` → 422; `{"Do":"merge"}` → 200 +
|
||||
`{"state":"closed","merged":true,"merge_commit_sha":"..."}`. Verify with a plain
|
||||
GET /pulls/<n> afterwards (merge endpoint's own stdout may not parse).
|
||||
Merge is the `--no-ff` merge commit on main; then `git -C ~/.omp fetch && git
|
||||
merge --ff-only origin/main` propagates the live deployment.
|
||||
- **Issues**: same shape at `/issues` (`POST {title, body}`), `/issues/<n>/comments`
|
||||
(`POST {body}`, `PATCH .../comments/<id>`) — both work with the temp-cfg auth.
|
||||
- **Labels**: `GET /labels` (list incl. numeric `id`); `POST /labels`
|
||||
`{"name":"<name>","color":"#rrggbb"}` to create; attach by ID with
|
||||
`POST /issues/<n>/labels {"labels":[<id>,...]}` (names are rejected);
|
||||
detach with `DELETE /issues/<n>/labels/<id>`. serverAppConfigs gained
|
||||
needs-triage / needs-info / ready-for-agent / ready-for-human / wontfix
|
||||
(2026-10-20) alongside the older `human-only`.
|
||||
|
||||
```bash
|
||||
tok=$(sed -nE 's#^http://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
[ -n "$tok" ] && echo "token-extracted(len=${#tok})" # length check, never echo the value
|
||||
```
|
||||
## Review comments: the API lies on 1.27.3 (learned 2026-10-18)
|
||||
- `GET /pulls/<n>/comments` (inline review comments) → **404 `not found` on every
|
||||
route** (token or not, `?pull_request_id=` or not) even when a review with
|
||||
comments exists.
|
||||
- `GET /pulls/<n>/reviews` works (shows `state`, `comments_count`) but bodies are empty.
|
||||
- **The comments are readable from the PR HTML**:
|
||||
`curl -sS http://192.168.0.143:3000/dank/command-center/pulls/<n> -o /tmp/pr.html`
|
||||
then extract `<div class="content comment-content">` blocks
|
||||
(python: `re.finditer(r'comment-content', src)` → widen ±800 chars, strip tags).
|
||||
Duplicates appear (suggested-change UI shows text twice); dedupe mentally.
|
||||
- Per-conversation resolve API also 404s — flag resolutions in an issue comment instead.
|
||||
|
||||
## Keep the token out of argv (repo secret law)
|
||||
`curl -H "Authorization: token $tok"` is world-readable in `/proc/<pid>/cmdline`.
|
||||
Use a mode-600 config file:
|
||||
## Body-footer law (AGENTS.md)
|
||||
Every issue/PR body ends with `---` line then `cwd: <wd> · session: <id>`. Keep
|
||||
it inside the --arg string (not a `---` CLI arg).
|
||||
|
||||
```bash
|
||||
cfg=$(mktemp); chmod 600 "$cfg"
|
||||
printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||
```
|
||||
`cwd` is THIS session's own working directory (note a build worktree separately if they
|
||||
differ). When `/status` is unreachable (headless/subagent context), recover the session id
|
||||
from the session store instead of inventing one:
|
||||
`ls -t ~/.omp/agent/sessions/<cwd-slug>/ | head -2` → `<timestamp>_<uuid>` (the same uuid as
|
||||
`<uuid>.jsonl`); the slug is the cwd with `/` replaced by `-`. Subagent files live under the
|
||||
**parent** session's directory, so the newest entry there is the parent id.
|
||||
|
||||
- POST JSON needs a second header line: `header = "Content-Type: application/json"`.
|
||||
There IS NO `content-type` config option — curl rejects the file.
|
||||
- `rm -f "$cfg"` when done (a mode-600 token file left in /tmp is its own leak).
|
||||
|
||||
## Operations
|
||||
```bash
|
||||
# list open PRs
|
||||
curl -sS -K "$cfg" ".../pulls?state=open" | jq -r '.[] | [.number,.head.name,.base.name,.title] | @tsv'
|
||||
# create (build the payload as a FILE, see below)
|
||||
curl -sS -K "$cfg" -o /tmp/pr.json -w '%{http_code}\n' -X POST -d @/tmp/pr-payload.json ".../pulls"
|
||||
# verify: 201 is not enough — check mergeable
|
||||
curl -sS -K "$cfg" ".../pulls/1" | jq '{number,state,merged,mergeable,base:.base.ref,head:.head.ref,url:.html_url}'
|
||||
```
|
||||
|
||||
## Merging: the field is `Do`, and it is a STRING naming the strategy
|
||||
|
||||
Measured 2026-10-18 on Gitea at 192.168.0.143:3000 (`POST .../pulls/<n>/merge`).
|
||||
`MergePullRequestOption` is `{From, To, When, Do}` — **there is no `DoMerge` and
|
||||
no `merge_method`**. Every wrong shape is a 422 with a distinct message, so read it:
|
||||
|
||||
|body sent|http|message|
|
||||
|---|---|---|
|
||||
|`{"DoMerge":true,"merge_method":"merge"}`|422|`[Do]: Required`|
|
||||
|`{"Do":true,"merge_method":"merge"}`|422|`[]: json: cannot unmarshal JSON boolean into Go string within "/Do"`|
|
||||
|`{"Do":"true"}`|422|`[Do]: In…` (validation: not a strategy name)|
|
||||
|form body `Do=true&merge_method=merge`|422|`[Do]: Required` (the value must be a strategy name)|
|
||||
|`{"Do":"merge"}`|**200**|merged|
|
||||
|
||||
```bash
|
||||
curl -sS -K "$cfg" -H 'Content-Type: application/json' -X POST -d '{"Do":"merge"}' ".../pulls/4/merge"
|
||||
# then prove it: .merged true, .state closed, .merge_commit_sha set, and origin/main moved
|
||||
curl -sS ".../pulls/4" | jq -c '{number,state,merged,merge_commit_sha}'
|
||||
```
|
||||
|
||||
Strategy names: `merge`, `rebase`, `rebase-merge`, `squash`, `manually` — a
|
||||
strategy the repo does not allow is rejected too.
|
||||
|
||||
**Merge trap (measured 2026-10-18):** on `POST /merge` the `Content-Type` header
|
||||
must go on the **command line** (`-H 'Content-Type: application/json'`). Same
|
||||
curl that succeeds on `POST /pulls` with Content-Type supplied only via the `-K`
|
||||
config file returns **405 with an EMPTY body** on `/merge`. A 405 there is this
|
||||
trap (or a base-moved mergeability recheck — GET the PR, see `mergeable:true`,
|
||||
then retry with `-H`), NOT a permission problem.
|
||||
|
||||
## Building issue/PR bodies without quote carnage
|
||||
Hand-written `jq -n --arg t "..." '{title:$t, body:"..."}'` one-liners with long
|
||||
markdown bodies WILL break on a stray quote (and the broken quoting can swallow
|
||||
the following commands). Build bodies as files (heredoc or write tool), then:
|
||||
```bash
|
||||
b=$(jq -Rs . body.md)
|
||||
jq -n --arg t "$title" --argjson b "$b" '{title:$t, body:$b, head:"<branch>", base:"main"}' > payload.json
|
||||
curl ... -d @payload.json
|
||||
```
|
||||
|
||||
## Gotchas measured 2026-10-14 / 2026-10-17 / 2026-10-18
|
||||
- Unauthenticated/bad token → HTTP 401 body `{"message":"not found"}`; `jq '.[]|[.number,..]'` then fails with *cannot index "not found"* — that is an auth failure, not a missing repo.
|
||||
- **Insufficient token scope ≠ missing data.** The credential-store token
|
||||
(measured 2026-10-17) carries only `write:repository,write:user`; the issues
|
||||
API answers HTTP 403 `{"message":"token does not have at least one of required
|
||||
scope(s), required=[read:issue], token scope=write:repository,write:user"}`.
|
||||
jq surfaces that JSON as the *cannot index* error — do NOT conclude the issue
|
||||
list is empty, and note POSTs under this token create NOTHING (verify, don't
|
||||
assume retry safety). Filing issues needs a fresh token minted in the Gitea
|
||||
web UI with `read:issue`+`write:issue` (Frank-class credential action; ask) —
|
||||
OR use the main-checkout `.env` `GITEA_TOKEN`, which already has issue scope.
|
||||
PR endpoints work fine with either token (`pulls` POST/GET/merge all 200/201).
|
||||
- On the created-PR response `.head.name` can be null; verify with the `GET /pulls/<n>` and use `.head.ref`.
|
||||
- **Swagger JSON is disabled on this instance** — `/api/swagger.v1.json`, `/api/swagger.json` and `/api/v1/swagger.json` all 404. Do not go looking for the schema there; the 422 messages name the field and its Go type, which is the faster instrument.
|
||||
## Gotchas
|
||||
- Content-Type: POSTs with `-d @file.json` MUST carry
|
||||
`-H 'Content-Type: application/json'` on the command line. `-H` entries put in
|
||||
the curl config file are ignored for this (header ordering quirk).
|
||||
- `xdg-open <pr-url>` lands the PR in Frank's Firefox the moment it is opened —
|
||||
law, not courtesy.
|
||||
- `jq: cannot index "not found" with "user"` = endpoint 404'd into an error
|
||||
object; check `-w '%{http_code}'` before blaming jq.
|
||||
- The token in `.git/config`'s remote URL is the same secret; the credential
|
||||
file (`~/.config/secrets/gitea`) is the one git actually consults (global
|
||||
credential.helper) and the one to reuse for API calls.
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: gitea-credential-store-url-format
|
||||
description: "Auth to the Dankclaw Gitea API without the empty-header trap: ~/.config/secrets/gitea is a git-credential-store URL file, so awk '/^password=/' returns empty; resolve the token via git credential fill, and Gitea writes need Authorization: token (reads pass anonymously on a public repo)."
|
||||
---
|
||||
|
||||
# Gitea auth on Dankclaw: the URL-format credential store
|
||||
|
||||
`~/.config/secrets/gitea` is `git config credential.helper = store --file ~/.config/secrets/gitea`, i.e. a **git-credential-store URL file**:
|
||||
|
||||
```
|
||||
http://dank:<40hex-token>@192.168.0.143:3000
|
||||
```
|
||||
|
||||
It does NOT contain `username=`/`password=` lines. So:
|
||||
|
||||
```bash
|
||||
# WRONG — silently returns empty string
|
||||
p=$(awk -F= '/^password=/{print substr($0,10)}' ~/.config/secrets/gitea)
|
||||
```
|
||||
|
||||
An empty `$p` makes every curl send `Authorization: token ` (blank) → Gitea replies `{"message":"token is required"}` on **writes**, while **reads still return 200** (the repo is public-readable, so a green `GET /labels` or `/issues` proves nothing about auth). Symptom: PR-create returns `token is required` even though a `GET` just "worked".
|
||||
|
||||
## Resolve the real token via git
|
||||
|
||||
```bash
|
||||
p=$(printf 'protocol=http\nhost=192.168.0.143:3000\n\n' \
|
||||
| git credential fill 2>/dev/null \
|
||||
| awk -F= '/^password=/{print substr($0,10)}') # 40-hex token
|
||||
```
|
||||
|
||||
## Write auth = token scheme, via a 600-perm -K file
|
||||
|
||||
```bash
|
||||
cfg=$(mktemp); chmod 600 "$cfg"
|
||||
printf 'header = "Authorization: token %s"\n' "$p" > "$cfg"
|
||||
curl -s -K "$cfg" http://192.168.0.143:3000/api/v1/user # confirms identity
|
||||
curl -s -K "$cfg" -X POST -H 'Content-Type: application/json' --data @body.json "$API/pulls"
|
||||
```
|
||||
|
||||
- Never pass the token with `-H` — it lands in `/proc/<pid>/cmdline`, world-readable.
|
||||
- Confirm identity with `GET /user` before trusting a write path; a `token is required` there means the token never authenticated.
|
||||
- Merges are a separate shape (form-encoded `do=merge`, auth-only cfg, NO `Content-Type` header) — see the gitea-merge-form-encoded skill.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
name: gitea-issue-labels-and-pr-curl-traps
|
||||
description: "Dankclaw Gitea API curl traps: PR create needs Content-Type application/json (fake \"[Head]: Required\"), issue labels only via POST /issues/N/labels (PATCH labels silently dropped), token from URL-format secret file."
|
||||
---
|
||||
|
||||
Operate Dankclaw Gitea (`http://192.168.0.143:3000`, repo `dank/franksign` and siblings) with curl without the three traps that produce misleading errors.
|
||||
|
||||
## Token extraction (URL-format store, NOT key=value)
|
||||
|
||||
`~/.config/secrets/gitea` is one line: `http://user:TOKEN@192.168.0.143:3000`. `cat`-ing it yields the whole URL, not the token.
|
||||
|
||||
```bash
|
||||
TOK=$(sed -n 's|.*://[^:]*:\([^@]*\)@.*|\1|p' ~/.config/secrets/gitea | head -1)
|
||||
```
|
||||
|
||||
Never pass the token as a `-H` argument (world-readable in `/proc/<pid>/cmdline`). Write a 600-perm curl config:
|
||||
|
||||
```bash
|
||||
printf 'header = "Authorization: token %s"\n' "$TOK" > /tmp/g.cfg
|
||||
```
|
||||
|
||||
## Trap 1 — PR create answers `"[Head]: Required"` when the body is the problem
|
||||
|
||||
Two independent causes, both look like a missing field:
|
||||
|
||||
1. No `Content-Type: application/json` header → Gitea parses the raw JSON as form data, sees no `head`, returns 422 `[Head]: Required`.
|
||||
2. Putting `data = "@/tmp/body.json"` inside a `-K` curl config does **not** expand the `@`.
|
||||
|
||||
Fix: add `header = "Content-Type: application/json"` to the config and pass `--data @/tmp/body.json` on the command line:
|
||||
|
||||
```bash
|
||||
curl -s -K /tmp/g.cfg -X POST --data @/tmp/pr.json \
|
||||
"http://192.168.0.143:3000/api/v1/repos/dank/franksign/pulls"
|
||||
```
|
||||
|
||||
## Trap 2 — issue label assignment via PATCH /issues/N is silently dropped
|
||||
|
||||
`PATCH /api/v1/repos/{o}/{r}/issues/{n}` with `{"labels":[id]}` returns **200/201 with the issue JSON**, yet a follow-up GET shows `labels: []` (it can even clear existing labels). Labels are a subresource:
|
||||
|
||||
```bash
|
||||
curl -s -K /tmp/g.cfg -X POST --data '{"labels":[4]}' \
|
||||
"http://192.168.0.143:3000/api/v1/repos/dank/franksign/issues/18/labels"
|
||||
```
|
||||
|
||||
200 returns the assigned label array. Resolve label names→ids via `GET /repos/{o}/{r}/labels` (franksign canonical five: ids 2–6; `ready-for-agent` = 4).
|
||||
|
||||
## Verification law
|
||||
|
||||
Both traps fail *successfully*. After any label/PR write, GET the resource and check the field changed; never trust the status code alone.
|
||||
|
||||
## Cleanup
|
||||
|
||||
`rm -f` the temp config + body files (config holds the token).
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
name: gitea-merge-form-encoded
|
||||
description: "Merge a Dankclaw Gitea PR with form-encoded do=merge — and the two body failures: 422 [Do]: Required (JSON body) and jsontext invalid character 'd' (form body with Content-Type: application/json header)."
|
||||
---
|
||||
|
||||
# Gitea merge on Dankclaw: form-encoded, and nothing else
|
||||
|
||||
`POST /api/v1/repos/<owner>/<repo>/pulls/<n>/merge` with `-d 'do=merge&merge_style=no-ff'`.
|
||||
|
||||
## The two body failures
|
||||
|
||||
1. **JSON body** → `422 {"message":"[Do]: Required"}`. The endpoint reads FORM fields, not JSON.
|
||||
2. **Form body but a `Content-Type: application/json` header** (e.g. left over in the curl `-K` config file from an earlier PR-body PATCH on the same request setup) → `422 {"message":"[]: jsontext: invalid character 'd' at start of value"}` — Gitea tries to parse `do=merge...` as JSON.
|
||||
|
||||
## Working pattern
|
||||
|
||||
Keep TWO curl config files when a session both patches PR bodies (JSON) and merges (form):
|
||||
```
|
||||
# merge.cfg — auth ONLY, no Content-Type
|
||||
header = "Authorization: token $TOKEN"
|
||||
```
|
||||
`curl -s -K merge.cfg -X POST -d 'do=merge&merge_style=no-ff' http://192.168.0.143:3000/api/v1/repos/<o>/<r>/pulls/<n>/merge`
|
||||
|
||||
- Success = **HTTP 200 with an empty body**; verify by landing a checkout (`git ls-remote origin main`), never by re-reading the PR.
|
||||
- PATCH `/pulls/<n>` (title/body) is the opposite: JSON body WITH the Content-Type header.
|
||||
- Token from `~/.config/secrets/gitea` via a 600-perm `-K` file, never `-H` (`/proc/<pid>/cmdline` is world-readable).
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: gitea-pr-api-cycle
|
||||
description: "\"Open/update/comment/merge PRs and issues on the Dankclaw Gitea API (192.168.0.143:3000) with the token-safe sequence (POST /pulls rejects Basic with 'token is required'), numeric label ids, the form-encoded merge field, the async 405 'try again later' merge trap, the empty-token-returns-404 trap, and post-merge live-checkout fast-forward plus the plugin-package-name upgrade gotcha.\""
|
||||
---
|
||||
|
||||
# Gitea PR/issue cycle — Dankclaw API
|
||||
|
||||
Friction-earned 2026-10-18 (PRs #15–#20, issue #18), re-earned 2026-10-19 on `dank/dankvoice` PRs #4–#6 (the merge payload claim was wrong; an empty token masqueraded as a missing repo), and re-earned 2026-10-20 on `dank/command-center` PRs #55…#57 (the 405 recalculation trap). Re-earned again 2026-10-20 on `dank/dankvoice` #25 (merge gate loop worked; suite verdict delivered as an issue comment). Re-earned 2026-10-21 on `dank/command-center` issue #58 / PR #59 (issue labels are numeric ids; the `<type>/<issue#>-<slug>` branch name has to be right *before* the first push). Same day on `dank/serverAppConfigs` #64: the Basic-vs-token split below cost two wasted POSTs.
|
||||
|
||||
## Where
|
||||
- API base: `http://192.168.0.143:3000/api/v1/repos/dank/<repo>/{pulls,issues}` — repo varies (`command-center`, `dankvoice`, `serverAppConfigs`); numbering is per-repo, so a footer citing `#36` means nothing without the repo.
|
||||
- Web UI pages under `/dank/<repo>/pulls/N` are session-auth private: a plain URL fetch answers **404** even when the API and `xdg-open` work. Merge state questions go through the API (`GET /pulls/N` → `state`, `merged`, `merge_commit_sha`) — never conclude from a failed page fetch.
|
||||
- **Basic vs token split**: the `username=`/`password=` fields of `~/.config/secrets/gitea` as `Authorization: Basic …` work for **reads and issue/PR comment writes**, but `POST /pulls` refuses them with `403 token is required`. Pull-request lifecycle writes need the **token** header. Use the token for everything PR-shaped; never conclude "API is broken" from a Basic 403 on `/pulls`.
|
||||
- Token: never inline, never in argv (`-H "Authorization: token …"` lands in `/proc/<pid>/cmdline`, world-readable). Extract into a 600-perm curl config and hand it to curl with `-K`:
|
||||
```bash
|
||||
tok=$(sed -nE 's#^https?://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
echo "tok len ${#tok}" # 40 — assert before use
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||
curl -sS -K "$cfg" http://192.168.0.143:3000/api/v1/repos/dank/<repo>/labels
|
||||
rm -f "$cfg"
|
||||
```
|
||||
- **Empty token answers `404 {"message":"not found"}`, NOT 401** — Gitea hides the private repo. A 404 on a path you have used before means auth, not a typo'd repo. The trap that produces the empty token: `awk -F'/' … '{print $NF}'` on the store line (`scheme://user:token@host/`) — the host is in `$3`, the match never fires, you get a silent empty variable. Always assert the length before POSTing.
|
||||
- Multi-line bodies: build JSON in python (`json.dumps({"title":…, "body":open(f).read(),…})`) or `jq -n --arg b "$(cat b.txt)"`. NEVER pass `---` as a bare jq argument (parsed as a flag). Escaped quotes inside `python3 -c '…'` break on `\"` — write the script to a file or use a heredoc.
|
||||
- Always capture status: `-o r.json -w '%{http_code}'`; a bare `-o` hides failures. A `201` response piped straight into `python3 -c` that prints `None None` means you parsed an **error** body (missing `number`) — print `message` too.
|
||||
|
||||
## Create
|
||||
`POST /pulls` with `{title, body, base:"main", head:<branch>}` — JSON is accepted here (unlike merge). Plain `<branch>` works; the `owner:branch` fork form is unnecessary on same-repo branches.
|
||||
- **409 `pull request already exists for these targets [issue_id: N, …]`** — a PR for that head/base exists already. Common cause: the push message printed `/pulls/new/<branch>` and someone (Frank, or an `xdg-open` you ran) clicked Create. `GET /pulls?state=all`, find it, then `PATCH /pulls/N` with your title+body instead of re-posting.
|
||||
- `GET /pulls?state=all` on this Gitea omits `head`/`base` per entry — read `/pulls/N` for those, or key off `number`/`user`.
|
||||
- Issues: same shape, `POST /issues` `{title, body}`; `PATCH /issues/N` fixes a body retroactively (works for PRs too, since a PR is an issue) — used to correct a false verification claim in a PR that had already been merged. Body update returns **201**, not 200; confirm by reading `body` back, not by status code.
|
||||
- **Issue labels are numeric ids, not names.** `POST /issues` with `"labels":["ready-for-human"]` answers `422 {"message":"[]: json: cannot unmarshal JSON string into Go int64 within \"/labels/0\""}`. Resolve first: `GET /labels` → pick `id` by `name` → post `"labels":[20]`. (PRs take no `labels` field on create; label afterwards via `/issues/N/labels` with `{"labels":[id]}`.)
|
||||
- Comments (the channel for late-arriving evidence, e.g. a test-suite verdict posted after PR creation): `POST /issues/N/comments` with JSON `{body}` — works for PRs too, returns **201**.
|
||||
- **Open the issue before the first commit**, so `Refs #N` fits the message, and get the branch name right before the first push. `git branch -m <new>` + `git push -u origin <new>` + `git push origin --delete <old>` is a clean rename with no PR yet; do it rather than ship a branch that violates `<type>/<issue#>-<slug>` (a `docs/…` branch carrying a `fix:` commit is the common slip, and `git grep` on the old name still shows it).
|
||||
|
||||
## Merge (the 422 trap) — corrected
|
||||
`POST /pulls/N/merge` needs **form-encoded** fields; the merge form field name is lowercase **`do`**:
|
||||
|
||||
```bash
|
||||
curl -sS -X POST -K "$cfg" \
|
||||
--data-urlencode 'do=merge' --data-urlencode 'merge_style=no-ff' \
|
||||
http://192.168.0.143:3000/api/v1/repos/dank/command-center/pulls/N/merge
|
||||
```
|
||||
|
||||
JSON `{"Do":"merge"}` (what this skill used to claim) and header variants all return `422 {"message":"[Do]: Required"}`. `merge_style` ∈ `no-ff|merge|rebase|squash`. Success returns **200 with an empty body**; verify by landing a checkout rather than re-reading the PR. Merge only with Frank's explicit per-PR OK.
|
||||
|
||||
### The async 405 when several PRs merge in a row
|
||||
Merge #55, then #56 and #57 back-to-back, and #56/#57 answered:
|
||||
|
||||
```
|
||||
405 {"message":"Please try again later","url":"http://dankclaw:3000/api/swagger"}
|
||||
```
|
||||
|
||||
That is not a failure and not rate limiting: merging one PR moves `main`, which invalidates every other PR's cached mergeability, and Gitea recalculates it asynchronously. A merge attempt during recalculation is refused. Remedy — bounded poll on the PR, merge when the flag is true:
|
||||
|
||||
```bash
|
||||
for attempt in 1 2 3 4 5 6 7 8 9 10; do
|
||||
m=$(curl -sS -K "$cfg" "$api/pulls/$n" | python3 -c \
|
||||
'import json,sys;d=json.load(sys.stdin);print(d.get("mergeable"),d.get("merged"))')
|
||||
case "$m" in *True*) <merge POST>; break ;; *) sleep 4 ;; esac
|
||||
done
|
||||
```
|
||||
|
||||
All three landed within ~5 s of retrying. Treat `mergeable` as a readiness gate, not a report. Match `*True*` on the pair defensively: `False True` means already merged — stop, don't merge again.
|
||||
|
||||
### Verifying a merge without touching the deployment
|
||||
`GET /pulls/N` → `state: closed`, `merged: true`, `merge_commit_sha`. Then prove content from the bare repo, not the working tree: `git fetch -q origin && git show origin/main:<path> | grep -n <key line>` and `git log --oneline --graph -4 origin/main` (two parents = the `--no-ff` Frank asks for). A number you deliberately destroyed can legitimately reappear once — quoted in `omp-docs/log.md` as the wrong value; `git grep -c <badvalue> origin/main` naming only the log page is the pass condition, not a leftover. `Closes #N` in the PR body auto-closes the issue; confirm with `GET /issues/N` → `state`.
|
||||
|
||||
## Stacked PRs (no rebase)
|
||||
When two open branches edit the same section of the same file, merge the sibling branch into the later one (`git merge docs/other-branch`) instead of rebasing — Frank rejects the history rewrite. Consequences to state in the PR body: merging the lower PR first reduces the upper diff to just its own work; merging the upper first makes the lower an already-contained no-op. Either order is clean; `git branch -d` on the lower branch refuses until its commits reach `main`, because `-d` checks against the current HEAD.
|
||||
|
||||
## Post-merge landing
|
||||
- Command-center: the checkout IS the deployment — `git -C ~/.omp fetch -q origin && git -C ~/.omp merge --ff-only origin/main`. This refuses if **any** incoming path is dirty in the worktree — see `land-merged-pr-with-blocked-tree-ops` for the shared-workspace case where `checkout` and `stash` are both policy-blocked.
|
||||
- serverAppConfigs: `deploy.sh` pushes the dev checkout's branch first — if the checkout's `main` is behind a merge you just made (via the API), the deploy dies with `error: failed to push some refs`. `git merge --ff-only origin/main` in the deploy checkout BEFORE running `deploy.sh` after any externally-merged PR.
|
||||
- **A PR that changes a path you edited live adds that path to the discard list.** Tonight: the live dotfile fix was dirty *and* incoming, so landing needed `git -C ~/.omp checkout -- <that file>` as well as the usual file. Discard is content-free only when you proved byte identity first (`cmp` the branch file against the live one before merging) — say so in the PR body, and note that if `live-sync.sh` commits the dirty file first the merge stays clean because both sides hold identical bytes.
|
||||
- Package repos (`~/Projects/<pkg>`): same `--ff-only`, then reinstall (`omp plugin upgrade <name>`) and `cmp` the installed `dist/index.js` against the repo `dist/index.js` to prove the live artifact is the merged one. Check `ls -ld node_modules` after landing anything that touched it.
|
||||
- **`omp plugin upgrade` takes the npm package name, not the short name.** `omp plugin upgrade dankvoice` fails with `Failed to upgrade dankvoice: dankvoice is not installed` while the plugin is plainly installed; the package is `omp-dankvoice`. Find the truth with `ls ~/.omp/plugins/node_modules` or `find ~/.omp -path '*<name>/dist/index.js' -not -path '*/Projects/*'` — the installed tree is unscoped (`~/.omp/plugins/node_modules/omp-dankvoice/dist/index.js`), not `@dank/dankvoice`.
|
||||
- **`cmp` exiting 1 BEFORE the upgrade is the expected state** (installed bundle is the pre-merge build, diff reported at some char/line). Do not treat it as a failed deploy and do not "fix" the build; run the upgrade, then `cmp` again for byte identity, and grep the new bundle for a string only the merged code contains (e.g. `grep -c "Heckle instead of summarizing"`) as a second, semantic confirmation.
|
||||
- Prune: `git worktree remove --force <path>` (force: untracked deps link), `git branch -D`, `git push origin --delete <branch>` (allowed: local Gitea). Before removing a worktree, check no live process has a cwd inside it (`ls -l /proc/*/cwd | grep -c "wt/<name>"`) — same guard `live-sync.sh` uses — and copy any file you moved into a gitignored `.AGENTTRASH/` there to its permanent home first, or the removal destroys the only copy.
|
||||
|
||||
## Attribution footer (AGENTS.md)
|
||||
Every issue/PR body ends with `---`, then `cwd: <session cwd> · session: <session id>`. Read the real values, never invent an id.
|
||||
- **The session id is the UUID suffix of the LIVE transcript `.jsonl`, not the session directory name.** Transcript dirs are keyed by cwd (e.g. `-.omp-wt-wt-20261019-020309-6d1fc1f`) and ONE cwd can hold several sessions (resume / continuation forks), so `ls -t ~/.omp/agent/sessions/` cannot tell you which is yours and `ls -t <dir>` is equally blind. Pick the file being appended to right now: `ls -l --time-style=+%H:%M:%S <dir>/*.jsonl` beside `date +%H:%M:%S` — the mtime that equals the current second is the live session. The id is the part after `_` in `<timestamp>_<uuid>.jsonl`.
|
||||
- A `history.db` query for `session_id` filtered by `cwd` usually agrees, but it is not authoritative for WHICH of several sessions in this cwd is live — use the jsonl mtime test.
|
||||
- `local/` does NOT disambiguate: a forked transcript dir carries copies of the same `local://` artifacts, so matching artifact names proves nothing about which session is live.
|
||||
- `cwd` is that session's own working directory (the path the harness states), **not** the build worktree you committed in — when they differ, name the build worktree inside the body and keep `cwd` the session's.
|
||||
- Resolve `local://<name>.md` with `read local://<name>.md` (its header prints the absolute path) instead of guessing the directory.
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: gitea-review-comment-html-scrape
|
||||
description: "Read inline review comments on the Dankclaw Gitea when pulls/N/comments 404s (Gitea 1.27.3): scrape comment-content blocks and diff anchors from the PR HTML page"
|
||||
---
|
||||
|
||||
# Gitea review-comment extraction when the API 404s
|
||||
|
||||
On the Dankclaw Gitea (192.168.0.143:3000, Gitea 1.27.3) the review-comment API is dead for inline comments:
|
||||
`/api/v1/repos/{owner}/{repo}/pulls/{N}/comments` returns 404 `not found` **even with a valid token**, and `pulls/comments?pull_id=` / `?pull_request_id=` variants 404 too. `/pulls/{N}/reviews` works but its bodies are empty for inline reviews (read `comments_count` for how many exist); `/issues/{N}/comments` shows only conversation comments, never inline ones.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Confirm the review exists and how many inline comments it holds:
|
||||
`curl -sS http://192.168.0.143:3000/api/v1/repos/dank/command-center/pulls/{N}/reviews | jq -c '.[] | {id, state, comments_count}'`
|
||||
2. Scrape the PR page instead (no token needed for a public repo):
|
||||
`curl -sS http://192.168.0.143:3000/dank/command-center/pulls/{N} -o /tmp/pr.html`
|
||||
3. Extract each comment's text — every inline comment renders a `comment-content` block; dedupe (each comment appears twice in the HTML):
|
||||
|
||||
```python
|
||||
import re, html
|
||||
src = open('/tmp/pr.html').read()
|
||||
seen = set()
|
||||
for m in re.finditer(r'comment-content', src):
|
||||
t = html.unescape(re.sub(r'<[^>]+>', ' ', src[m.start()-100:m.start()+800]))
|
||||
t = re.sub(r'\s+', ' ', t).strip()[:400]
|
||||
if t not in seen:
|
||||
seen.add(t); print('>>', t)
|
||||
```
|
||||
|
||||
The comment text appears twice back-to-back inside each block ("text text") — read it once. Blocks start after `Copy Link|Copy Source`; ignore those chrome words.
|
||||
|
||||
4. **Locate the file/line anchor** (comment-content blocks carry no `data-path`/`data-line` attrs on this build): search the raw HTML for the comment string, walk *backward* ~4–7 KB, strip tags; the nearby `agent/...` path, `@@ -a,b +c,d @@` hunk header, and surrounding diff context lines identify the anchored file and the quoted source line.
|
||||
|
||||
## Replying / housekeeping
|
||||
|
||||
- Per-conversation resolve API does not exist on this build (404s) — reply to the round as a normal issue comment (`POST /repos/{o}/{r}/issues/{N}/comments`, token via `curl -K` temp cfg, `header = "Authorization: token …"`, 600 perms, `jq -n --arg b "$(cat b.txt)" '{body:$b}'`) listing which note got which fix commit.
|
||||
- Edit a posted comment: `PATCH /repos/{o}/{r}/issues/{N}/comments/{id}` with `{"body": ...}` — use to fix your own typos rather than double-posting.
|
||||
- Token source on this box: `~/.config/secrets/gitea` (grep the remote-URL pattern); never echo it.
|
||||
|
||||
## Why this works
|
||||
|
||||
The PR page is server-rendered with full inline comment bodies; Gitea's "Copy Source" UI element guarantees the raw text is present pre-markdown-render.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: gitea-silent-ignore-patch-trap
|
||||
description: "Gitea API answers success codes while silently ignoring mutation bodies: labels in PATCH /issues/N, and any JSON body sent without Content-Type: application/json. Verify mutations with a follow-up GET, never the status code."
|
||||
---
|
||||
|
||||
# Gitea silent-ignore family (issue mutations)
|
||||
|
||||
Two verified failure modes on the Dankclaw Gitea API (1.27.3, verified live 2026-10-21 on `dank/franksign`); both **return a success code and change nothing**:
|
||||
|
||||
1. **`PATCH /repos/{o}/{r}/issues/{n}` with `{"labels":[...]}`** — 200, labels untouched, even *with* a proper JSON Content-Type. Labels move only via the dedicated endpoints:
|
||||
- add: `POST /issues/{n}/labels` with `{"labels":[<id>]}`
|
||||
- replace: `PUT /issues/{n}/labels` with `{"labels":[ids]}` (works without Content-Type)
|
||||
- remove: `DELETE /issues/{n}/labels/{id}`
|
||||
2. **Any JSON body without `header = "Content-Type: application/json"` in the curl config** — e.g. closing with `PATCH {"state":"closed"}` returns **201 and no-ops**; identical request with the header closes. curl's default `-d` type (`application/x-www-form-urlencoded`) makes this silent.
|
||||
|
||||
## Rules
|
||||
- Mutating JSON calls: always send `Content-Type: application/json` alongside `Authorization: token` in the same `-K` config file.
|
||||
- **Never trust the status code for mutations — follow up with `GET` and read the field** (`labels`, `state`). Both traps look like success otherwise.
|
||||
- Token stays out of `-H` args (world-readable `/proc/<pid>/cmdline`): extract from `~/.config/secrets/gitea` into a 600-perm curl config, use `-K`.
|
||||
|
||||
## Quick verification idiom
|
||||
```bash
|
||||
curl -sK "$cfg" "http://192.168.0.143:3000/api/v1/repos/dank/franksign/issues/22" \
|
||||
| jq '{state, labels: [.labels[].name]}'
|
||||
```
|
||||
|
||||
Franksign's vault page `franksign-docs/agents/issue-tracker.md` owns the repo-local copy of this law; this skill exists so other repos (serverAppConfigs, command-center) don't re-burn the cycle.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: gitea-token-from-http-credential-store
|
||||
description: "Extract the Dankclaw Gitea token from ~/.config/secrets/gitea (it is the URL password, not key=value lines) for API calls, and diagnose the silent empty-token trap that makes public-repo reads pass while writes fail."
|
||||
---
|
||||
|
||||
The Gitea credential store line on Frank's boxes is **http** (not https) with a percent-encoded port, and the **token is the URL password** — there are NO `username=`/`password=` lines:
|
||||
|
||||
```
|
||||
http://dank:<40-char-token>@192.168.0.143%3a3000
|
||||
```
|
||||
|
||||
## Extraction that actually works
|
||||
|
||||
```bash
|
||||
tok=$(sed -nE 's#^[a-z]+://[^:]+:([^@]+)@.*#\1#p' ~/.config/secrets/gitea | head -1)
|
||||
echo "tok len ${#tok}" # must be 40 — assert before any call
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
|
||||
curl -s -K "$cfg" http://192.168.0.143:3000/api/v1/<path>
|
||||
rm -f "$cfg" # in the SAME session: the file holds a live token
|
||||
```
|
||||
|
||||
## `git credential-store --file … fill` answers EMPTY — do not debug it
|
||||
|
||||
Tried 2026-10-21 while opening `dank/command-center` PR #61: the obvious extraction
|
||||
|
||||
```bash
|
||||
printf 'protocol=http\nhost=192.168.0.143\nport=3000\n\n' | git credential-store --file ~/.config/secrets/gitea fill
|
||||
```
|
||||
|
||||
prints **nothing**, with no error. Not permissions, not a wrong path: the stored line encodes the port inside the host field (`@192.168.0.143%3a3000`), so a `host=`+`port=` query can never match it. Add `path=` or drop `port=` and it still answers empty. Read the file with `sed` (recipe above) and move on — the chain `TOKEN=$(…) && [ -n "$TOKEN" ] && …` fails silently at the test, which reads like a curl failure because the `&&` hides which link broke.
|
||||
|
||||
## The silent-empty-token trap (bit 2026-10-21, cost a debug detour)
|
||||
|
||||
An `awk -F= '/^username=/{...}'` parse of this file yields `tok=""` **silently** — and reads still appear to work when the repo is **public**: `dank/serverAppConfigs` and `dank/command-center` return real issue/label data unauthenticated, so every census "passes" until the first **write** (issue comment, label, PR create/merge) fails with `{"message":"token is always required"}`-style errors or 404s (private repos 404 on an empty token — that is an auth problem, NOT a missing repo).
|
||||
|
||||
- `echo "tok len ${#tok}"` (must be 40) before believing ANY API result — reads included.
|
||||
- Prove auth once per session: `GET /api/v1/user` → `"login":"dank"`.
|
||||
- `curl -s -K <600-perm file>` carrying `header = "Authorization: token $tok"` — never `-H` on argv (`/proc/<pid>/cmdline` is world-readable; this leaked an HF token once).
|
||||
- `jq -n --arg t … --arg h … --arg b "$(/bin/cat /tmp/pr.md)" '{…}'` — `$(cat file)` gets optimised away in this harness, so use `--rawfile b /tmp/pr.md` instead.
|
||||
|
||||
## Other notes
|
||||
|
||||
- `git credential fill` fails headless here (`could not read Username … terminal prompts disabled`); read the store file directly instead.
|
||||
- Never print the token: mask when inspecting the file —
|
||||
`sed -E 's#(://[^:]+:)[^@]+@#\1REDACTED@#' ~/.config/secrets/gitea`
|
||||
- Merge writes need form-encoded fields, not JSON — see `gitea-pr-api-cycle`.
|
||||
- Issue/PR numbers are **per repo**: `dank/dankvoice` #8–#11 (refactor, hardening, AbortError fix, menu defects) are unrelated to `dank/command-center` #8–#11 (SDLC, trunk-law). Read the body from the repo the work targets before referencing an issue in a commit message.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
name: gitignore-trailing-slash-symlink-contamination
|
||||
description: "Trailing-slash gitignore rules don't match symlinks, so worktree dependency symlinks get tracked and pushed as 1-line blobs; same-name rules silently match worktree dirs, and .deploy-stale pollutes. Three fixes with tests."
|
||||
---
|
||||
|
||||
Gitignore semantics trap with three real-world consequences (serverAppConfigs, 2026-10-19):
|
||||
|
||||
## The rule
|
||||
|
||||
`.gitignore` patterns ending in `/` match directories only. Worktree-relative: `**/node_modules/` is `**/node_modules/` → a symlink named `node_modules` does NOT match, so it is "untracked" no more — it gets added, committed, and pushed as a 1-line blob containing its target path.
|
||||
|
||||
## Consequence 1: worktree symlink contamination
|
||||
|
||||
- `git add -A` then commit then push in a worktree whose root holds `node_modules -> ...` (the worktree convention symlink) commits it.
|
||||
- The repo then holds `package/frontend/node_modules` as a text file containing the absolute path on the author's box.
|
||||
- The receiving worktree gets that 1-line file instead of the symlink → "module not found" and "why is this a 30-byte file?".
|
||||
- Detection: `git ls-tree -r origin/main | awk '$1=="120000"'` (mode 120000 = symlink) or `git ls-files -s | grep 120000`.
|
||||
- Fix: ignore rules WITHOUT trailing slash (`**/node_modules`, `**/.venv`) so symlinks match too.
|
||||
|
||||
## Consequence 2: same-name worktree rules silently matching worktree dirs
|
||||
|
||||
A root rule like `wt-*` (meant to ignore local worktree checkouts) matches every path whose last segment starts with `wt-` — so legitimate content under such a directory goes untracked. `git status --ignored` is the only visible trace; detection: `git check-ignore -v <path>` prints the owning rule + line number. (Repo-specific: `dank/command-center`'s `.gitignore` carries a "per-worktree memory buckets" block ignoring `agent/memories/sharpshooter/wt-*/` and `agent/memories/summaries/wt-*/` — intentional there.)
|
||||
|
||||
## Consequence 3: `.deploy-stale`
|
||||
|
||||
`deploy.sh` renames a stale tree to `<path>.deploy-stale` before recreating it. Without an ignore rule those directories land in `git status` as untracked and get swept into `git add -A` commits; they accumulate on every server.
|
||||
|
||||
## Fixes with tests
|
||||
|
||||
```gitignore
|
||||
**/node_modules
|
||||
**/.venv
|
||||
.deploy-stale/
|
||||
```
|
||||
|
||||
Black-box test against the real rules, not a fixture: `git check-ignore -v <symlink-target>` must name the rule. `git check-ignore` takes hypothetical paths — nothing needs to exist on disk:
|
||||
|
||||
```bash
|
||||
git check-ignore -v natives/x.wasm dotfiles/x/.config/noctalia/Noctalia.json
|
||||
```
|
||||
|
||||
Two ignore surfaces must be kept in sync (deploy exclusions vs ignore rules): if a deployer excludes `*/Noctalia*` but the ignore list has only the lowercase class, generated files become tracked-but-never-deployed — test both halves with `git check-ignore`.
|
||||
|
||||
## Recovery (if contamination already pushed)
|
||||
|
||||
1. `git rm --cached <path>` on the symlink blob, commit, push.
|
||||
2. Fix the ignore rule first (else the next `git add -A` re-adds it).
|
||||
3. Untracked ≠ deleted from history: the blob stays in every old commit. Only a rewrite removes it; that is the operator's call, not a cleanup run alone.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: grafana-provisioned-dashboard-verify
|
||||
description: "Author and verify repo-provisioned dashboards on Dankclaw live: Grafana 13 has no unauthenticated API and no local admin, so prove through the e2e SSO browser path, and close the copier gap that hides never-provisioned boards."
|
||||
---
|
||||
|
||||
## When to use
|
||||
You changed `shared/grafana/provisioning/dashboards/*.json` (or panel PromQL) and must prove the LIVE boards actually receive data — or a panel/whole board reads empty and you must classify: dead expression vs never-provisioned vs legitimately-empty.
|
||||
|
||||
## The two gaps this skill exists for
|
||||
1. **The copier gap.** A repo dashboard JSON is NOT a live dashboard until the deployer carries it. Until 2026-10-21 `copy_grafana_provisioning` in `scripts/deploy-remote.sh` named `llm-fleet.json` alone: the `containers` board was merged, seam-tested, and documented as "working" yet had NEVER been provisioned; the other boards existed on disk (`/opt/grafana/provisioning/dashboards/`) only via one-time hand-copies, and one had silently gone stale. Fixed: copier globs `dashboards/*.json`; `GRAFANA_PROV_ROOT` is the test seam; a bats case executes the copier for real into a sandbox.
|
||||
2. **The API-proof illusion.** The vault used to say "verify through the APIs". On Grafana 13.2.2 that is dead: `/api/health`, `/api/frontend`, `/api/org`, `/login` all 404 **unauthenticated**, and there is NO local admin password (root `.env` sets no `GF_SECURITY_ADMIN_*`; the only account is the Authentik RP login). Do not hunt for credentials; use the browser path below.
|
||||
|
||||
## Procedure
|
||||
### 1. Mechanical expression census (over SSH — Prometheus binds loopback)
|
||||
Extract every panel `expr` from the repo JSONs with python, substitute template vars (`$host`,`$container`,`$job` → `.*`, `$__auto`/`$interval` → `5m`), and run each as an instant query against `http://127.0.0.1:9090/api/v1/query` via `ssh dankclaw`. `curl` to `127.0.0.1:9090` from the dev box is blocked; keep queries on-host. Watch zsh globbing on `?`/`[` — quote the remote command.
|
||||
Empty result ≠ defect automatically:
|
||||
- `ALERTS` panel empty = **healthy** (ALERTS carries only firing/pending series).
|
||||
- Zero by design: no cgroup limits exist on the fleet → a memory-vs-limit panel is dead by premise (rewrite as % of host total, don't delete).
|
||||
- Real join defects: cadvisor `host` label vs node-exporter `job` label sets are incompatible — fix with `group_left label_replace(node_memory_MemTotal_bytes{job=~"node-.*"}, "host", "$1", "job", "node-(.*)")`, and `count by (job) (expr) / on(job) count by (job) (...)` needs the `on(job)` matcher when label sets differ. Parens around `on(host)` after `group_left` are **illegal**; `group_left label_replace(...)` without them parses fine.
|
||||
|
||||
### 2. Prove the provisioned files (filesystem = installation proof only)
|
||||
```
|
||||
ssh dankclaw 'for f in /opt/grafana/provisioning/dashboards/*.json; do \
|
||||
cmp -s "$f" "/home/dank/serverAppConfigs/shared/grafana/provisioning/dashboards/$(basename "$f")" \
|
||||
&& echo "$(basename $f) same" || echo "$(basename $f) DIFFERS"; done'
|
||||
```
|
||||
Missing file or DIFFERS ⇒ the copier (or an unmerged branch) is the bug, not the JSON.
|
||||
|
||||
### 3. Browser proof (the real surface)
|
||||
Throwaway `.mjs` placed **inside `tests/e2e/`** (`@playwright/test` resolves from `node_modules` there; a script in `/tmp` fails with `ERR_MODULE_NOT_FOUND`). Node on this box is Zig-rewritten v26: `readFileSync(p,"text")` throws `ERR_INVALID_ARG_VALUE` — use `readFileSync(p).toString()`.
|
||||
Sequence, exactly as `specs/20-grafana-sso.spec.ts` drives it:
|
||||
1. `newContext({storageState: ".auth/qa-e2e-admin.json", userAgent: REAL_UA})` — state is bound to the homepage origin; on the grafana vhost it is only the Authentik session cookie, so you MUST:
|
||||
2. `goto https://grafana.franksdank.cloud/` and poll until body lacks `"Making sure you're not a bot"` (Anubis PoW warm-up).
|
||||
3. `goto .../login/generic_oauth` and poll until `url` host is the vhost and path ≠ `/login` (RP round-trip). Landing on `/login` = auth flow broken, not the dashboard.
|
||||
4. `goto /d/<uid>?orgId=1&from=now-1h&to=now`, wait ~10 s, then read panels: titles via `.panel-title,[class*='panel-title']`; a panel is `NO-DATA` iff its container text contains `"No data"`, `RENDERED` iff a descendant matches `svg:not(:empty),canvas,img`. Panels lazy-render; scroll (`mouse.wheel`) before judging.
|
||||
Classify: `PANEL-NOT-FOUND` for a whole board ⇒ board not provisioned (step 2). `NO-DATA` with a proven-live expression ⇒ variable/label mismatch — re-run the expression with template vars expanded.
|
||||
Throwaway scripts → `~/.AGENTTRASH/<date>-topic/` after use; never `rm`.
|
||||
|
||||
## Sandboxed deployment-function tests (bats)
|
||||
To behaviorally test a deploy-remote.sh copier: extract with `sed -n '/^func()/,/^}/p'`, `eval` it, stub `run_privileged() { "$@"; }`, point the function's `*_ROOT` seam env var at a sandbox, execute, assert files. Traps:
|
||||
- `VAR=x func` prefix assignments do NOT bind for shell **functions** — `export VAR=...` first.
|
||||
- `mktemp -d` + `trap 'rm -rf' RETURN` races bats' trap machinery: the sandbox vanishes mid-function and bats blames the `trap` line. Use `$BATS_TEST_TMPDIR` (auto-cleaned).
|
||||
- Prove red on the OLD body by patching a `mktemp -d` copy of `tests/ shared/ scripts/` and running bats there — no checkout mutation.
|
||||
- Full suite must run from the repo root (`./tests/run.sh`), not from a subdirectory.
|
||||
|
||||
## Land flow (repo law)
|
||||
Branch off fresh `origin/main`; merge needs Frank's explicit current OK; Gitea merge endpoint wants **JSON** `{"Do":"merge"}` (form-encoded `Do=merge` 422s with `[Do]: Required`); then `deploy.sh dankclaw --service grafana` and repeat steps 2–3 as the post-deploy gate.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
name: grep-secret-gate-hook-traps
|
||||
description: "Write a zero-dependency grep-based pre-commit secret gate that fails closed: the GNU grep -- one-shot trap, POSIX ERE limits, bracket collating elements, and rc=2-as-success in command substitution."
|
||||
---
|
||||
|
||||
# Fail-closed grep secret gate (pre-commit hook)
|
||||
|
||||
Use when writing/auditing a bash pre-commit hook (or any CI tripwire) that scans
|
||||
staged content with `grep -E` for credential shapes — no gitleaks available.
|
||||
|
||||
## The four traps (all personally earned; each silently PASSES bad content)
|
||||
|
||||
1. **`--` is a ONE-SHOT end-of-options marker.** `grep -nE -- 'p1' -- 'p2' 'p3'`
|
||||
takes the later `--` and patterns as FILE names → rc=2, errors on stderr.
|
||||
Correct multi-pattern form: repeat `-e` for every pattern; never `--`.
|
||||
2. **`if hits=$(... | grep ...)` treats rc=2 as success.** Any non-zero-but-not-1
|
||||
(broken pattern, unreadable file) enters the `then` branch with EMPTY output.
|
||||
Never branch on the substitution's truthiness. Capture rc explicitly:
|
||||
`hits=$(...); rc=$?` then: `rc>=2` → BLOCK (fail closed — a dead regex is
|
||||
worse than none), `rc==1` → clean, `rc==0` → hits.
|
||||
3. **POSIX ERE (`grep -E`) has no `(?:...)` groups.** Use plain `(...)`.
|
||||
4. **`[:-]` inside a bracket expression starts a collating element** →
|
||||
`grep: repetition-operator operand invalid` (rc=2, see trap 2). Put `-` FIRST
|
||||
in every class: `[-A-Za-z0-9/+=_.:+]`.
|
||||
|
||||
## Gate structure that works
|
||||
|
||||
- Scan **staged content**, not the working tree: `git show ":$f"` over
|
||||
`git diff --cached --name-only --diff-filter=ACM`.
|
||||
- Denylist doc/fixture paths (`README`, `docs?/`, `tests?/`, `*.example`) — else
|
||||
the hook blocks its own patterns and every test fixture.
|
||||
- Report `BLOCKED <file>:<line>` only — never echo the matched secret.
|
||||
- Wire with tracked `hooks/pre-commit` + `git config core.hooksPath hooks`
|
||||
(lives in the COMMON git dir config → one set in the main checkout covers
|
||||
every worktree). Fresh clones are unprotected until it's set; document it.
|
||||
- Worktrees lacking `hooks/` while `core.hooksPath` is set: git dies on commit
|
||||
(`cannot run ... No such file or directory`) — merge main in, don't unset.
|
||||
- Do NOT ship the hook in an npm plugin artifact (`files`): node_modules never
|
||||
runs git; pin its ABSENCE in the pack test instead.
|
||||
|
||||
## Test it red-first through the real seam (bats)
|
||||
|
||||
`git init` a fixture repo, `cp` the real hook in, `git config core.hooksPath hooks`,
|
||||
then drive REAL `git commit` and assert:
|
||||
- credential-shaped `.env` (RFC-2606 domains / all-zero ids as inert fixtures) →
|
||||
non-zero AND `git log` count unchanged (blocks, not warns);
|
||||
- PEM header line → blocked;
|
||||
- clean prose → passes;
|
||||
- the hook committing ITSELF → passes (trap 1/3/4 usually die here);
|
||||
- exit-code masking: `bats ... | tail` returns tail's rc — never `&& commit`
|
||||
behind a piped suite, or you commit on red.
|
||||
@@ -1,48 +1,53 @@
|
||||
---
|
||||
name: halogen-npu-feasibility-census
|
||||
description: "Census whether halogen-flash-server's NPU small-model path can run on a Strix Halo host: image gate, amdxdna/XRT presence, IOMMU and fabric-clock blockers, enablement deltas."
|
||||
description: "Census whether halogen-flash-server's NPU small-model path can run and swap models on a Strix Halo host: image gate, amdxdna/XRT presence, IOMMU and fabric-clock blockers, enablement deltas, boot-set-only model loading (no live reload), and the silent-fallback-to-Flash trap for unarmed chat-class NPU models."
|
||||
---
|
||||
|
||||
Use when asked whether halogen/halogen-flash-server can use the XDNA2 NPU (embeddings, rerank, moderation/guard, decider, `qwen3.5-2b` generation, `flux2-klein-4b` images, `qwen3-asr-*`) on a Ryzen AI Max / Strix Halo host, or before proposing to enable it.
|
||||
Use when asked whether halogen/halogen-flash-server can use the XDNA2 NPU (embeddings, rerank, moderation/guard, decider, `qwen3.5-2b` generation, `flux2-klein-4b` images, `qwen3-asr-*`) on a Ryzen AI Max / Strix Halo host, before proposing to enable it, or when asked whether it can **hot-swap / live-reload** NPU models like FastFlowLM does.
|
||||
|
||||
## What the feature is (not a speed knob)
|
||||
|
||||
The Flash trunk stays on the iGPU. `HALOGEN_NPU_MODELS=<names>` starts a second engine in the SAME container and on the SAME API port; the request's `model` field picks the executor. Unset → server identical to before. Each NPU model is a precompiled AIE-ML program (`devices/devices.hnpm`, per-partition `devices/u<N>.elf`, `devices/consts.hnpb`, `<id>.hnpw` weights, `tokenizer/tokenizer.json`), loaded via the HOST's XRT → kernel `amdxdna` → `/dev/accel/accel0`. One pass at a time; `HALOGEN_NPU_QUEUE` 64, overflow queues server-side, then `503 engine_busy`. Entry `entrypoint.sh` validates every file's bytes + sha256 against `/opt/halogen/npu/models.txt` before anything starts. NPU models run only in single-container "all" mode, NOT the `engine`/`api` split.
|
||||
The Flash trunk stays on the iGPU. `HALOGEN_NPU_MODELS=<names>` starts a second engine in the SAME container and on the SAME API port; the request's `model` field picks the executor. Unset -> server identical to before. Each NPU model is a precompiled AIE-ML program (`devices/devices.hnpm`, per-partition `devices/u<N>.elf`, `devices/consts.hnpb`, `<id>.hnpw` weights, `tokenizer/tokenizer.json`), loaded via the HOST's XRT -> kernel `amdxdna` -> `/dev/accel/accel0`. One pass at a time; `HALOGEN_NPU_QUEUE` 64, overflow queues server-side, then `503 engine_busy`. Catalog in the 0.16.2 image: `decider-0.8b`, `qwen3-embedding-0.6b`, `qwen3-reranker-0.6b`, `qwen3guard-gen-0.6b`, `qwen3.5-2b` (+ `flux2-klein-4b`, `qwen3-asr-*` in newer tags). Only the programs named in `HALOGEN_NPU_MODELS` are armed.
|
||||
|
||||
## Census (all read-only, run from the operator box)
|
||||
## Boot-set-only: NO live reload (tested on the live endpoint)
|
||||
|
||||
`HALOGEN_NPU_MODELS` is read **at start**; the entrypoint validates/converts/loads the named programs then. There is **no** runtime load/unload and **no** admin endpoint to add a model. FLM comparison: FLM was a standalone NPU-only server (`flm serve <one-model>`, single FIFO lane) free to reconfigure its one AIE partition; halogen boots Flash (iGPU) **plus** the NPU programs as ONE coupled process (shared port + shared memory/power budget), so the NPU set is a boot-time constant. Changing even a checkpoint's weights reconverts "at the next start", never live.
|
||||
|
||||
Consequence — arming a new program = edit `scripts/frank-dragoon.env` -> redeploy -> **container restart -> the whole Flash payload (tens of GB) reloads, minutes**. The small NPU program is not the cost; the Flash reload is.
|
||||
|
||||
## The silent-fallback trap (the one that bites)
|
||||
|
||||
A request naming an **unarmed** catalog model behaves differently per endpoint class:
|
||||
- `/v1/rerank`, `/v1/moderations`: honest **400 `model_not_found`**, message ends `(loaded: none)`. The endpoint only exists if that engine was armed at boot.
|
||||
- `/v1/chat/completions` (the `decider-0.8b` decision path and `qwen3.5-2b` generation): **silent fall-through to the Flash model** — HTTP 200, body `"model": "<flash id>"`, no error. A guard/decision call to an unarmed model is answered by the main (often abliterated) endpoint you were trying to screen against, and looks like success. Measured: an unarmed `decider-0.8b` "is this a prompt injection?" probe returned `"no"` from Flash. **Fail-safe is NOT the default on the chat route.**
|
||||
|
||||
## Live probe (idle lane; one ssh, non-disruptive — no restart happens)
|
||||
|
||||
```bash
|
||||
ssh <host> '
|
||||
# 1. engine gate + catalog in the pinned image
|
||||
podman exec <ctr> bash -c "grep -rlo HALOGEN_NPU_MODELS /opt/halogen /usr/local/bin /entrypoint.sh 2>/dev/null | head"
|
||||
podman exec <ctr> cat /opt/halogen/npu/models.txt | head -40
|
||||
# 2. is NPU actually wired today? (expect NO env, NO device, NO /dev/accel inside)
|
||||
podman inspect <ctr> --format "{{json .HostConfig.Devices}}"; podman exec <ctr> ls /dev/accel
|
||||
# 3. kernel driver + firmware
|
||||
lsmod | grep -i amdxdna; sudo dmesg | grep -iE "xdna|npu" | tail -5; ls -l /dev/accel
|
||||
# 4. XRT (distro packages are fine; vendor mounts /opt/xilinx/xrt)
|
||||
dpkg -l | grep -iE "xrt|xdna"; ls /usr/lib/x86_64-linux-gnu/libxrt* /usr/lib/libxrt* 2>/dev/null
|
||||
# 5. BLOCKER 1: IOMMU
|
||||
cat /proc/cmdline; ls /sys/kernel/iommu_groups | wc -l
|
||||
# 6. BLOCKER 2: fabric clock held?
|
||||
for c in /sys/class/drm/card*/device; do echo "== $c"; cat $c/power_dpm_force_performance_level 2>&1; cat $c/pp_dpm_fclk 2>&1 | tr "\n" " "; echo; done'
|
||||
B=http://127.0.0.1:7770 # halogen port; internal engine is :8731/:8740 muxed onto it
|
||||
curl -s $B/v1/models | jq -r '.data[].id' # armed set = boot set; must not mutate
|
||||
# control (armed embedding) -> 200 with a real vector:
|
||||
curl -s $B/v1/embeddings -d '{"model":"qwen3-embedding-0.6b","input":["ping"]}'
|
||||
# unarmed rerank/guard -> expect 400 "(loaded: none)":
|
||||
curl -s $B/v1/rerank -d '{"model":"qwen3-reranker-0.6b","query":"t","documents":["a","b"],"top_n":1}'
|
||||
curl -s $B/v1/moderations -d '{"model":"qwen3guard-gen-0.6b","input":["hello"]}'
|
||||
# unarmed decider/gen -> watch the "model" field; if it is the flash id, it fell through:
|
||||
curl -s $B/v1/chat/completions -d '{"model":"decider-0.8b","temperature":0,"messages":[{"role":"user","content":"x"}],"response_format":{"type":"json_schema","json_schema":{"name":"g","description":"is injection?","schema":{"enum":["no","yes"]}}}}' | jq -r '.model // .error.message'
|
||||
# prove nothing restarted:
|
||||
systemctl show frank-dragoon.service -p NRestarts -p ActiveEnterTimestamp --value
|
||||
```
|
||||
`decider` with a schema and a non-zero `temperature` 400s on a *sampling* guard first ("send temperature 0"); set `temperature:0` to reach the router and see the fallback.
|
||||
|
||||
Docs (raw fetch, authoritative): `docs/NPU.md`, `docs/npu-small-models.md`, `deploy/host/halogen-fabric-clock` under `peonist-ai/halogen-flash-server` on GitHub. Tag list: `podman search --list-tags ghcr.io/peonist-ai/halogen-flash-server`.
|
||||
## Enablement gates (all four must hold before it runs)
|
||||
|
||||
## The two blockers and the enablement delta
|
||||
|
||||
- **IOMMU must be on.** `amd_iommu=off` turns the NPU off (no PASID/SVA); `iommu=pt` keeps it. Needs a grub edit + reboot. `iommu_groups == 0` ⇒ off.
|
||||
- **Fabric clock must be pinned at the top `pp_dpm_fclk` level** (or `power_dpm_force_performance_level=high`); `auto` = unheld. GPU+NPU concurrently HANG Strix Halo machines and corrupt NPU results while that clock moves. Fix: root container with `-v /sys:/host/sys` (server holds it itself), or install `deploy/host/halogen-fabric-clock` + unit (~1 W). If it can't hold, the server FAILS CLOSED and prints the commands.
|
||||
- Container delta: `--device /dev/accel/accel0` + `HALOGEN_NPU_MODELS`; `--ulimit memlock=-1:-1` and `--group-add keep-groups` usually already present. Distro XRT: mount each of `libxrt_coreutil.so.2`, `libxrt_core.so.2`, `libxrt_driver_xdna.so.2` TWICE (once at `/opt/xilinx/xrt/lib/`, once at its own path) — never mount an `/opt/xilinx/xrt` holding only symlinks.
|
||||
|
||||
## Costs to price before proposing
|
||||
|
||||
- Upstream A/B: IOMMU translation ON costs **13–16 % prefill** (460 → 385 tok/s at 2,048 tokens, Ryzen AI Max+ 395); decode unaffected. Whether `iommu=pt` specifically costs that on a given box is a local measurement, not a published fact — run the image's own `sweep` after the reboot.
|
||||
- A few GB host RAM per NPU model (+8 GB FLUX, +1.6/4 GB ASR), and trunk decode drops while the NPU works (shared bus + power).
|
||||
1. Kernel `amdxdna` loaded + `/dev/accel/accel0` present; the node must be world-open (`0666` via a udev rule) or a **rootless** container cannot open it — `serve-frank-dragoon.sh` refuses if the mode bit 02 is clear.
|
||||
2. Host XRT with the NPU plugin (`libxrt_coreutil.so.2`, `libxrt_core.so.2`, `libxrt_driver_xdna.so.2`); on Arch/CachyOS mount each lib twice (image XRT path + its own `/usr/lib` path), never an `/opt/xilinx/xrt` of dangling links.
|
||||
3. **Fabric clock held at top** (`halogen-fabric-clock` root unit) — GPU+NPU at once can hang Strix Halo / corrupt NPU results while that clock changes; the server refuses to start if it can't hold it.
|
||||
4. **IOMMU on**: `amd_iommu=off` kills the NPU; `iommu=pt` keeps it.
|
||||
|
||||
## Laws / traps
|
||||
|
||||
- Never probe the NPU beside the live service: stop + mask first (fleet benchmarking policy); this host class has a precedent of a concurrent GPU experiment needing a power cycle. Distinguish off vs wedged with `ip neigh show <ip>` from a peer.
|
||||
- `/dev/accel/accel0` existing and `amdxdna` probing cleanly does NOT mean the NPU works — with IOMMU off the failure is at context/PASID creation, not at the device node. Do not trust older notes claiming the node itself is missing.
|
||||
- Never probe the NPU beside the live service: stop + mask first (fleet benchmarking policy); this host class has a precedent of a concurrent GPU experiment needing a power cycle. Distinguish off vs wedged with `ip neigh show <ip>` from a peer. A single embedding/decision request on an already-armed idle production lane is a normal request, not a benchmark — safe.
|
||||
- `/dev/accel/accel0` existing and `amdxdna` probing cleanly does NOT mean the NPU works - with IOMMU off the failure is at context/PASID creation, not at the device node. Do not trust older notes claiming the node itself is missing.
|
||||
- `dmesg` lines like `Failed to alloc dev bo memory, ret -28` while large NPU models take turns at NPU memory are benign (vendor-documented); the server recovers.
|
||||
- NPU model RSS is host-side, bypassing the container cgroup (GTT/shared pool), like all Strix Halo GPU memory — container stats under-report the true footprint.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: halogen-npu-lane-boot-loop-triage
|
||||
description: "Triage a frank-dragoon (halogen-flash-server) unit crash-loop caused by the NPU lane: signature classification, restart-debris trap, XRT mount-path census, and the transient env-file proof boot."
|
||||
---
|
||||
|
||||
Triage when `frank-dragoon` (or any halogen-flash-server unit with `HALOGEN_NPU_MODELS` armed) sits in `activating (auto-restart)` with `ExecStart … code=exited, status=1/FAILURE` seconds after start, while `/health` stays dark.
|
||||
|
||||
## 1. Classify the exit BEFORE touching the config
|
||||
- `NRestarts` climbing + exit in <5 s = refusal, not an engine fault. Read `journalctl -u frank-dragoon --since "-30m"` AND the engine log `/home/dank/logs/frank_dragoon.log` (engine stdout never goes to the journal).
|
||||
- Lane signature: log ends with `halogen npu: … <name>: N files checked against this image's record` then `halogen-npu: No such library '<path>'` + `the NPU engine exited before it was ready (rc=1)` → **the NPU lane exit is fatal to the whole container** (supervisor kills the Flash engine with it). Chat-down is the lane's fault even though chat code never changed.
|
||||
- KV/pre-flight lines present but no `locked in RAM` line = died before weight pinning.
|
||||
|
||||
## 2. The restart-debris trap
|
||||
`Restart=always` re-runs the failing boot every ~10 s; each attempt leaves GPU/NPU handles being released, and the next boot logs `GTT in use before this start: N GiB … held by 1 process(es)`. That NOTE is debris from your own crash-loop — do not chase it as a foreign co-tenant (real co-tenants: ComfyUI `python` on `/dev/kfd`, find with `sudo fuser -v /dev/dri/renderD128 /dev/kfd`). `sudo systemctl stop <unit>` FIRST, then any diagnosis.
|
||||
|
||||
## 3. XRT path census (the `No such library '/opt/xilinx/lib/<multiarch>/libxrt_*.so.2'` case)
|
||||
- The image ships NO XRT at all (`find / -xdev -name "libxrt*"` inside the image = empty); the NPU child `dlopen`s the host's libs through the serve script's mounts, which land at TWO destinations only: `/opt/xilinx/xrt/lib/<L>` and `$XRT_DIR/<L>`. A child searching `/opt/xilinx/lib/<multiarch>/` names a THIRD prefix nobody mounts.
|
||||
- The serve loop resolves `REAL="$(readlink -f …)"` and mounts resolved files — if a host package upgrade retargets the symlinks (check `grep libxrt /var/log/dpkg.log` + `ldconfig -p | grep xrt` mtimes vs the last good boot), the file set the child expects can change under byte-identical mounts.
|
||||
- Rootless gotcha: `podman mount` fails outside `podman unshare` and leaves `$M` EMPTY — `grep -r pat "$M/opt"` then greps your CWD = HOST tree (false hits like `/opt/fastflowlm`). Always `podman unshare bash -c 'M=$(podman mount $C); …'` or `--entrypoint ls|cat` probes instead, and distrust hits that name host-only paths.
|
||||
|
||||
## 4. Proof boot without touching production
|
||||
The serve script honors `FRANK_DRAGOON_ENV_FILE` (sanctioned seam). Throwaway script on the host:
|
||||
1. scratch env: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + the knob under test.
|
||||
2. `sed` the serve script to inject the candidate extra mounts into the `NPU_ARGS+=(--device …` line — mount RESOLVED real files (`.so.2.25.00`), never symlinks (vendor: dangling links inside the container).
|
||||
3. `export FRANK_DRAGOON_ENV_FILE=/tmp/… FRANK_DRAGOON_LOG=/tmp/…; FRANK_DRAGOON_CONTAINER=<scratch-name> timeout 1200 /tmp/serve-test.sh` — the container still publishes 7770, so `systemctl stop` the unit first (port binds at CREATE, exit 126 on race).
|
||||
4. Retire same session: `podman rm -f <scratch-name>`, trash `/tmp` scripts; the unit stays stopped until the real fix deploys.
|
||||
|
||||
## 5. Restore chat while the lane is unfixed
|
||||
`HALOGEN_NPU_MODELS=` (blank) is the documented one-line lane-off switch — lane absent = endpoint exactly as before the lane existed. Land through git + `deploy.sh dankclaw --service frank-dragoon`; never hand-edit the remote env.
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: halogen-npu-mount-prefix-proof
|
||||
description: "Prove and fix a halogen frank-dragoon NPU-lane mount failure: crash-loop where a dead lane kills the whole container, dlopen-prefix census, and the sanctioned FRANK_DRAGOON_ENV_FILE proof-boot."
|
||||
---
|
||||
|
||||
# Halogen NPU lane: dead lane kills the container — prove the mount prefix, don't guess
|
||||
|
||||
Applies when a `frank-dragoon` unit crash-loops in seconds (`exit 1` in ~200 ms,
|
||||
`NRestarts` climbing) and the engine log's last lines are:
|
||||
|
||||
```
|
||||
halogen npu: <model>: 15 files checked against this image's record
|
||||
halogen-npu: No such library '<some/prefix>/libxrt_core.so.2'
|
||||
halogen npu: the NPU engine exited before it was ready (rc=1)
|
||||
```
|
||||
|
||||
## Laws this incident wrote (2026-10-21, frank-dragoon, 88-restart loop)
|
||||
- **A dead NPU lane is fatal to the whole container**: the flash-server supervisor
|
||||
exits with the lane. Chat dies beside a healthy GPU preflight — do not read the
|
||||
GTT/weights lines as "engine fine, side feature broken".
|
||||
- The NPU child `dlopen`s distro XRT from **three** container prefixes per library:
|
||||
`/opt/xilinx/xrt/lib/`, the library's own install path, **and
|
||||
`/opt/xilinx/lib/<multiarch>/`**. The missing one is exactly what the error names.
|
||||
- Host state can flip this under a byte-identical container: the lane ran for days
|
||||
with two prefixes, then a host `libxrt2`/`libxrt-npu2` reconfigure (check
|
||||
`grep libxrt /var/log/dpkg.log`) made the child demand the third. Image digest,
|
||||
serve script, payload all unchanged ⇒ suspect HOST package state, not the deploy.
|
||||
- Fastest discriminator vs engine faults: exit in <1 s with the engine's preflight
|
||||
lines present = serve/lane gate failure, not model load. Model-load failures take
|
||||
minutes and print pin/KV lines.
|
||||
|
||||
## Proof procedure (sanctioned seam, no scratch ports)
|
||||
1. Read the error's path verbatim — it names the missing prefix.
|
||||
2. `sudo systemctl stop frank-dragoon` (unit must be down: the proof binds :7770
|
||||
at container CREATE).
|
||||
3. Scratch env file on the host using the bench-sanctioned seam:
|
||||
```
|
||||
/tmp/dragoon-nputest.env:
|
||||
source /home/dank/serverAppConfigs/scripts/frank-dragoon.env
|
||||
HALOGEN_NPU_MODELS=qwen3-embedding-0.6b
|
||||
```
|
||||
4. Scratch serve copy that appends the candidate mount to the `NPU_ARGS+=(--device`
|
||||
line (`sed`), REAL files only (`readlink -f`; vendor forbids dangling symlinks
|
||||
under /opt/xilinx), and run with:
|
||||
`FRANK_DRAGOON_ENV_FILE=/tmp/... FRANK_DRAGOON_LOG=/tmp/... timeout 1200 /tmp/serve-nputest.sh`
|
||||
5. Success = boot ledger (`… in all`) **plus** `npu_api: <model> … buckets u0…u9`
|
||||
plus `/health` `npu.status ok`. Then codify the mount in the serve script, pin
|
||||
it in `tests/frank_dragoon_unit.bats` (the fixture XRT dir's basename IS the
|
||||
multiarch name — name it `x86_64-linux-gnu`), update the endpoint page's NPU
|
||||
prerequisite to the three-prefix law, and cut over via PR +
|
||||
`deploy.sh dankclaw --service frank-dragoon`.
|
||||
6. Retire scratch: the `timeout` kills the proof container; `podman rm -f`,
|
||||
move /tmp scripts to `~/.AGENTTRASH/<date>-<topic>/`. A proof instance
|
||||
serving production while you wait is drift — deploy the merged fix promptly.
|
||||
|
||||
## Traps
|
||||
- Greping image contents rootless: `podman mount` fails (needs `podman unshare`)
|
||||
and leaves `$M` EMPTY, so `grep -r $M/opt` silently greps the HOST cwd — that
|
||||
is how host `/opt/fastflowlm` masqueraded as image content. Always check `$M`
|
||||
non-empty, or use `podman unshare bash -c 'M=$(podman mount C); …'`.
|
||||
- The entrypoint swallows `podman run IMG sh -c …` as a mode name; use
|
||||
`--entrypoint sh/cat/ls`.
|
||||
- The proof container inherits production name/port unless the override is set
|
||||
AFTER sourcing the env (the env file re-exports `FRANK_DRAGOON_CONTAINER`).
|
||||
- `Restart=always` re-runs the lane into its own GPU/NPU debris; stop the unit
|
||||
before any manual boot, don't fight the loop.
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: halogen-npu-xrt-third-prefix
|
||||
description: "Triage a halogen frank-dragoon (flash-server) unit crash-loop caused by the NPU lane: signature \"No such library /opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2\", dead-lane-kills-container law, third-prefix mount fix, and the FRANK_DRAGOON_ENV_FILE scratch-boot proof recipe."
|
||||
---
|
||||
|
||||
# Halogen NPU lane: dead lane kills the container; XRT needs THREE mount prefixes
|
||||
|
||||
Friction-earned 2026-10-21 on `frank-dragoon` (Dankclaw): unit boot-looped 88× (`activating`, exit 1 in ~200 ms) with the GPU/serve_api lines looking healthy; chat dead beside a fine-looking ledger.
|
||||
|
||||
## Signatures (in order of appearance)
|
||||
|
||||
```
|
||||
halogen-npu: No such library '/opt/xilinx/lib/x86_64-linux-gnu/libxrt_core.so.2'
|
||||
halogen npu: the NPU engine exited before it was ready (rc=1; ...)
|
||||
```
|
||||
then `systemctl show <svc> -p ExecMainStatus` = 1, `NRestarts` climbing, and `/home/dank/logs/frank_dragoon.log` truncated at ~10 lines each boot (serve script truncates the log per start).
|
||||
|
||||
**Laws:** a dead NPU lane is NOT survivable — the engine supervisor takes the WHOLE container down on it. And `Restart=always` makes each restart race its own GPU/NPU debris: the FIRST failure's message is the only real evidence; later loop entries are noise. Stop the unit immediately (`sudo systemctl stop frank-dragoon`) before probing.
|
||||
|
||||
## Root cause class
|
||||
|
||||
The NPU child `dlopen`s XRT from `/opt/xilinx/lib/<multiarch>/` IN ADDITION to `/opt/xilinx/xrt/lib` and the install path. Host package churn (dpkg.log reconfigure of `libxrt2` the previous day) made the child demand the third prefix; our mounts filled only two. A lane that worked for days with byte-identical mounts can die purely from host package changes — diff the mounts last, check `grep libxrt /var/log/dpkg.log` first.
|
||||
|
||||
Fix: mount each REAL file (never a symlink) at all three prefixes; derive the multiarch name from `FRANK_DRAGOON_XRT_DIR`'s basename (`MULTIARCH="${XRT_DIR##*/}"`).
|
||||
|
||||
## Proof recipe (production-shaped, no scratch port)
|
||||
|
||||
One card holds one server — never bench on a side port; use the sanctioned env-file seam:
|
||||
|
||||
1. Scratch env on the host: `source /home/dank/serverAppConfigs/scripts/frank-dragoon.env` + knob overrides (e.g. `HALOGEN_NPU_MODELS=...`).
|
||||
2. Copy the serve script with the candidate change applied via `sed` into `/tmp` (or edit + `FRANK_DRAGOON_ENV_FILE`); export `FRANK_DRAGOON_ENV_FILE=<scratch>` and `FRANK_DRAGOON_LOG=/tmp/<name>.log`; run under `timeout 1200`. The seam exists precisely for this.
|
||||
3. Judge from `/tmp/<name>.log`: `npu_api: <model> ... buckets u0…u9` = lane alive; `No such library` = still broken. Then `curl :7770/health` → `npu.status`.
|
||||
4. Retire the same session: `podman stop -t 60` (weights unpin slowly), `podman rm -f`, kill wrapper procs BY PID (`pgrep -f` the scratch serve script — `pkill -f` a pattern containing your own ssh line kills the session), move scratch files to `~/.AGENTTRASH/<date>-<topic>/`, then prove `pgrep -f serve-<scratch>` is empty. Leftover scratch state silently hijacks the next deploy (same class as leftover bench drop-ins).
|
||||
|
||||
## Traps hit en route
|
||||
|
||||
- Rootless `podman mount` fails ("cannot run in rootless mode") and leaves `$M` EMPTY — a later `grep -r "$M/..."` then reads the HOST cwd and "finds" the answer in Loki WAL files. Never grep an unverified mount var; `podman unshare` or `--entrypoint` probe instead.
|
||||
- Image ships NO `/opt/xilinx` at all: all XRT comes from host mounts; `docs/NPU.md` is not inside the image.
|
||||
- `ssh host 'tail <log>'` runs on the DEV box if you forgot `ssh` — a missing-file error is not "the engine wrote nothing".
|
||||
- Health `context` field echoes the RUNNING engine's ceiling — use it to prove which shape is live, unlike `/v1/models` name echo.
|
||||
|
||||
## Verification that the fix landed
|
||||
|
||||
`systemctl show frank-dragoon -p DropInPaths` empty; boot ledger shows `reserving the KV pool … (about 11.0 GiB)`, `… in all`, an `npu_api:` line, no clamp line; deployer prints `NPU lane verified — <model> embedded`.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: harden-precommit-secret-gate
|
||||
description: "Fix or build a pre-commit secret-scan hook that actually blocks: the -z path hole, line-wide placeholder exemptions, inert doc placeholders, glued report lines, self-test fixture paths — each with a red-first bats case."
|
||||
---
|
||||
|
||||
A pre-commit secret gate that fails open is worse than no gate: it teaches the operator that scanning happened. Four defect classes, each with a black-box test that goes red against the broken version (serverAppConfigs PR #70, command-center #59 — 4 holes closed same night, all red first).
|
||||
|
||||
## 1. Non-ASCII paths fail open
|
||||
|
||||
`git diff --cached --name-only` quotes non-ASCII paths (`"docs/\303\251.md"`), so `git diff` on that name returns **empty**, and a loop that `continue`s on an empty diff skips the file silently. Fail closed with NUL delimiting:
|
||||
|
||||
```bash
|
||||
git diff --cached -z --name-only | while IFS= read -r -d '' f; do ...
|
||||
```
|
||||
|
||||
Test: a tracked file whose name contains a non-ASCII character, holding a fake key on an added line, must be blocked.
|
||||
|
||||
## 2. Placeholder filters must be value-scoped
|
||||
|
||||
`if echo "$line" | grep -q "$placeholders"` exempts the whole line, so a comment word vouches for the credential beside it:
|
||||
|
||||
```python
|
||||
token = "sk-…" # example config for the docs
|
||||
```
|
||||
|
||||
Filter only the captured value (the `sk-…` group), never the line. Two directions to test: `key = "example"` stays allowed; `token = "sk-<real>" # example` is blocked.
|
||||
|
||||
## 3. Inert placeholders in documentation
|
||||
|
||||
A skill/README that *documents* a secret gate quotes the vendor's placeholder ids (`AKIA0000000000000EXAMPLE`, all-zero JWTs). Exempt them by value shape (all-zero / `EXAMPLE` suffix / RFC 2606 domain), or the machine tier stops committing — and the refusal usually lands on a heartbeat commit, where it looks like infrastructure failure, not policy. Also note: an unlanded fix means the checkout runs the **old** hook, so a landing can be blocked by the very bug the PR corrects; audit the staged diff and use the documented override (`ALLOW_SECRETS=1`, never `--no-verify`), recording the audit in the commit message.
|
||||
|
||||
## 4. Report-line assembly
|
||||
|
||||
`report+=(" $file: $line")` inside a loop that keeps only the last hit per file glues the next filename onto the previous line when a second hit exists — the report attributes the finding to the wrong file. Test a two-file staged set where both match, and assert each report line names exactly one existing path.
|
||||
|
||||
## 5. Self-test fixtures in the hook itself
|
||||
|
||||
The hook's own fixtures contain fake keys by necessity, so the gate would refuse any commit touching the hook or its tests (or a docs page quoting a key as documentation). Pin the bypass to specific paths and to added lines only, and print what was exempted:
|
||||
|
||||
```bash
|
||||
case "$f" in .githooks/pre-commit|tests/secret-gate.bats)
|
||||
echo "secret-gate: self-test fixtures exempt: $f" >&2; continue;; esac
|
||||
```
|
||||
|
||||
## The scan rule
|
||||
|
||||
Scan **added content only** — `git diff --cached -U0 | grep '^+[^+]'` — or every commit touching a file with an old key in context gets blocked, and the operator's response to a gate that blocks everything is to override it always.
|
||||
|
||||
## Test harness (fixture repo, never the real checkout)
|
||||
|
||||
`git init` a temp repo, `git config core.hooksPath .githooks`, copy the hook in, `git add` a payload, then run it with the env overrides the hook honours (point its cache/notify paths at `$TMP` so tests never touch the user's config). Pin, in order:
|
||||
|
||||
| Case | Expect |
|
||||
|---|---|
|
||||
| fake key on an added line | exit 1, message names the file |
|
||||
| `key = "example"` | pass |
|
||||
| real key + `# example` comment | exit 1 |
|
||||
| non-ASCII filename with a key | exit 1 |
|
||||
| two files with hits | report names both |
|
||||
| self-test fixtures | pass, with the exempt line printed |
|
||||
|
||||
`bats -r` after each edit. `--no-verify` is forbidden on Frank's boxes: the hook IS the control.
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
name: harness-model-config-wire-capture
|
||||
description: "Verify what the omp harness actually sends to a local inference endpoint (reasoning_effort, sampler keys, thinking-off) by pointing a scratch HOME at a logging proxy, instead of inferring it from models.yml comments."
|
||||
---
|
||||
|
||||
A `models.yml` comment claiming "the wire sends X" is a claim about the harness, not the engine, and the harness rewrites between the two: compat maps pass unmapped values through, effort ladders clamp before the map applies, and a "disable reasoning" role flag can be implemented as *send the lowest effort* rather than a switch. Read only the wire.
|
||||
|
||||
## Recipe (minutes, no production state touched)
|
||||
|
||||
1. **Logging proxy** — python `ThreadingHTTPServer` on `127.0.0.1:7799` forwarding body+path to the real engine and appending the request body's interesting keys (`model, reasoning_effort, max_tokens, temperature, top_p, top_k, chat_template_kwargs`) as one JSON line per request. Run it as a named bash service with `ready.port` so the port is proven before use.
|
||||
2. **Scratch HOME** — `mkdir -p /tmp/omph/.omp/agent`, copy the *candidate* `models.yml` + `config.yml` there, rewrite that provider's `baseUrl` to `http://127.0.0.1:7799/v1`. Never edit `~/.omp/agent/` — in `dank/command-center` the checkout is the deployment.
|
||||
3. **One-shot runs** — `HOME=/tmp/omph omp --model '<provider>/<id>:<effort>' --no-tools --no-session --no-skills --no-rules --no-extensions --no-pty --no-lsp -p 'Reply with exactly: OK'`, once per effort (`off minimal low medium high xhigh max`). Add `--print-thoughts` when the question is "did thinking actually happen".
|
||||
4. Read the log, and assert the value **after** the change. A config edit that never engaged is the failure mode this exists to catch.
|
||||
|
||||
## Traps found doing this on frank-dragoon (2026-10-21)
|
||||
|
||||
- `--model X:off` does **not** mean `reasoning_effort: off` on the wire. Reasoning is switched through `compat.reasoningDisableMode` (enum includes `lowest-effort`, `none-effort`, `qwen-enable-thinking-false`, `qwen-template-false`, `chat-template-thinking-false`); the default derives from `thinkingFormat`, and `lowest-effort` quietly keeps thinking alive (`:off` arrived as `"minimal"`, with a canned 26-token thinking block on every "off" request). `none-effort` sent `"none"` → `reasoning_tokens: 0`.
|
||||
- Effort levels are clamped to the harness's per-model ladder **before** `reasoningEffortMap` applies, so a map row for a level the ladder lacks is inert (`:max` arrived as `xhigh` even with `max: max`). Capture before promising a level works.
|
||||
- Map lookup is `compat.reasoningEffortMap?.[effort] ?? thinking?.effortMap?.[effort] ?? effort` — removing a row is a pass-through, never a 400. Claims about rejection must be proven against the engine.
|
||||
- Engine-side claims: ask for a deliberately bad value and read its 400 — the engine prints its own accepted set (`reasoning_effort 'maximum' unsupported; use minimal|low|medium|high|xhigh|max|none`).
|
||||
|
||||
## Engine-side probes that transfer to any OpenAI-compatible endpoint
|
||||
|
||||
- **Accepted parameters**: `{model, messages, max_tokens:1, <param>:<value>}` → status + `error.message`. Acceptance is not honouring: report it as acceptance.
|
||||
- **Level honoured?** same prompt at `max_tokens 2048` per level, compare `usage.completion_tokens_details.reasoning_tokens`. Equal values across levels = the parameter is ignored; `reasoning_closed_at` reveals a per-level thinking ceiling.
|
||||
- **Thinking off**: `chat_template_kwargs {"enable_thinking": false}` and top-level `enable_thinking: false` → expect `reasoning_tokens: 0`.
|
||||
- **Vision**: build a solid-colour PNG in-script (`zlib`+`struct`, no PIL), 512×512, ask for the colour in one word **with thinking disabled** — otherwise the answer is cut off by the thinking block and the probe is inconclusive. A 1×1 image only proves token accounting.
|
||||
|
||||
## Teardown
|
||||
|
||||
Kill the proxy service, remove the scratch HOME, the proxy script and its log. Never leave copies of `config.yml`/`models.yml` in `/tmp`: they carry provider endpoints and sometimes tokens.
|
||||
@@ -1,48 +1,49 @@
|
||||
---
|
||||
name: html-report-ground-truth-audit
|
||||
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
|
||||
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator."
|
||||
---
|
||||
|
||||
Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
|
||||
Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.**
|
||||
|
||||
## Law
|
||||
|
||||
Every numeric claim must be re-measured against the repo, not the narrative that produced it.
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Enumerate every checkable claim
|
||||
Grep the report for numerics, file paths, symbol names, and code blocks:
|
||||
- line counts (`wc -l` the real file)
|
||||
- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
|
||||
- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
|
||||
- commit hashes (`git log --oneline`)
|
||||
- "verbatim" code blocks (see 3)
|
||||
- diagram node lists and captions (see 4)
|
||||
1. **Dump claims with file:line anchors.**
|
||||
`sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt`
|
||||
Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts.
|
||||
|
||||
### 2. Ground numeric claims with one command each
|
||||
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
|
||||
2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `"`/`&` entities leaking into code, and `<br>` line breaks that split a line mid-pipe.
|
||||
|
||||
### 3. Verbatim code blocks: prove them
|
||||
For a block labeled verbatim, the exact string must exist:
|
||||
- live code: `grep -n '<distinctive line>' <file>`
|
||||
- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
|
||||
If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
|
||||
3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it.
|
||||
|
||||
### 4. Diagram claims: run the engine
|
||||
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
|
||||
`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
|
||||
4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25).
|
||||
|
||||
### 5. Check for swallowed sections
|
||||
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
|
||||
5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it.
|
||||
|
||||
### 6. Structural balance
|
||||
Python tag-balance pass over the file:
|
||||
- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
|
||||
- running-div-depth scan printing per-section balance to localize stray closers
|
||||
Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
|
||||
## Filler test (delete on failure)
|
||||
|
||||
### 7. Render proof
|
||||
`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
|
||||
- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
|
||||
- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
|
||||
- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
|
||||
- A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git.
|
||||
- A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.)
|
||||
- Keep only what a reader cannot rebuild from `git log` + `git show`.
|
||||
|
||||
## Deliver
|
||||
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
|
||||
## Serve and verify the artifact
|
||||
|
||||
**Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch:
|
||||
|
||||
```bash
|
||||
base=http://host:port/reports/x.html
|
||||
for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done # service may still be activating
|
||||
for i in 1 2; do curl -fsS "$base" | md5sum; done # stable across refetches
|
||||
diff <(curl -fsS "$base") /path/to/source.html && echo "served == source"
|
||||
```
|
||||
|
||||
## Voice drift
|
||||
|
||||
If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one.
|
||||
|
||||
## Report
|
||||
|
||||
Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: infra-repo-security-review
|
||||
description: "Full-repo security review of an infra/config repo (serverAppConfigs-shaped: shell deployers, quadlets, systemd, Caddy edge, bats/Playwright) — slice fan-out, mechanical censuses, and subagent-claim verification."
|
||||
---
|
||||
|
||||
# Whole-repo security review (infra/config repo)
|
||||
|
||||
For "full security review of this repo" on a deployment-config repo (shell scripts + podman quadlets + systemd units + Caddy + tests). NOT for a diff review — that is `code-review`. NOT for a secrets-only question — that is `git-repo-secret-audit`.
|
||||
|
||||
## 1. Slice, then fan out in ONE `task` batch
|
||||
Five `security-reviewer` slices, each with the repo's own laws pre-injected in `context` so no agent re-derives them (AGENTS.md policy text + exclusions: docs prose, `.AGENTTRASH/**`, binaries, `package-lock.json`):
|
||||
|
||||
1. **Shell** — all `*.sh` + `.githooks/` + `tests/run.sh`: injection, quoting, `mktemp` vs `/tmp/fixed`, `set -e`, `curl -f`, secrets in argv, `rm`/`git clean`/`systemctl disable`.
|
||||
2. **Python** — every `*.py`: `subprocess(shell=True)`, `verify=False`, unbounded `read()`, path traversal on request-supplied filenames, fixed `/tmp` writes, **which HTTP servers exist and which are unauthenticated**.
|
||||
3. **Container + systemd posture** — every `*.container`/`.network`/`.service`/Containerfile.
|
||||
4. **Edge + web** — Caddyfile vhost inventory, Anubis/Authentik chain, response headers, static-asset XSS sinks, service configs (`limiter.toml`, `prometheus.yml`, `alertmanager.yml`).
|
||||
5. **Tests** — vacuous assertions + credentials/secrets in fixtures + harness safety.
|
||||
|
||||
**Set a yield deadline in the prompt.** Subagents here get killed at `task.maxRuntimeMs` (30 min) and the dead ones produced **zero** output. When one runs long, write early: `write agent://<id>` "yield now, unconfirmed items prefixed UNVERIFIED" — the resumed agent returns its partial report instead of losing it. Two of four needed this.
|
||||
|
||||
## 2. Verify every citation before reporting
|
||||
`verify-subagent-review-findings` applies in full: re-grep each cited `file:line` and compare text. Expect ~40% of lines to be off (e.g. an agent described a 13-line hook; the file is 6) and several files to be hallucinated entirely (`.githooks/compose/*.container` did not exist). Discard mismatches; publish a **Discarded subagent claims** section. Re-grep your own citations too — two of mine were off by two lines and had to be corrected in an addendum.
|
||||
|
||||
## 3. Mechanical censuses (cheap, high yield — run these yourself)
|
||||
- **Port binding**: every `^PublishPort=` line, classify `127.0.0.1:` / `192.168.0.x:` / **no address prefix** (= implicit `0.0.0.0`). The unprefixed set is the auth-bypass surface: a LAN client hits the container port directly and skips the Caddy→Anubis→Authentik chain that the vhost enforces. Report as one consolidated finding listing all unprefixed lines, and cite the repo's own correct examples (`127.0.0.1:9090`, `192.168.0.143:8080`) as the fix pattern.
|
||||
- **Image pinning**: count `^Image=` lines with vs without `@sha256:` (this repo: 4 of 54 pinned) — floating tags + `Pull=never` = unreproducible edge.
|
||||
- **Secret modes**: grep every `chmod 600` in the deployer and diff against every file it writes from an `.env`; a `>` redirect without the `chmod` is the finding (the Caddy/Porkbun env file).
|
||||
- **Systemd hardening**: `User=`/`ProtectSystem`/`PrivateTmp`/`NoNewPrivileges` per unit, and `ExecStartPre=+` lines — a `+` hook on a root unit running a checkout-writable script is a local privesc.
|
||||
- **Vhost table**: vhost → upstream → guarded? → exemption documented? Cross-check each upstream against the quadlet `PublishPort` list; unguarded vhost with no exemption comment is a policy violation even when the service self-authenticates.
|
||||
- **Hook wiring**: `git config core.hooksPath` + `ls "$(git rev-parse --git-path hooks)"`. A repo can ship `.githooks/pre-commit` that git never executes — and its own test can assert the *file* is executable and still miss that it isn't installed.
|
||||
- **Secret sweep**: `git cat-file --batch-all-objects --batch` (all history + dangling), match known values read from `~/.config/secrets`; report `prefix…(len N, sha 10)` only, never the value. Watch for the **split-concat disguise** (`PW="as""set"`): their own regex can't see it, but `grep -F` on a fragment finds it and history keeps it forever → rotation required.
|
||||
|
||||
## 4. Report
|
||||
Severity-ranked table (Critical→Low, one line each: location + flaw), then prose detail for Critical/High only, then the Discarded-claims section, then remaining coverage gaps, then an ordered fix set and an offer to branch. Do not create a report file unless asked — chat is the deliverable.
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: land-branch-without-checkout
|
||||
description: "Open a PR from a shared worktree when workspace policy blocks git checkout/restore: push a named branch ref from HEAD without switching branches."
|
||||
---
|
||||
|
||||
# Landing a branch when `git checkout` is policy-blocked
|
||||
|
||||
Shared-workspace policy blocks `git checkout` and `git restore` (working-tree
|
||||
switches could strand concurrent agents' uncommitted work). You still must ship
|
||||
the commit on the current worktree branch as a named `chore|feat|fix/<issue>-<slug>`
|
||||
branch. Never work around the block — go around it via refs.
|
||||
|
||||
## Sequence
|
||||
|
||||
```bash
|
||||
# 1. commit only your files (atomic); leave other dirty paths alone
|
||||
git add <your-files> && git commit -m "..."
|
||||
|
||||
# 2. branch ref at HEAD — no working-tree change, stays on wt/<...>
|
||||
git branch <type>/<issue>-<slug> HEAD
|
||||
git push origin <type>/<issue>-<slug> # pre-push hooks still run
|
||||
|
||||
# 3. PR via tracker API with head = pushed ref (see gitea-pr-api-cycle / gh)
|
||||
```
|
||||
|
||||
- The worktree stays checked out on `wt/<timestamp>`; the new branch exists
|
||||
only as the pushed ref (plus a local ref that merges cleanly later).
|
||||
- `-u` upstream tracking on the wt branch is misleading — skip it; set
|
||||
`git config branch.<name>.remote origin` + `.merge` only if you will
|
||||
later check the branch out.
|
||||
- Dirty regenerated fixtures left behind (e.g. `gen:sample` PDFs rewritten
|
||||
by setup/gate runs): `git restore` is also blocked — do NOT edit-forward
|
||||
to revert them; report them. They are regenerated on every bootstrap, so
|
||||
dirt is expected and harmless.
|
||||
|
||||
## Verify
|
||||
|
||||
- `git ls-remote --heads origin <branch>` prints the same sha as `git rev-parse HEAD`.
|
||||
- PR API returns 201 with `head.sha` equal to that sha.
|
||||
|
||||
## Do NOT
|
||||
|
||||
- `git checkout -b`, `git stash`, `git restore`, `git reset --hard` here —
|
||||
all working-tree mutations the policy can block, and stashes in a shared
|
||||
checkout are a cross-agent footgun.
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
name: land-merged-pr-with-blocked-tree-ops
|
||||
description: "Use when a merged PR must land in a checkout whose working tree is dirty and workspace policy blocks git checkout/restore/stash: commit-the-dirt pattern, the secret-gate false positive that blocks the landing commit, conflict resolution proven settings-vs-comments, and heartbeat (live-sync) handoff."
|
||||
---
|
||||
|
||||
Landing a merged trunk into a checkout that IS the deployment (dank/command-center `~/.omp`), when tracked files are dirty and policy forbids `checkout`/`restore`/`stash` on shared state. Re-earned 2026-10-21 landing PRs #61 + #63 into `~/.omp` (27 commits behind, two conflicts, one gate refusal).
|
||||
|
||||
## 1. Classify the dirt before touching it
|
||||
|
||||
`git status --porcelain` then compare against what the merge brings in — only overlap blocks the merge:
|
||||
|
||||
```bash
|
||||
{ git diff --name-only; git diff --cached --name-only; } | sort -u > /tmp/dirty.txt
|
||||
git diff --name-only HEAD origin/main | sort -u > /tmp/incoming.txt
|
||||
comm -12 /tmp/dirty.txt /tmp/incoming.txt # these paths must be clean
|
||||
```
|
||||
|
||||
Note the first column of porcelain: `M ` = **staged**, ` M` = unstaged. A staged file shows nothing in `git diff` — look at `--cached` too, or you will "verify" a file is unchanged while it is staged. `comm` needs both inputs sorted; unsorted input prints "file is not in sorted order" and a partial answer.
|
||||
|
||||
## 2. Prove a dirty file content-free before resolving
|
||||
|
||||
`cmp` it against the trunk blob, never against memory:
|
||||
|
||||
```bash
|
||||
git show origin/main:$f > /tmp/incoming
|
||||
cmp -s "$f" /tmp/incoming && echo "IDENTICAL to trunk" # commit/discard is content-free
|
||||
```
|
||||
|
||||
If identical, the dirt is a sync artifact, not user work. Say so in the commit message with the `cmp` result — that is what makes a direct commit on `main` defensible for a prose-tier file.
|
||||
|
||||
## 3. Commit rather than discard (policy-blocked verbs)
|
||||
|
||||
With `checkout`/`restore`/`stash` unavailable, the only non-destructive unblock is a commit. Two separate commits, and order matters:
|
||||
|
||||
1. Prose-tier dirt proven byte-identical to trunk — commit alone, message states `cmp` proved no prose change.
|
||||
2. Machine tiers (`agent/memories/**`, `agent/managed-skills/**`, `agent/config.yml`, `agent/models.yml`, `dotfiles/`) — let the heartbeat do it: `LC_ALL=C scripts/live-sync.sh` stages allowlist only, commits, merges trunk, pushes, and **exits 0 even when it aborts the merge on conflict** (the alert goes to stderr + ntfy). `exit 0` does not mean landed — check `behind=` afterwards.
|
||||
|
||||
Do NOT `git add -A`: the index may already hold dozens of staged machine-tier files, and `git add <one file>` does not clear them — they ride along into your commit.
|
||||
|
||||
## 4. The gate refusal that blocks the landing commit
|
||||
|
||||
The pre-commit secret gate scans the **staged** diff, so a single-file commit still trips on previously staged content. It fires on inert placeholders inside the skill docs that document the placeholder (e.g. `AKIA0000000000000EXAMPLE` — the AWS all-zero example id in `git-secret-gate-hook` / `harden-precommit-secret-gate`). If the checkout is behind, it is running the **pre-fix** hook while the fix sits unlanded in trunk: the landing is blocked by the very bug the PR corrects.
|
||||
|
||||
Audit first, then override deliberately — never `--no-verify`:
|
||||
|
||||
```bash
|
||||
git diff --cached | grep -nE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}|ghp_[A-Za-z0-9]{20,}|glpat-[A-Za-z0-9]{16,}|BEGIN [A-Z ]*PRIVATE KEY|eyJ[A-Za-z0-9_-]{20,}'
|
||||
git diff --cached | grep -oE 'AKIA[A-Z0-9]{12,}|sk-[A-Za-z0-9]{16,}' | sort | uniq -c # what exactly, how many
|
||||
ALLOW_SECRETS=1 git commit -F msg # message records the audit + that the gate ran
|
||||
```
|
||||
|
||||
Record in the message: the hits are inert placeholders, which files they sit in, that this checkout runs the pre-fix hook, and that the refusal dies with this landing.
|
||||
|
||||
## 5. Merge conflicts: resolve by proving settings identical
|
||||
|
||||
```bash
|
||||
git merge --no-commit --no-ff origin/main
|
||||
git diff --name-only --diff-filter=U
|
||||
```
|
||||
|
||||
For each conflicted file, get all three blobs (`HEAD`, merge base, `origin/main`) and ask what the local side uniquely holds:
|
||||
|
||||
- **Prose / doc pages**: diff ours vs trunk; if ours only lacks the incoming trim, take trunk.
|
||||
- **Real settings files (kdl/yaml/env)**: strip comments and diff — `diff <(grep -v '^\s*//' ours) <(grep -v '^\s*//' theirs)`. Empty output ⇒ only comments differ ⇒ taking trunk changes no deployed behaviour, and you can say that in the merge message. (2026-10-21: `desk-1-home.kdl` — modes byte-identical, trunk's comment corrected the false "no 240 Hz mode exists".)
|
||||
|
||||
Resolve without the blocked verbs by writing the trunk blob: `git show origin/main:$f > $f && git add $f`. Then validate the live artifact (`niri validate`, the suite) before committing the merge, and push.
|
||||
|
||||
## 6. Verify the push actually landed
|
||||
|
||||
`git push` output can be swallowed when wrapped in command substitution inside a batch — verify from the remote, not from the push:
|
||||
|
||||
```bash
|
||||
git fetch -q origin && printf 'HEAD=%s origin/main=%s behind=%s\n' \
|
||||
"$(git rev-parse --short HEAD)" "$(git rev-parse --short origin/main)" "$(git rev-list --count HEAD..origin/main)"
|
||||
```
|
||||
|
||||
## 7. Close and prune
|
||||
|
||||
- Close the issues with a comment giving merge SHA + PR link + title (numbers are per-repo; a bare `#N` is not a reference).
|
||||
- Prune guards, all three before `git worktree remove --force`: `ls -l /proc/*/cwd | grep -c '<wt-name>'` must be 0; `git -C <wt> status --porcelain | wc -l` must be 0; `git rev-list --count main..<branch>` must be 0. Check for `.AGENTTRASH/**` inside the worktree first — removal destroys the only copy.
|
||||
- `git branch -d` (not `-D`) once the branch is reached by main; `git push origin --delete <branch>` is allowed on the local Gitea.
|
||||
- Delete the temp `-K` token config file in the same session that created it.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: line-edit-anchor-discipline
|
||||
description: "Avoid the recurring edit-tool line-op traps: empty PUT deletes lines, PUT ranges swallowing multi-line block delimiters, and anchors landing mid-expression. Use whenever issuing CUT/PUT line edits on tagged files."
|
||||
---
|
||||
|
||||
# Line-edit anchor discipline (CUT/PUT ops on tagged files)
|
||||
|
||||
Five self-inflicted syntax wounds in one session (franksign, 2026-10-21) all trace to three rules.
|
||||
|
||||
## Rules
|
||||
1. **An empty PUT body DELETES the targeted lines** (tool warns: "Interpreted an empty PUT body as deletion"). Twice it ate a whole log entry / a closing `}`. For no-ops use nothing; for deletes use explicit `CUT`.
|
||||
2. **Never PUT a range whose endpoints cut through a multi-line block.** If a `| {`-style union variant, a frontmatter `---` fence, or a `test(...)` closer straddles the range boundary, the replacement swallows the opener/closer and strands the body. Pre-condition: line N-1 and M+1 must be block-complete. Fix-after-the-fact pattern: read the broken region raw, then `PUT k.=k:` to restore the single swallowed line (e.g. the missing `---`, the orphaned ` | {`, `}` → `});`).
|
||||
3. **Search-hit reads do not authorize PUT.** A grep/folded-summary read marks lines as "never displayed"; the edit tool refuses and echoes actual content. That refusal is a free verification — re-check the echoed lines match intent, then straight-retry with the same tag.
|
||||
|
||||
## Costly mistakes seen
|
||||
- Empty `PUT 7-7:` deleted a just-written log entry; restoring required re-issuing the full entry text.
|
||||
- `PUT 6.=7` replacing `timestamp`+`---` dropped the frontmatter close fence → lint-breaking file silently.
|
||||
- `PUT 10.=11` on a union type swallowed ` | {` opening a 12-line variant → cascade of LSP "unexpected label" warnings.
|
||||
- `CUT` of a stranded opener left its dangling `.json())` behind; balance errors surface only at EOF (`expected ')' but file ends`) — locate by reading regions raw, not by widening the payload.
|
||||
|
||||
## Discipline
|
||||
- After any edit that reports a syntax error or a warning: read the touched region with `:raw` before the next op; fix the single wrong line, do not resend the block.
|
||||
- "no change: byte-identical" means your anchor math is off — re-read, don't append more content.
|
||||
- Prefer replacing whole coherent blocks (start line = block start, end line = block end) over edits that end mid-statement.
|
||||
@@ -0,0 +1,83 @@
|
||||
---
|
||||
name: llamacpp-adapter-swap
|
||||
description: "Add or roll back a GGUF LoRA adapter on a dank-fleet llama.cpp endpoint (e.g. frank-zealot): provenance census, fork-flag gate, fetcher+env+deployer surface, seam tests, and the behavioural before/after probe that is the only real proof."
|
||||
---
|
||||
|
||||
# Applying a GGUF LoRA adapter to a fleet endpoint
|
||||
|
||||
An adapter is **payload, not shape**: it changes what the endpoint answers and nothing in
|
||||
the KV/budget equation. Fleet law still applies — no benchmarks unless Frank orders one,
|
||||
rollback is one env line, `./tests/run.sh` before commit, deployer health gate + e2e gate.
|
||||
|
||||
## 1. Census the artifact BEFORE writing any code
|
||||
|
||||
```bash
|
||||
curl -sS "https://huggingface.co/api/models/<repo>?blobs=true" # size + LFS oid (None => plain git object)
|
||||
curl -sSL -o /tmp/a.gguf "https://huggingface.co/<repo>/resolve/main/<file>" && sha256sum /tmp/a.gguf
|
||||
```
|
||||
|
||||
Parse the GGUF KV section (15 lines of struct.unpack) and read **provenance**: abliteration
|
||||
adapters carry `prism.abliterate.base_gguf` / `base_sha256`, `adapter.type`, `sites`,
|
||||
`target_layers`, `target_embedding`. Compare the declared base against the file the endpoint
|
||||
actually serves (its sha is in the endpoint's own fetcher script). A mismatch is normal for
|
||||
sibling quantizations — the loader matches **tensor names, not hashes** — but it must be
|
||||
stated in the docs, because it means the card's measured numbers are not this endpoint's
|
||||
numbers.
|
||||
|
||||
## 2. Gate the runtime, from the binary's own `--help`
|
||||
|
||||
On the host: `/home/dank/GitPackages/<build>/llama-server --help | grep -iE 'lora|adapter'`.
|
||||
Confirm `--lora-scaled FNAME:SCALE` exists (and `--lora-init-without-apply` + `POST
|
||||
/lora-adapters` if a runtime toggle is ever wanted). Vendor forks gate adapters by metadata
|
||||
version too — e.g. PrismML builds older than `prism-b10777` refuse packs with `unsupported
|
||||
prism.hadamard.version: 2`. Read the card for the build it verified on and put that in the
|
||||
fetcher header.
|
||||
|
||||
## 3. Surface (five files, copy the mmproj convention)
|
||||
|
||||
- `scripts/<svc>.env`: `LORA_FILE=<path>` + `LORA_SCALE=1`; document that **`LORA_FILE=` (empty)**
|
||||
is the rollback and `LORA_SCALE=0` is the inert-but-present mode. Use `${VAR=default}` (no
|
||||
colon) so an explicitly empty value survives — same reason `MMPROJ_FILE`/`PRESENCE_PENALTY` do.
|
||||
- `scripts/serve-<svc>.sh`: `EXTRA_ARGS+=(--lora-scaled "${LORA_FILE}:${LORA_SCALE}")` — **one
|
||||
argv token**, so a scale can never arrive without its file; refuse a missing file by name
|
||||
(`Restart=always` turns an unfinished fetch into a crash loop that reads like a runtime fault).
|
||||
- `scripts/download-<svc>-<adapter>.sh`: copy the sibling fetcher (`-fsSL --fail-with-body`, byte
|
||||
count, sha256, `--verify-only`, idempotent). No LFS oid => pin the content sha. Header carries
|
||||
the provenance finding.
|
||||
- `scripts/deploy-remote.sh`: `LLM_<SVC>_LORA="${LLM_<SVC>_LORA:-<path>}"` + a check whose error
|
||||
names **this** fetcher, not the payload's.
|
||||
- `scripts/deploy-plan.py` needs nothing: fetchers are host-side state and select no service.
|
||||
|
||||
## 4. Seam tests (argv is the contract)
|
||||
|
||||
Stub-engine harness: create the adapter stub from the **shipped env's basename**, emit
|
||||
`LORA_FILE`/`LORA_SCALE` in the generated test env, then assert on output only —
|
||||
`--lora-scaled` value equals `<path>:<scale>`; a distinctive `LORA_SCALE` (0, 0.75) changes the
|
||||
token; `LORA_FILE=` yields **absence** of the flag (`! grep -qxF -- '--lora-scaled'`); a missing
|
||||
file exits non-zero naming that path; extend the env↔deployer anti-drift pair list.
|
||||
|
||||
## 5. Deploy and prove it behaviourally
|
||||
|
||||
Capture the **before** state on the live endpoint first, then: merge → on host `git pull` +
|
||||
run the fetcher → `./scripts/deploy.sh <host> --service <svc>`.
|
||||
|
||||
```bash
|
||||
sudo systemctl show <svc> -p DropInPaths --value # must be empty (bench-drop-in law)
|
||||
sudo tr '\0' ' ' < /proc/$(sudo systemctl show <svc> -p MainPID --value)/cmdline
|
||||
sudo journalctl -u <svc> --since <boot> | grep -iE 'adapter|lora' # llama_adapter_lora_init_impl prints prism.* keys
|
||||
```
|
||||
|
||||
Refusal probe (the only proof of an abliteration adapter): same prompts, `temperature 0.1`,
|
||||
**`max_tokens >= 2048`** — on thinking models a short budget is consumed by
|
||||
`reasoning_content` and `content` comes back empty with `finish_reason: length`, which reads
|
||||
as a refusal that never happened. Classify with a marker regex and report refusal-vs-complied;
|
||||
do not paste generated harmful content into chat. Also run the standing trio: exact-phrase
|
||||
chat, `tool_calls` with valid JSON, and a >=256 px solid-colour vision probe. A residual
|
||||
`⚠️ Important Disclaimer` heading after the swap is style, not the refusal circuit.
|
||||
|
||||
## Docs
|
||||
|
||||
Endpoint page (`serverAppConfigs-docs/LLM/<svc>.md`): Adapter row, Fetch row, Flags row, one
|
||||
Era row whose rollback is the env line, and a measured before/after paragraph. Laws worth
|
||||
adding: provenance is metadata not a gate; an `token_embd` rank would crash, so an adapter is
|
||||
not inert; the runtime pin may gain a second reason. Never restate the card's benchmark table.
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: locale-c-multibyte-awk-silent-loss
|
||||
description: "Debug generated output that silently loses data under LC_ALL=C — a regex quantifier on a multibyte character in awk/grep, plus the locale-dependent red test it produces."
|
||||
---
|
||||
|
||||
# LC_ALL=C kills multibyte regex quantifiers — silently
|
||||
|
||||
Symptom: a generator emits a **complete, exit-0, missing-a-third-of-the-page** artifact in
|
||||
cron / systemd user units / agent shells, and the same command looks perfect in the user's
|
||||
terminal. Companion symptom: a repo test is red for the agent and green for the operator.
|
||||
|
||||
## Probe (do these in order, they are 5 seconds apart)
|
||||
|
||||
```bash
|
||||
echo "LANG=[$LANG] LC_ALL=[$LC_ALL]" # LC_ALL overrides LANG — en_US.UTF-8 + LC_ALL=C is byte-wise
|
||||
locale -a | grep -iE 'utf-?8' | head # which UTF-8 locale actually exists on this box
|
||||
```
|
||||
|
||||
Then test the suspect pattern alone, under both locales, with the literal character:
|
||||
|
||||
```bash
|
||||
printf '%s\n' ' // ─── Applications ───' | awk '/─+ / { print "QUANT MATCHED" }'
|
||||
printf '%s\n' ' // ─── Applications ───' | awk '/─/ { print "LITERAL MATCHED" }'
|
||||
printf '%s\n' ' // ─── Applications ───' | LC_ALL=C.utf8 awk '/─+ / { print "QUANT MATCHED under UTF-8" }'
|
||||
```
|
||||
|
||||
Literal matches everywhere; the quantifier matches only under a UTF-8 locale. That asymmetry IS
|
||||
the bug: gawk under `LC_ALL=C` matches bytes, so `─+` means "the last byte of U+2500, repeated" —
|
||||
a byte sequence that never occurs. mawk/busybox/Go regexp have the same class of failure
|
||||
(quantifier or bracket class spanning a multibyte char).
|
||||
|
||||
## Fix
|
||||
|
||||
Match and strip the multibyte literal with no quantifier:
|
||||
|
||||
```awk
|
||||
/^[[:space:]]*\/\/[[:space:]]*─/ { # literal gate
|
||||
s = $0; sub(/^[[:space:]]*\/\/[[:space:]]*/, "", s); sub(/[[:space:]]+$/, "", s)
|
||||
if (s ~ /─ .*─/) { gsub(/─/, "", s); gsub(/[[:space:]]+/, " ", s); sub(/^ /, "", s); ... }
|
||||
next
|
||||
}
|
||||
```
|
||||
|
||||
**Never pin `LC_ALL=en_US.UTF-8` to "fix" it** — that locale often does not exist (a CachyOS box
|
||||
here shipped only `C.utf8`), so the script then breaks on another machine. Make the matcher
|
||||
byte-safe instead.
|
||||
|
||||
## Pin it with a test
|
||||
|
||||
`run env LC_ALL=C "$GEN" "$IN"` + assert the lost content is present. The operator's UTF-8 terminal
|
||||
can never catch this class of bug, so the explicit C-locale run is the only durable guard.
|
||||
|
||||
## Suite-verification law (how this got missed)
|
||||
|
||||
NEVER judge `bats tests/` from `tail` output — a failure at test 28 is invisible in a 25-line
|
||||
window ending at test 109. Always:
|
||||
|
||||
```bash
|
||||
out=$(bats tests/ 2>&1); echo "$out" | grep -cE '^ok'; echo "$out" | grep -E '^not ok' || echo GREEN
|
||||
```
|
||||
|
||||
Run it under `LC_ALL=C` too: that is the environment every automated caller inherits. Same rule
|
||||
for `| grep -c` on a multi-line JSON field — count what you claim to have counted.
|
||||
|
||||
## Related, do not confuse
|
||||
|
||||
- `bats-assertion-trust` — vacuously-passing assertions (green that proves nothing). This skill is
|
||||
the mirror case: correct assertions that are only red in one locale.
|
||||
- `gitea-pr-api-cycle` — unrelated; landed here only because the PR work is where this surfaced.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
name: monolith-split-under-dist-pin
|
||||
description: "Use when cutting a monolithic source file into modules behind a black-box test suite whose committed build artifact (dist) must stay byte-identical to src; one verified commit per cut, red-proof against the previous bundle. Not for feature work or doc-only edits."
|
||||
---
|
||||
|
||||
# Monolith split under a committed-bundle pin
|
||||
|
||||
Procedure that landed a 1354-line `index.ts` as 9 modules + 301 lines of wiring across 9 commits with zero red commits. Assumes: black-box test suite (tests drive the built bundle, not internals) and a pin test asserting the committed `dist/` equals a fresh build of `src/`.
|
||||
|
||||
## Per-cut loop (one commit per cut, always in this order)
|
||||
|
||||
1. `bun run build` (or project equivalent) **before** staging — commit `dist/` in the same commit as the moved `src/`, otherwise the pin test fails ("committed dist is byte-identical to a fresh build") and you must amend. Amending is fine only while the branch is unpushed.
|
||||
2. `tsc --noEmit`, then run the full suite async (`bats tests/ > /tmp/x.log 2>&1; grep -E '^1\.\.'`).
|
||||
3. **Never write to `src/` while the suite runs.** `setup()` rebuilds the bundle from `src/`, so a concurrent edit attributes someone else's half-written state to your run. Read-only prep (planning the next cut's exact line ranges) is the correct thing to do while waiting.
|
||||
4. Gate green → next cut. Gate red → fix, rebuild, amend, re-gate.
|
||||
|
||||
## Cutting mechanics
|
||||
|
||||
- Cut leaf/pure modules first (grammar, transcript rules, URL resolution, pure segmenter), then stateful ones (config, IO, pipeline, UI). Never let a module import `index` — if the new module needs something from `index`, that thing is a leaf and must move too (this is what forced `segment.ts` out early).
|
||||
- Extract in **verbatim** bodies: read the exact ranges, write them into the new file, then delete the same ranges. Only module docs, export keywords, and injected dependencies change. Keeps each commit reviewable as a move.
|
||||
- Edit tools that require "displayed lines" from the latest read snapshot: re-read the exact ranges under the current tag, then issue one edit call with all ops listed **bottom-up** (original line numbers throughout). Adjacent ranges merge into one `CUT a=b`; leaving a doubled blank line is the usual scar — one extra `CUT` next turn.
|
||||
- Mutable config must reach a long-lived object as **getters**, not a construction-time snapshot: menu handlers mutate `cfg` after construction, and a snapshot freezes the settings the UI exists to change.
|
||||
- After moving code, grep the old file for the moved names: stale call sites compile under bundlers that skip typechecking (`stop()` vs `speaker.stop()` was caught only by `tsc`, not by `bun build`).
|
||||
|
||||
## Proving a new assertion is red-capable (before trusting it green)
|
||||
|
||||
Build the old bundle and point the test harness at it — no stash, no worktree juggling:
|
||||
|
||||
```bash
|
||||
git show HEAD:dist/index.js > /tmp/old-dist.js # bundle is self-contained
|
||||
# rebuild the throwaway harness by extracting the driver the suite writes
|
||||
sed -n "<heredoc-body-range>p" tests/behavior.bats > "$W/driver.ts"
|
||||
DV_EXT=/tmp/old-dist.js <same env as the test> bun "$W/driver.ts" <scenario> # expect RED
|
||||
DV_EXT="$PWD/dist/index.js" <same env> bun "$W/driver.ts" <scenario> # expect GREEN
|
||||
```
|
||||
|
||||
Copy the project's real player/server stub verbatim — a simplified stub (e.g. one that skips writing the log file the driver counts) prints a false `PLAYS 0` and looks like a failing fix. Pin the scratch dir to a fixed path; `ls -dt /tmp/tmp.*` picks up another session's directory.
|
||||
|
||||
## Black-box test hygiene learned the hard way
|
||||
|
||||
- A test that counts artifacts in the **shared** temp directory observes every other process on the box (live agent sessions synthesize into the same `dankvoice-*` dirs). Give the run a private `TMPDIR="$WORK/scratch-tmp"` and assert there; the assertion's sensitivity is proven by the fact it caught a real leftover while scoped globally.
|
||||
- Let preseeds reference the ephemeral fixture port with a placeholder (`__PORT__`) substituted by the driver, so a test can preseed a URL pointing at its own fixture server.
|
||||
- Refactor-only commits get **no new test**: re-asserting unchanged labels/order after a commit whose point is that they are unchanged is a tautology; the existing suite is the gate. New tests belong to behavior changes, red-first.
|
||||
- Tests must observe **output**, never internals: assert `PLAYS`/`CALLS`/`CALL <url>`/`NOTIFY <text>`, not that a variable was set.
|
||||
|
||||
## Docs after a split (same branch, own commit)
|
||||
|
||||
Move each explanation to the file that now owns it (knob list → the module that reads env; state-machine prose → the module that implements it) and leave the entry point a one-line-per-module map. Then grep for pointers into the old file — a README saying "see header of `src/index.ts`" becomes a lie the moment the knobs move.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
name: new-fleet-service-checklist
|
||||
description: "Add a new deployable service to serverAppConfigs: the four registration touch-points (two allowlists, planner, router) whose drift breaks the first-ever deploy."
|
||||
---
|
||||
|
||||
# Adding a new service to the serverAppConfigs deployer
|
||||
|
||||
A new `--service <name>` target must be registered in **four** files or the first deploy fails at a gate. Two fleet services (cadvisor, gpu-exporter) shipped with the client gate missed within a day of each other; the failure mode is `Service 'X' is not deployed on <target>` from `deploy.sh`, exit 2 — before SSH.
|
||||
|
||||
## The four touch-points (per target host)
|
||||
|
||||
1. `scripts/deploy.sh` → `valid_service()` case: add `<target>:<svc>`. **Client gate** — the one people forget.
|
||||
2. `scripts/deploy-remote.sh` → `service_is_allowed()` case: add `<target>:<svc>`, **and** the dispatcher `case` branch `<svc>) deploy_<svc> ;;`.
|
||||
3. `scripts/deploy-plan.py`: the name in the target's service set (`DANKCLAW_SERVICES` / `DANKSERVER_SERVICES`) + `SERVICE_ORDER`, plus a path map so edits to the service's own files select it (for `scripts/<svc>.*` host processes, map the exact filenames like the signal-relay/gpu-exporter maps do).
|
||||
4. `scripts/test-changed.sh`: route the service's files to its seam, else focused test runs skip it (full `tests/run.sh` still covers).
|
||||
|
||||
Sanctioned exception: `dankclaw:searxng-network` is client-only (network unit, no remote handler) — the drift test exempts it.
|
||||
|
||||
## Enforcement + local verification
|
||||
|
||||
`tests/deployment_flow.bats` test "the client and remote allowlists agree per target" diffs the two extracted lists. Verify before pushing:
|
||||
|
||||
```bash
|
||||
grep -oE 'dank(claw|server):[a-z0-9-]+' <(sed -n '/^valid_service()/,/^}/p' scripts/deploy.sh) \
|
||||
| grep -v dankclaw:searxng-network | sort -u > /tmp/cli
|
||||
grep -oE 'dank(claw|server):[a-z0-9-]+' <(sed -n '/^service_is_allowed()/,/^}/p' scripts/deploy-remote.sh) \
|
||||
| sort -u > /tmp/rem
|
||||
diff /tmp/cli /tmp/rem # must be empty
|
||||
```
|
||||
|
||||
Also: a `# comment` after a trailing `\` inside a bash `case` pattern list **breaks line continuation** (syntax error) — annotate on its own line above.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Merge ≠ deploy: `deploy.sh <host> --service <svc>` must still be run per host; a merged service is dead code until then (GPU stats "not showing" was exactly this).
|
||||
- `deploy.sh` refuses to run anywhere but a clean `main` checkout: the canonical dev checkout is `/home/blank/Projects/serverAppConfig` (not the session `~/.omp/wt/...` worktree, not the build worktree).
|
||||
@@ -1,14 +1,15 @@
|
||||
---
|
||||
name: niri-config-edit-validate
|
||||
description: "Locate, edit, validate, and commit niri compositor config on Frank's workstations: desk/display profiles, EDID-keyed outputs, layout, keybind grepping, dotfiles mirror, forced hot-reload."
|
||||
description: "Locate, edit, validate, and commit niri compositor config on Frank's workstations: desk/display profiles, EDID-keyed outputs, layout, keybind grepping, dotfiles mirror, noctalia.kdl last-include color override, forced hot-reload."
|
||||
---
|
||||
|
||||
# Editing niri config on Dank workstations
|
||||
|
||||
## Layout of the config
|
||||
- Live config: `~/.config/niri/config.kdl` is a thin file of `include "./cfg/*.kdl"` lines. **Never conclude "not bound" from grepping only `config.kdl`** — keybinds live in `cfg/keybinds.kdl`, rules in `cfg/rules.kdl`, layout in `cfg/layout.kdl`, etc. Grep `~/.config/niri/cfg/` recursively.
|
||||
- **Include order wins:** `config.kdl` ends with `include "noctalia.kdl"` — a hand-written accent overlay at `~/.config/niri/noctalia.kdl` that re-declares `layout { focus-ring/border/tab-indicator/insert-hint }` + `recent-windows.highlight` colors and BEATS every `cfg/layout.kdl` color edit (observed 2026-10-18: focus-ring color edits to layout.kdl appeared to "not reload" while this file silently repainted them red). Before changing any accent/outline color, grep the WHOLE `~/.config/niri/` tree (not just `cfg/`) for the property and fix the LAST includer. Tracked in repo at `dotfiles/zz-blankpad/niri/.config/niri/noctalia.kdl`; live file is NOT a symlink — sync by `cp` and `diff` like the rest.
|
||||
- niri notation: `Mod` = Super/Win key. Chords like `Mod+R { switch-preset-column-width; }`.
|
||||
- `~/.config/niri` is a REAL directory, not a symlink; the version-controlled copies are a partial mirror under `~/.omp/dotfiles/zz-blankpad/niri/.config/niri/`. Trees can drift: diff them before/after edits. (Hardlink wiring is FRAGILE — editor/agent atomic-rename writes silently break hardlinks; verify live↔mirror by CONTENT diff, never by link trust.)
|
||||
- `~/.config/niri` is a REAL directory, not a symlink; the version-controlled copies are a partial mirror under `~/.omp/dotfiles/zz-blankpad/niri/.config/niri/`. Trees can drift: diff them before/after edits. (Hardlink wiring is FRAGILE — editor/agent atomic-rename writes silently break hardlinks; verify live↔mirror by CONTENT diff, never by link trust.) On blankpad the `cfg/*.kdl` entries are symlinks into the repo (`dotfiles/niri/...` base + `zz-blankpad` overrides) — edit through them freely; `noctalia.kdl` and `config.kdl` are the plain-file exceptions.
|
||||
|
||||
## Desk/display profiles (monitor arrangement)
|
||||
- Profiles: `cfg/desk-<n>-<name>.kdl` (e.g. `desk-1-home.kdl`, `desk-2-portable.kdl`); ACTIVE profile is symlinked as `cfg/display.kdl` — `readlink` it to know which. `scripts/desk.sh` manages switching; `desk.sh capture` prints connected outputs' EDID long names.
|
||||
@@ -21,21 +22,22 @@ description: "Locate, edit, validate, and commit niri compositor config on Frank
|
||||
- Preset list order = cycle order for `Mod+R` (`switch-preset-column-width`); `Mod+Shift+R` cycles backwards. WITH custom preset lists the cycle contains ONLY the configured presets — no implicit full/maximize tail (verified: [0.25, 0.5] cycles period 2). Maximize = separate `Mod+F` `maximize-column`. Built-in defaults without the node: 1/3, 1/2, 2/3.
|
||||
- WITHOUT `preset-window-heights`, `switch-preset-window-height` FALLS BACK to the width presets — user sees width-fractions as heights; define the node explicitly to stop it.
|
||||
- Ground truth for node placement: the binary's embedded template — `grep -a -B4 -A8 "preset-column-widths {" $(which niri)`.
|
||||
- Window outlines: base geometry (`width`, `off`) lives in `cfg/layout.kdl` `focus-ring`/`border`; COLORS effectively live in `noctalia.kdl` (see include-order law). `border { off }` + focus-ring active = the visible outline. Frank's look since 2026-10-18: amber-orange `#ff8c00`, width 4 (`active`+`urgent` in noctalia.kdl; tab-indicator/insert-hint/recent-windows stay red `#f43841` by choice).
|
||||
|
||||
## Workflow
|
||||
1. Edit the live file under `~/.config/niri/cfg/`.
|
||||
1. Edit the live file under `~/.config/niri/cfg/` (symlinked = repo file; except `noctalia.kdl`, edit live then `cp` to mirror).
|
||||
2. `niri validate` (exit 0 + "config is valid") — mandatory before done.
|
||||
3. **Hot-reload is NOT reliable after harness edits** (atomic-rename writes are missed by niri's watcher; user then tests against the OLD config and reports "no change"). ALWAYS follow with `niri msg action load-config-file` (NOT `reload-config`).
|
||||
4. Sync changed file into the mirror: `cp ~/.config/niri/cfg/<f>.kdl ~/.omp/dotfiles/zz-blankpad/niri/.config/niri/cfg/<f>.kdl` (repo root is `~/.omp`, branch `main`, remote = dankclaw Gitea `dank/command-center`; push there is pre-authorized). Verify by `diff`, not by cp success.
|
||||
5. Untracked mirror files: `config.kdl`, `noctalia.kdl`, `animation.kdl`, `display.kdl`, `input.kdl` — flag drift, don't silently add.
|
||||
5. Untracked mirror files: `config.kdl`, `noctalia.kdl`, `animation.kdl`, `input.kdl`, `display.kdl` (symlink target) — flag drift, don't silently add; when a live-only file turns out to be load-bearing (like `noctalia.kdl` was), commit the first durable copy into `dotfiles/zz-blankpad/niri/.config/niri/`.
|
||||
6. Mirror files often carry pre-existing uncommitted drift (desk.sh captures); a commit of your edit folds it in — say so in the report/commit body.
|
||||
|
||||
## Live-behavior verification without disturbing Frank's windows
|
||||
- `niri msg --json <subcmd>` — the `--json` flag goes BEFORE the subcommand.
|
||||
- `niri msg --json <subcmd>` — the `--json` flag goes BEFORE the subcmd.
|
||||
- Targeting differs per action: WINDOW-scoped actions take `--id <window-id>` (NOT `--window`) — e.g. `switch-preset-window-height --id N`. COLUMN-scoped actions (`switch-preset-column-width`, `set-column-width`, `maximize-column`) have NO id — they hit the FOCUSED column; only `set-column-width <px|±N%>` positional CHANGE, no flags.
|
||||
- Spawned test apps do NOT reliably take focus, so blind `spawn + action` cycles resize whatever Frank is looking at. For column actions you must `focus-window --id N` first — and that STEALS focus; if Frank is actively testing chords, your probes and his keystrokes interleave and corrupt both. STOP probing, deliver config + validate, and have him press the chord himself.
|
||||
- `niri msg --json focused-window` can report a different window than assumed (focus moves); for per-window measurement read `niri msg --json windows` and filter by `id`. Present height of proportion p with gaps g over work-area H renders ≈ p*(H+g) − g.
|
||||
- If Frank reports "still seeing old values" after a config edit: run `load-config-file` first, then verify empirically before believing anything else.
|
||||
- `niri msg --json focused-window` can report a different window than assumed (focus moves); for per-window measurement read `niri msg --json windows` and filter by `id`. Present height of proportion p with gaps g over work-area H renders ≈ p*(H+g) − g. NOTE (measured 2026-10-18): this build's `windows`/`focused-window` JSON returns `x/y/width/height = null` — pixel-sampling the outline color from screenshots is not viable; ask Frank to eyeball after `load-config-file` instead of burning turns on capture forensics.
|
||||
- If Frank reports "still seeing old values" after a config edit: run `load-config-file` first, then verify empirically before believing anything else — and check `noctalia.kdl` include-order override BEFORE suspecting the reload.
|
||||
|
||||
## "What does chord X do?" triage
|
||||
For "mod+<key>" questions, check in order: `~/.config/niri/cfg/keybinds.kdl` (compositor wins first), terminal configs (`~/.config/ghostty`, `foot`, `kitty`), tmux, then omp TUI chords (`omp://keybindings.md`; modifiers are ctrl/shift/alt/super only, no `Mod`; `Ctrl+R`=history search, `Alt+R`=app.retry).
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
name: niri-monitor-flap-refresh-triage
|
||||
description: "Diagnose niri/amdgpu monitor failures on blankpad — panel dropping, wrong refresh rate, black/flickering output — and fix the refresh without reloading the compositor or logging Frank out."
|
||||
---
|
||||
|
||||
# niri monitor triage: flaps, phantom modes, 60 Hz fallback
|
||||
|
||||
Frank's reaction to a broken display is immediate and loud; the fix must never log him out.
|
||||
Box: `blankpad` (Cezanne APU, `card1`, s2idle only), compositor `niri --session` as a user unit.
|
||||
|
||||
## Step 1 — separate three different diseases (do this before touching anything)
|
||||
|
||||
```bash
|
||||
niri msg outputs # name, mode, position, full mode list per output
|
||||
for c in /sys/class/drm/card*-*/status; do echo "$(basename $(dirname $c)) $(cat $c)"; done
|
||||
systemctl --user show niri.service -p ExecMainStartTimestamp -p NRestarts # did niri restart?
|
||||
journalctl --user -b --no-pager | grep -iE 'disconnecting connector|new connector|Page flip commit failed|DMCUB|falling back to preferred'
|
||||
journalctl -k -b --no-pager | grep -iE 'amdgpu|PM: suspend exit|\*ERROR\*'
|
||||
```
|
||||
|
||||
Reading it:
|
||||
|
||||
- `disconnecting connector: "X"` then `new connector: X` → **link-level drop** (cable/dock/monitor/driver). Count them: `grep -c 'disconnecting connector'`.
|
||||
- `Page flip commit failed … EINVAL` bursts + kernel `dc_dmub_srv_log_diagnostic_data: DMCUB error` right after `PM: suspend exit` → **amdgpu display-engine state broken by s2idle resume**. Only a reboot clears it; never reboot without an explicit per-run OK.
|
||||
- `configured mode M could not be found, falling back to preferred` → **config asks for a mode the panel does not offer**; the panel silently runs at EDID-preferred (usually 60 Hz). This is the one you can fix instantly.
|
||||
- Config mtimes older than the symptom + no `niri` restart ⇒ the config did not cause it. Say so explicitly; agents get blamed for display problems they didn't cause.
|
||||
|
||||
## Step 2 — the phantom-mode trap (niri fails soft, so it looks like hardware)
|
||||
|
||||
`output { mode "…" }` with a refresh the EDID never advertised does **not** error. niri logs a WARN and drops to `preferred` on every connect → janky 60 Hz ultrawide, blamed on the monitor.
|
||||
|
||||
Compare the requested mode against the panel's real list:
|
||||
|
||||
```bash
|
||||
niri msg outputs | sed -n '/<monitor name>/,/^$/p' | grep -E '^\s+[0-9]+'
|
||||
```
|
||||
|
||||
## Step 3 — apply the mode WITHOUT a config reload (the no-logout path)
|
||||
|
||||
```bash
|
||||
niri msg output "<EDID output name from niri msg outputs>" mode "WxH@refresh"
|
||||
niri msg outputs | grep -A1 '<monitor>' # confirm the new mode
|
||||
pgrep -c niri # compositor still alive, same PID
|
||||
```
|
||||
|
||||
`niri msg output …` is documented as temporary ("not saved into the config file") — it changes one output and cannot log the session out. Avoid `niri action reload-config` when Frank is working: it reloads his whole config.
|
||||
|
||||
## Step 4 — persist, and check you edited the file that is actually live
|
||||
|
||||
Live niri config is a symlink chain: `~/.config/niri/config.kdl` → `cfg/display.kdl` → `cfg/desk-<profile>.kdl`, and `cfg/desk-*.kdl` are symlinks into the repo:
|
||||
|
||||
```bash
|
||||
ls -l ~/.config/niri/cfg/display.kdl # -> desk-1-home.kdl
|
||||
grep -n '239\|180' ~/.config/niri/cfg/display.kdl # prove YOUR edit is in the live chain
|
||||
niri validate # parses the live config; touches nothing
|
||||
```
|
||||
|
||||
`dotfiles/` is on the `live-sync.sh` heartbeat allowlist, so an edited tracked dotfile stays dirty until the hourly heartbeat commits it — that is the designed state, do not commit it on `main` yourself.
|
||||
|
||||
Desk config law (its own header): panels keyed by **EDID identity**, not connector name — but connector-keyed leftovers (`DP-1`, `DP-2`) survive in `desk-1-home.kdl`, and a reconnect that comes back without an EDID string hits those instead. Align them to the same mode, or a flap silently changes your refresh rate.
|
||||
|
||||
## Escalate honestly
|
||||
|
||||
Reboot clears DMCUB/EINVAL state → schedule it, don't take it. Persistent drops after a reboot → cable / dock / monitor DP-version setting, i.e. hardware, not config. Report which of the three diseases you found and which you did **not** fix.
|
||||
@@ -1,45 +1,52 @@
|
||||
---
|
||||
name: noctalia-bar-layout-edit
|
||||
description: "Edit/verify Noctalia v5 bar layout and clock widgets (multi-timezone instances, capsule groups); GUI→repo propagation chain and settings.toml symlink audit on Frank's dotfiles workstations"
|
||||
description: "Edit/verify Noctalia v5 bar layout and clock widgets (multi-timezone instances, capsule groups); GUI-repo propagation chain, settings.toml symlink audit, and stash-clobber recovery on Frank's dotfiles workstations"
|
||||
---
|
||||
|
||||
# Noctalia v5 bar layout edit
|
||||
|
||||
Bar widget layout on Frank's noctalia v5 workstations is GUI-override state, not config.toml.
|
||||
|
||||
## Where
|
||||
- Layout file: `~/.local/state/noctalia/settings.toml` → `[bar.<name>]` keys `start` / `center` / `end` (arrays of widget ids; `[widget.<id>]` subtables hold per-widget settings).
|
||||
- On Frank's boxes this file MUST be a **symlink into the command-center repo** (e.g. `~/.omp/dotfiles/noctalia/.local/state/noctalia/settings.toml`) — edit the repo file, not the link path (atomic-rename writes through the link path would replace the symlink).
|
||||
- Layout file: `~/.local/state/noctalia/settings.toml` -> `[bar.<name>]` keys `start` / `center` / `end` (arrays of widget ids; `[widget.<id>]` subtables hold per-widget settings).
|
||||
- On Frank's boxes this file MUST be a **symlink into the command-center repo** (e.g. `~/.omp/dotfiles/noctalia/.local/state/noctalia/settings.toml`) - edit the repo file, not the link path (atomic-rename writes through the link path would replace the symlink).
|
||||
- Layer order: built-in defaults < `~/.config/noctalia/config.toml` (+`[include]`) < `settings.toml` (wins). Noctalia writes through symlinks and keeps them.
|
||||
- Defaults when a key is absent: `start = ["launcher","wallpaper","workspaces"]`, `center = ["clock"]`.
|
||||
- **Noctalia rewrites this file itself** (GUI actions + normalizes keys/format, e.g. strips `{:%...}` braces, reorders widget lists). Always re-read the exact lines immediately before editing; stale line anchors WILL drift between reads.
|
||||
- Defaults when a key is absent: `start = ["launcher","wallpaper","workspaces"]`, `center = ["clock"]`, `end = ["notifications","clipboard","network","bluetooth","tray","battery","session","control-center","clock"]`.
|
||||
|
||||
## GUI→repo propagation (checkout-is-deployment)
|
||||
- `~/.config/noctalia/config.toml` is symlinked into the repo → GUI settings writes land in the repo working tree automatically.
|
||||
- `~/.local/state/noctalia/settings.toml` can SILENTLY become a plain file (observed 2026-10-17; hardlink/symlink wiring decays) → GUI bar-layout changes then diverge from the repo with no signal. AUDIT: `readlink ~/.local/state/noctalia/settings.toml` must print the repo path; if it's a plain file and repo copy is content-identical, re-arm: `ln -sfn <repo-path> <live-path>` + `noctalia config validate`.
|
||||
- Landing in the working tree ≠ propagated: other boxes only see it after commit + push to Gitea main. After GUI changes, check `git -C ~/.omp status` and commit (Noctalia's runtime churn — wallpaper path, nightlight, panel positions — rides along; normal).
|
||||
- niri has NO settings GUI; niri "GUI changes" don't exist, and `niri msg output ...` runtime overrides are not persisted. niri cfg mirror↔live links also decay (all inodes decoupled once) → audit by CONTENT: `diff` every `~/.config/niri/cfg/*.kdl` against its `dotfiles/zz-blankpad/niri/...` mirror, never trust links.
|
||||
## Clobber window and recovery (2026-10-18 incident law)
|
||||
- GUI-built layout is **working-tree-only until the hourly live-sync heartbeat commits it**. A merge/sync session that "clears the dirty tree" can `git stash` those edits, and later stash operations can destroy the entries - customizations then vanish from the live bar with the repo copy stale.
|
||||
- Never blame the GUI or noctalia first: diff live file vs `HEAD`, check heartbeat commits (`git log -- dotfiles/noctalia/.local/state/noctalia/settings.toml`), then the stash chain.
|
||||
- Destroyed stashes are recoverable: `git fsck --lost-found` lists dangling commits; `git show <dangling>:dotfiles/noctalia/.local/state/noctalia/settings.toml` dumps each candidate; pick by `[bar.default]` richness (capsule groups, extra clock instances). Validate a candidate in a sandboxed `XDG_*_HOME` before touching the live file.
|
||||
- After recovery, commit the file immediately so the window closes.
|
||||
|
||||
## Apply/verify loop (no restart, no reload IPC)
|
||||
1. Edit the repo `settings.toml`.
|
||||
2. `noctalia config validate` (checks merged layers; also `noctalia config export [full] | yq -p toml` to inspect effective values).
|
||||
3. Both layers are file-watched → hot-reload. Prove visually: `noctalia msg screenshot-fullscreen` saves a PNG to `~/Pictures`; read the image and inspect the bar.
|
||||
4. **Screenshot race:** the `ok` return precedes the PNG write. Never chain `noctalia msg screenshot-fullscreen && magick "$(ls -t ~/Pictures/screenshot_*.png | head -1)" ...` in one command — you crop the PREVIOUS capture. List `ls -t` in a separate later call and crop the named file.
|
||||
5. Crop for reading: `magick <png> -crop 100%x8%+0+0 +repage /tmp/strip.png`, then read the strip; cross-check clocks against `date -u` / `date`.
|
||||
## Widgets (v5 census, verified 2026-10-17)
|
||||
- Full roster in `settings.toml` widget_order plus docs: activity, backlinks, battery, bluetooth, brief, bright, cat, cheatsheet, chart, clipby, clipboard, control-center, cpu, controls, cava (now playing), disk, gpu, help, keyboard, language, launcher, mediaplayer, memory, mic, network, night-light, noctalia-palette, notification-history, notifications, niri-colors, power-section, power-profile, screenrecord, scratchpad, screenshot, session, shell (custom shell command), slowouts, slowouts-visible (per-output bar assignment), speedtest, swayosd-*, taskbar, tray, voiceline, volume, wallpaper, weather, workspaces.
|
||||
- `clock` widget: `format` (chrono strftime), `schema` (12/24), `multi_line`, `show_seconds`, `popup`, and **timezone** = IANA name. Multiple clock widget instances are supported: ids like `clock`, `clock-utc`, `clock-sea` each get a `[widget.<id>]` subtable -> one capsule per instance. This is how multi-timezone (Seattle work + UTC dev) lands: add `clock` + `clock-utc` to `end` (or `center`), set `timezone = "Etc/UTC"` on the utc one.
|
||||
- `cat` widget: shows random ASCII art, `rows` setting (4-10).
|
||||
- `shell` widget: `command`, `run interval` (seconds), `icon`, `terminal`; output lands in its capsule popup - can wrap any CLI (hyprstatus-style).
|
||||
- `taskbar` widget: shows window buttons **only on outputs where `niri-bridged` is enabled** (noctalia-launcher enables it). Buttons render **icon-only** (app icon from icon theme; fallback = first letter of title); the window title string is NOT shown in the bar. Hover tooltip carries title + app_name. Per-widget settings: `show_icons`, `show_actions`, `disable_middle_click`, `preferred_app_names` (map app_name -> icon for missing icons).
|
||||
- Widget ids usable in `[bar.*].start|center|end` are restricted to "bar widgets" (`is_bar_widget` allowlist in `bar.rs`); non-bar widget ids are silently dropped at load. `group:<id>` entries render capsule groups.
|
||||
- Capsule groups: `[bar.<name>.capsule_groups]` id = `["widget",...]`.
|
||||
- Niri layout law for bars: window width must stay **odd** (strut = (W-1)/2); bars reserve via layer-shell, niri's reserved area = layer surface size, so bar height changes need no niri edit.
|
||||
|
||||
## Clock widgets
|
||||
- `[widget.clock]` settings: `format`, `vertical_format`, `tooltip_format`, `timezone` (IANA id, e.g. `"UTC"`, `""` = system local). Tokens are strftime-compatible (`%H` 24h, `%I:%M %p` 12h, `%F` ISO date); `{:%...}` brace fields optional; `\n` = second line. Docs: docs.noctalia.dev → bar/widgets/clock + configuration/date-format-tokens.
|
||||
- **One timezone per widget; multiple instances allowed:** any `[widget.<id>]` needs `type = "clock"` for custom ids (bare `clock` id infers it). Each instance converts independently — e.g. local 12h + `timezone = "UTC"` 24h side by side.
|
||||
- Placement: add ids to `[bar.<name>]` start/center/end, or to a capsule group: `[[bar.default.capsule_group]]` with `id = "g1"` + `members = [...]`, referenced from `start` as `"group:g1"` (member order = left-to-right inside the capsule).
|
||||
- Local date + local 12h fit in ONE widget (same tz); the UTC 24h clock must be a second widget.
|
||||
- Expect the date widget (local) and UTC clock to disagree on the day after ~5 PM Seattle — that's correct UTC, not a bug; offer `%F` inside the UTC widget if it irks.
|
||||
## Bar styles (GUI-visible, settings.toml keys)
|
||||
- `[bar.*]`: `thickness`, `font_family`, `font_weight`, `font_size`, `opacity`, `border_width`, `border_offset`, `rounded`, `paddings`, `icon_ratio`, `follow_layout`, `output`, `exclusive`, `layer` (`bottom` = below window layer; `overlay` = above windows), `position` (`top`/`bottom`).
|
||||
- Per-monitor bars: add a `[bar.<name>]` with `output = "<monitor>"` and `follow_layout = true` (niri) -> per-workspace strut follows tall windows on that monitor. This is the mechanism for "bar shrinks when a window is tall" (the bar is a layer-shell strut, NOT a niri layer rule).
|
||||
- The GUI "Bar" page is the reference list; anything absent from settings.toml is a built-in default.
|
||||
|
||||
## Taskbar recipes
|
||||
- Move window indicator beside workspaces: put `"taskbar"` in `start` after `"workspaces"`.
|
||||
- Combine workspaces+windows into ONE widget: `[widget.taskbar] group_by_workspace = true` — per-workspace capsules with disc labels (focused/occupied/empty/urgent colors); `workspace_group_content = "icons"|"count"|"dots"` (count/dots: left-click activates the workspace); `hide_empty_workspaces`, `show_workspace_label`, `minimal`, `workspace_label_placement = corner|centered|inside`. Requires compositor per-window workspace assignment (niri: yes; LabWC: no → stays flat).
|
||||
## Slowouts (window-height-aware bars/panels)
|
||||
- `slowouts` = hidden engine (not a bar capsule): `[slowouts]` keys `enabled`, `duration`, `easing`, `trigger_offset`; per-monitor `[slowouts.<monitor-name>]` overrides (e.g. `DP-1`) with the same keys plus `edge`.
|
||||
- `position = "top"` + `exclusive = false` + slowouts on the monitor -> panel hides when a window touches its edge, re-reveals otherwise. `slowouts-visible` = toggle capsule.
|
||||
- **Slowouts needs niri workspace-autoresize ON** (niri `workspace-autoresize` = "per-workspace" in cfg/layout.kdl `output ... { workspace-autoresize per-workspace }`): noctalia computes hide from workspace height < panel bottom edge; uniform-strut layouts never trigger it.
|
||||
- Per-output bars: slowouts config keys are monitor names, so one `[bar.<x>]` with `output` + `exclusive=false` + `layer=overlay` per slowout monitor is the supported shape.
|
||||
|
||||
## Layout modes and window title decorations
|
||||
- `[layout]`: `mode` (`tiling`|`floating`|`monocle`), `pseudotiled`, `focus_bar` (window-title bar at top, center, layer-shell; style via `[ui.controls...]`/bar-ish settings), `window_follows_cursor`, `gaps`, `border_width`, `rounded`, `blur`, `opacity`, `borderless_csd`, `show_native_decoration`.
|
||||
- `focus_bar` is the noctalia analog of niri's drawn window title, but ONE global bar (no per-window arrows). True per-window title tags = niri `draw-with-border` on window rules; noctalia adds no per-window decoration (it's a bar app + portal, composites nothing).
|
||||
- Floating mode: right-click desktop / `SUPER+Shift+F` / toggle widget; widgets `floating-directories`, `floating-files`, `pins` (sticky notes), `canvas`, `sketchpad`, `post-its`. Float windows snap to screen quarters (`SUPER+Ctrl+Alt+Arrows`) and tile into current layout (`SUPER+Shift+T`). `[layout.floating.files]` = icon/view/sort/preview size.
|
||||
- `[bar.*].window_display` = `disabled`|`icons`|`text`|`combined` - per-window chips in the bar strip (niri only; this is what niri-taskbar could not do). Values: `disabled`|`icons`|`centered`|`inside`. Requires compositor per-window workspace assignment (niri: yes; LabWC: no -> stays flat).
|
||||
- The `workspaces` widget itself has no window display; styles are regular/minimal/focus_hint only.
|
||||
|
||||
## Law
|
||||
- Full widget/setting tables: https://docs.noctalia.dev/noctalia/bar/ and .../bar/widgets/<name>/ . Commit bar changes atomically to the command-center repo (checkout is the deployment).
|
||||
- **Ask before applying.** Frank asks "how do I X" questions about the live bar; answer with the exact snippet + file, apply ONLY on explicit go. A settings.toml edit is a live-deployment change, not a demo.
|
||||
- Git `M` on settings.toml usually carries noctalia's own runtime churn (wallpaper path, nightlight, panel positions) alongside your edit — mention it when committing.
|
||||
- Git `M` on settings.toml usually carries noctalia's own runtime churn (wallpaper path, nightlight, panel positions) alongside your edit - mention it when committing.
|
||||
- Right after building layout in the GUI, commit settings.toml (or trigger `scripts/live-sync.sh`) before any branch/merge work: the uncommitted-GUI-edit window is exactly where the 2026-10-18 clobber happened.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: observability-stack-live-census
|
||||
description: "Census the Dankclaw observability stack (Prometheus/Grafana/Loki/Alloy/Alertmanager/signal-relay) live and audit per-endpoint alert coverage; reconcile against the owner page and rule-file comments, which go stale."
|
||||
---
|
||||
|
||||
# Observability stack — live census + alert-coverage audit
|
||||
|
||||
Use when asked "status of the observability stack" on Dankclaw, or before claiming a service is/isn't monitored. Everything runs rootless on Dankclaw: `ssh dankclaw bash -s <<'EOF' … EOF`. Verify live; the owner page (`serverAppConfigs-docs/Infrastructure/Observability.md`) and `shared/prometheus/rules/llm-fleet.yml` comments go stale.
|
||||
|
||||
## Loopback ports (do NOT probe the wrong one)
|
||||
- Prometheus `127.0.0.1:9090` · Alertmanager `127.0.0.1:9093` · Loki `127.0.0.1:3100` · Alloy `127.0.0.1:12345` · signal-relay `192.168.0.143:18093`
|
||||
- **Grafana host port is `127.0.0.1:3002`** (→ container :3000). Host `:3000` is a `rootlessport` for some *other* rootless container and answers 200-on-`/` but 404-on-`/api/health`. Probing :3000 and seeing 404 is a false alarm — always use 3002 for Grafana. `GF_SERVER_ROOT_URL=https://grafana.franksdank.cloud`.
|
||||
|
||||
## Units + containers
|
||||
`for u in prometheus grafana loki alloy alertmanager node-exporter blackbox signal-relay; do systemctl --user is-active $u.service; systemctl --user is-enabled $u.service; done`
|
||||
- Quadlet units report `generated` (healthy), not `enabled`; signal-relay is a real host unit (`enabled`).
|
||||
- `blackbox` = `not-found`/inactive → the known gap (no blackbox exporter).
|
||||
- `podman ps` for `Up N days (healthy)`.
|
||||
|
||||
## Prometheus — targets / rules / alerts
|
||||
- Targets: `GET :9090/api/v1/targets` → `.data.activeTargets[] | .health, .labels.job, .lastError`. Expect ~8 up / 4 down. **The 4 downs are structural, not outages:** `comfyui` + two `services` (HTML content-type, no real /metrics) and `gitea` (404, metrics disabled). Root remedy = deploy a blackbox exporter (not yet present).
|
||||
- Rules: `GET :9090/api/v1/rules` → count `.data.groups[].rules`. The `llm-fleet` group is **4 alert + 2 recording** (6 total). **AGENTS.md saying "5 alert + 1 recording" is wrong** — trust the rule file.
|
||||
- Firing: `GET :9090/api/v1/alerts`; Alertmanager: `GET :9093/api/v2/alerts` and `/api/v2/silences`.
|
||||
|
||||
## Alert-coverage per runtime — the sharp step (catches real blind spots)
|
||||
The `llm-fleet` perf alerts (`LlmRequestsQueueing`/`LlmPrefixCacheCollapse`/`LlmSpecDecodeOff`) key on the **`llamacpp:*`** family. Which endpoints actually emit it:
|
||||
```
|
||||
GET :9090/api/v1/query?query=count by (job)({__name__=~"llamacpp:.+"})
|
||||
GET :9090/api/v1/query?query=up{job=~"frank-.*"}
|
||||
GET :9090/api/v1/query?query=count by (__name__)({job="frank-archon"}) # what archon really exports
|
||||
```
|
||||
Ground truth (2026-10-20):
|
||||
- **frank-zealot** (llama.cpp) → `llamacpp:*` (17 series) → perf alerts cover it.
|
||||
- **frank-dragoon** (halogen) → still emits `llamacpp:*` (10 series; halogen re-uses the names) → perf alerts cover it.
|
||||
- **frank-archon** (radiance) → exports the **`vllm:*`** family (`vllm:num_requests_waiting`, `vllm:gpu_prefix_cache_hit_rate`, `vllm:num_preemptions_total`, **`vllm:time_to_first_token_seconds_bucket`** = a real latency histogram), **not `llamacpp:*`** → **NO perf alert fires on archon**; only `LlmEndpointDown` (`up{job=~"frank-.*"}`) catches it. Up/down yes, perf/queueing/prefix/p95 = collected but unalerted.
|
||||
- The rule-file header comment claims "frank-archon LEFT the scrape set 2026-10-05, FreeToken exports no /metrics" — **stale/false**: archon (radiance) is `up` and exports `vllm:*`. `LlmEndpointDown`'s description repeats the same stale claim. Never assert coverage from a comment; query the families.
|
||||
|
||||
## Log pipeline emptiness test
|
||||
Loki is "configured, healthy, empty": `GET :3100/loki/api/v1/labels` → `{"status":"success"}` with **no `data` array** = 0 label names = 0 streams. Confirm: `find /opt/loki/data -name '*.tsm' -o -name 'chunks-*' | wc -l` → 0 chunks (≈8 WAL/bookkeeping files only). Alloy ships 0 lines (journal counter 0, both file globs refused per the page — a config bug, not capacity). No Loki datasource in Grafana. Report the log half as non-functional, not merely "healthy."
|
||||
|
||||
## Reporting posture
|
||||
- Metrics half works (up/down for all 3 endpoints + node-exporter + self-scrape); log half dead-on-arrival; blackbox + archon-perf-alerts + Loki-datasource + Alertmanager-vhost are the open gaps.
|
||||
- Grafana provisioning is file-based — check `podman exec grafana ls /etc/grafana/provisioning/{datasources,dashboards}` (expect `prometheus.yml`, `llm-fleet.json`) rather than fighting the API for creds.
|
||||
- Alertmanager has no persistence: a reboot re-stamps `startsAt` on long-firing alerts.
|
||||
- Flag stale facts (AGENTS.md rule split, archon-rejoin comments) but let the user decide whether to fix docs.
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
name: okf-small-bundle-docs-audit
|
||||
description: "Audit + dedupe a small repo's docs bundle for OKF conformance when the kb-lint/kb-coverage toolchain does not exist: inline conformance and link-audit scripts, owner-per-fact cuts, index trio, one log."
|
||||
---
|
||||
|
||||
# OKF docs audit for a small repo bundle (no kb-lint toolchain)
|
||||
|
||||
Use for a repo whose KB is one docs directory (e.g. `dankvoice` → `dankvoice-docs/`), where `scripts/kb-lint.sh` / `kb-coverage.py` do not exist. For the big `dank/command-center` campaign see skill `docs-dedupe-verify-campaign`; same frame, no scouts — a corpus under ~1500 lines gets read fully yourself (scouts lie).
|
||||
|
||||
## Frame
|
||||
- OKF spec lives at `~/.omp/omp-docs/references/okf-spec.md` (the command-center deploy checkout). v0.1 conformance = every non-reserved `.md` has parseable frontmatter with non-empty `type`; `index.md` = listing with NO frontmatter; `log.md` reserved, exactly one, bundle root.
|
||||
- Branch off `origin/main` in a fresh worktree; docs-only + src-comment changes still need the repo's normal gate.
|
||||
|
||||
## Procedure
|
||||
1. Inventory `wc -l` every page; read all fully. Ledger per fact: which page OWNS it (laws page owns laws, decisions own rationale, contributing owns process, README owns user-facing, GLOSSARY owns terms). Everything else becomes a pointer.
|
||||
2. Hunt the rot classes that repeat: dead test citations (name in docs not in `grep -c tests/` — verify with `git log --all -S`), hand-maintained censuses (test counts, scenario lists) — replace with a pointer to the authoritative source, never refresh the copy; deleted doc lines for live env vars (compare `git show <fixed>:file | grep -c VAR` vs HEAD); sections contradicting the page's own head; open items already resolved in live state (check the live settings file, not the doc).
|
||||
3. Frontmatter: `type` + one-line `description` + `tags` + `timestamp`. On edit-rejection the tool prints the file's current `#TAG` — reissue with it, no re-read needed.
|
||||
4. Conformance check (inline python): walk bundle dirs, regex `^---\n(.*?)\n---` then `^type: \S` on each non-reserved `.md`.
|
||||
5. Link audit (inline python): per md file, `\]\(([^)#]+)`; skip http; `os.path.normpath(join(dirname, target))` must exist. Report `N checked / ALL RESOLVE`.
|
||||
6. Create `index.md` at bundle root + each subdir (listing, no frontmatter, relative links) and `log.md` at the bundle root seeded with laws/incidents only.
|
||||
|
||||
## Gotchas
|
||||
- Src docstring fixes are artifact-free only if the bundler strips comments — prove with `bun run build; cmp dist/... old` before claiming no rebuild needed.
|
||||
- README must NOT enumerate design records; point at `decisions/index.md` so new records need no README edit.
|
||||
- Mark superseded decision records in their frontmatter `description` ("its X design is superseded by the Y record") — never rewrite history bodies.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
name: omp-background-job-deadline-debug
|
||||
description: "Diagnose a background bash job/service killed mid-run (\"Command timed out after Ns\") in omp: find which deadline owned it (tools.maxTimeout cap vs async default), prove it from the bundle, and launch long jobs uncapped as ready-log services."
|
||||
---
|
||||
|
||||
# Omp background-job deadline forensics
|
||||
|
||||
Symptom: an async `bats`/build job dies with `[Command timed out after 300s|600s]`
|
||||
while the same launch once ran 18+ min ("no deadline").
|
||||
|
||||
## Cause map (proven against the pinned bundle)
|
||||
Resolver in `dist/cli.js`: `let v=n??300, R=v===0, k=R?void 0:ep("bash",v,x)` where
|
||||
`qx.bash={default:300,min:1,max:3600}` and `x` = config key `tools.maxTimeout`
|
||||
("Maximum timeout in seconds the agent can set for any tool", default 0).
|
||||
- Request absent → `qx.bash.default` 300 s.
|
||||
- Request > 0 → clamped by `tools.maxTimeout` when set (e.g. 600 caps a 1800 request).
|
||||
- Request 0 → meant "no deadline", but `0` sometimes arrives as missing (falsy drop
|
||||
upstream of the resolver) → silently 300 s. Filed as tool issue; treat any
|
||||
async job over ~5 min as untrustworthy.
|
||||
|
||||
## Diagnose in three moves
|
||||
1. `grep -o 'qx={bash:{[^}]*}' <omp bundle>` — policy table (`command -v omp` →
|
||||
`readlink -f` → `dist/cli.js`).
|
||||
2. `read cfg://tools` — live `maxTimeout` (value ≠ 0 explains any hard cap).
|
||||
3. Compare kill notices: the number in "killed once it has run Ns" is the resolved
|
||||
deadline; 300 = default reached (request dropped), other values = your request
|
||||
or the config cap.
|
||||
|
||||
## Launching jobs longer than the cap
|
||||
- Named **services are not deadline-capped**: `bash(name:"x", pty:false, command:"…; echo DONE >> log", ready:{log:"DONE", timeout:900})`.
|
||||
Readiness fires on the log pattern; result does NOT auto-deliver — `wait` on it or read `proc://x` after the supervisor notice.
|
||||
- NEVER end the service command with `echo "EXIT $?" >> log` and trust the
|
||||
supervisor's exit code — the trailing echo masks it (supervisor reports 0 while
|
||||
the log says EXIT 1). The log line is the truth; grep the log for the recorded
|
||||
code and `^not ok` before believing any "exit code 0".
|
||||
- Do not raise `tools.maxTimeout` unasked: session-scoped `write cfg://tools/maxTimeout`
|
||||
needs user approval and may be declined — respect it, use the service route.
|
||||
|
||||
## Multi-suite hazard
|
||||
Suites that rebuild shared artifacts per test (e.g. dankvoice bats rebuilds `dist/`)
|
||||
must not run concurrently — one suite at a time, or per-worktree copies only.
|
||||
@@ -0,0 +1,38 @@
|
||||
---
|
||||
name: omp-bundle-implementation-port
|
||||
description: "Extract and port a working implementation from omp's minified dist/cli.js bundle (copy-not-invent requests); anchors, python slicing, regex-constant sweep, trace-driven test prediction."
|
||||
---
|
||||
|
||||
# Porting built-in omp behavior out of the minified bundle
|
||||
|
||||
Use when Frank says "copy the omp implementation / work off known-working code" — i.e. port a built-in feature (vocalizer segmentation, retry logic, formatting rules) into our own file instead of hand-rolling a heuristic. For merely *auditing behavior claims*, use `omp-bundle-claim-audit`; this skill is extraction-for-porting.
|
||||
|
||||
Bundle: `~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js` (minified; single-letter names, private `#x` fields survive).
|
||||
|
||||
## Extraction recipe
|
||||
|
||||
1. **Find the setting/entry strings first.** `grep -o 'ce({id:"<area>[^"]*"[^}]\{0,200\}' cli.js` — setting descriptions state semantics verbatim (e.g. `speech.enhanced` = "Rewrite assistant output into natural spoken prose with the tiny/smol model"). Descriptions are quotable ground truth.
|
||||
2. **Anchor to code with a distinctive token**, not guesses: a method name (`pushDelta`), a state init, or a string. Then slice generously with python (bash heredoc), NOT grep -o which truncates:
|
||||
```python
|
||||
src = open('cli.js').read()
|
||||
i = src.index('pushDelta'); print(src[i-1500:i+3000])
|
||||
```
|
||||
3. **Follow the constructor**: pipeline classes instantiate collaborators there (`this.#t ??= new fHe` → `fHe` is the real worker; `#g()` returned `{speakable:new fHe, pending:[]…}`). Dump `class fHe{` +4000 chars.
|
||||
4. **Regex/helper sweep**: minified code declares shared constants in a trailing `var a,b,c,…` list and initializes them elsewhere (`Xyr=`, `Pwc=function`). Loop the names with `re.search(rf'{name}=', src)` and print ~400 chars each — the actual boundary/cleanup rules live in these one-liners.
|
||||
5. Write slices to `/tmp/*.txt` and `read` them; don't eyeball megabyte greps.
|
||||
|
||||
## Port rules
|
||||
|
||||
- Rename fields/methods to readable names but keep logic **1:1** — the value is behavioral fidelity (Frank's "known-working set of code").
|
||||
- Private `#x` fields port cleanly: TS classes in Bun extensions support `#`.
|
||||
- Global-regex `.lastIndex=0` before each scan loop — minified code relies on it; keep it.
|
||||
|
||||
## Verification shortcut
|
||||
|
||||
Before running tests, **hand-trace** each test input through the ported rules (boundary indices, state transitions) and predict exact segment counts/strings; then run — predicted-vs-actual mismatch localizes port errors faster than test output alone. Worked example (2026-10-19, dankvoice PR #26): traced `"First stop here. Second one goes!!! Really? Third ... done"` → predicted 3 segments incl. `"Third ... done"` merged via flush (10-char boundary < 24-char started-min) → bats green first run.
|
||||
|
||||
## Known anchors (2026-10-19 bundle)
|
||||
|
||||
- Speak-aloud vocalizer pipeline: class `I6o` (gating on `speech.enabled`, suspend counter, enhancer, `#g()` per-message state).
|
||||
- Utterance segmenter: class `fHe` + helpers `Pwc` (line-head classify), `Iwc`/`_wc`/`b6o`/`w6o` (boundary search), `Cwc`+`y6o` (cleanup: links→label, URL→host, path→basename). Regex consts: `Xyr` sentence-end runs, `pHe` weak breaks, `mwc` abbreviations, `hwc` undecided line-heads, `g6o` horizontal rules.
|
||||
- Port already living in `agent/extensions/dankvoice.ts` (`SpeechSegmenter`) — when the bundle updates, re-extract `fHe` and diff against it before assuming our copy still matches.
|
||||
@@ -0,0 +1,73 @@
|
||||
---
|
||||
name: omp-extension-event-probe
|
||||
description: "Prove which lifecycle events the omp harness actually dispatches to extensions, with a throwaway timestamping probe plugin, instead of inferring delivery from side effects."
|
||||
---
|
||||
|
||||
# Prove harness→extension event delivery with a probe plugin
|
||||
|
||||
Use when an extension handler seems never to fire, or when you must know *when* an
|
||||
event arrives relative to another (mid-run vs settle). Never infer dispatch from
|
||||
the feature's side effects: async work in the handler (LLM rewrites, synthesis,
|
||||
network) lags seconds and destroys the timing signal.
|
||||
|
||||
## Shape (no build step needed)
|
||||
|
||||
`/tmp/probe/package.json` — the installer validates the declared entry on disk:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "event-probe",
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"exports": "./index.js",
|
||||
"pi": { "extensions": ["./index.js"] }
|
||||
}
|
||||
```
|
||||
|
||||
`/tmp/probe/index.js` — append arrival times, not counts:
|
||||
|
||||
```js
|
||||
import { appendFileSync } from "node:fs";
|
||||
const LOG = "/tmp/probe.log";
|
||||
const stamp = (line) => appendFileSync(LOG, `${new Date().toISOString().slice(11, 23)} ${line}\n`);
|
||||
export default function probe(pi) {
|
||||
pi.on("agent_start", () => stamp("agent_start"));
|
||||
pi.on("turn_end", (e) => stamp(`turn_end turn=${e.turnIndex} toolResults=${(e.toolResults ?? []).length}`));
|
||||
pi.on("agent_end", (e) => stamp(`agent_end messages=${(e.messages ?? []).length} willContinue=${String(e.willContinue)}`));
|
||||
}
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
1. `cd ~/.omp/plugins && omp plugin link /tmp/probe` (link, never edit
|
||||
`node_modules`), then `omp plugin doctor` → expect a `✔ plugin:event-probe` line.
|
||||
2. Launch a **fresh** TUI session in a pty (`omp --cwd /tmp/probe-run`, `pty: true`).
|
||||
Extensions load at startup only — an already-running session never sees the probe.
|
||||
3. PTY gotcha: input sent before the splash clears is swallowed. Read the screen
|
||||
first, then submit text and finish with a separate write containing a bare
|
||||
`\r` (LF alone left the text sitting in the composer).
|
||||
4. Drive a task with ≥2 tool rounds so turns and settles separate in time
|
||||
(sequential reads work and need no approval; ask for steps "strictly one
|
||||
after the other" or the model parallelises them into one turn).
|
||||
5. Wait for a terminal line with an async bounded loop (`grep -q agent_end`,
|
||||
`sleep 1` inside one backgrounded bash call) — never poll across turns.
|
||||
|
||||
## Read
|
||||
|
||||
`turn_end` timestamps strictly earlier than `agent_end` ⇒ per-turn delivery
|
||||
works. `willContinue=true` on `agent_end` marks a non-terminal settle (harness
|
||||
maps `isTerminal === false` to it) — handlers that `return` on it never speak.
|
||||
|
||||
## Teardown (mandatory)
|
||||
|
||||
`omp plugin uninstall event-probe`; `omp plugin doctor`; delete `/tmp/probe*`.
|
||||
The link writes into `~/.omp/plugins/package.json` (tracked, live tier) —
|
||||
uninstall restores it; confirm with `grep` for the probe name returning nothing.
|
||||
|
||||
## Ground truth read for free
|
||||
|
||||
Event union + payloads: `~/.omp/plugins/node_modules/@oh-my-pi/pi-coding-agent/dist/types/extensibility/shared-events.d.ts`;
|
||||
the session→extension mapping lives in `src/extensibility/extensions/lifecycle-mirror.ts`
|
||||
(`extensionEventFromSessionEvent`). Extension-level `turn_end` carries
|
||||
`turnIndex`/`message`/`toolResults` but **no `willContinue`** — that field exists
|
||||
only on `agent_end`, so "is this the last turn" cannot be decided from `turn_end`.
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
name: omp-extension-live-tui-verify
|
||||
description: "Verify an omp extension's slash commands and event hooks against a real TUI session: pty submit semantics, observable side-effect instrument, transient-toast trap, agent_end willContinue law, PipeWire output triage."
|
||||
---
|
||||
|
||||
Verify that a `.ts` file in `agent/extensions/` actually works inside the interactive TUI — the black-box bats seam cannot see harness-side wiring (event names, arg passing, mode gating), and it will pass beside a dead hook.
|
||||
|
||||
Distinct from `omp-tui-live-visual-verify` (config/theme rendering proven by screenshot): here the instrument is **observable side effects**, not pixels, because extension notifications are transient toasts that vanish before the pty buffer is read.
|
||||
|
||||
## 1. Launch the exact build under test
|
||||
|
||||
```bash
|
||||
omp --no-extensions --no-session -e /abs/path/agent/extensions/<name>.ts
|
||||
```
|
||||
as a **pty service** (`pty: true`, `ready: {"log": "╭|❯|Tip:|─", "timeout": 60}`).
|
||||
`--no-extensions` + explicit `-e` proves the file loads by path and nothing else interferes. Neutral cwd (`/tmp`) so repo state cannot color the run.
|
||||
|
||||
## 2. Submit input correctly (the costly gotcha)
|
||||
|
||||
`write proc://<id>` with a trailing `\n` does **not** submit — the text just sits in the composer.
|
||||
|
||||
- **Works:** two writes — first `"/cmd args"` (no terminator), then a **separate** write whose whole content is `\r`.
|
||||
- **Broken:** `"/cmd args\r"` in one write. It still submits, but the trailing `\r` arrives inside the command args. Symptom: `unknown voice 'narrator1\r'`, an unexpected extra utterance, or a chat message whose transcript line ends in a literal `\r`. That is a harness artifact — never "fix" it by stripping control characters in the extension (`.trim()` already covers what a keyboard can produce).
|
||||
|
||||
## 3. Instrument with a stub, never with ears or eyes
|
||||
|
||||
Point the extension's player/handler at a script that appends evidence:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
echo "$1" >> /tmp/stub.log; cp "$1" "/tmp/out-$$.wav"; exit 0
|
||||
```
|
||||
Run the TUI with that stub in the env the extension reads (`DANKVOICE_PLAYER=…` style). Then every assertion is a file: `wc -l` of the log (how many times it fired), `ffprobe -show_entries format=duration` (what it produced), `ls -d <tmp-glob>` (that cleanup ran).
|
||||
|
||||
Sequence per behavior: command → CR → short `sleep` → count. **Count before and after** each command; the delta is the only honest proof a subcommand stayed silent.
|
||||
|
||||
## 4. Discriminate "fired twice" from "fired once per turn"
|
||||
|
||||
Stray CR writes can create extra empty turns, each with its own `agent_end` — two utterances for one intended prompt is usually **two turns**, not a duplicate speak. Confirm by counting assistant turns in the transcript (`read proc://<id>`), and by matching each clip's duration to its own answer's length.
|
||||
|
||||
## 5. Event-hook contract traps
|
||||
|
||||
- `agent_end` payload (`dist/types/extensibility/shared-events.d.ts`): `{ type, messages: AgentMessage[], willContinue? }`. **`willContinue: true` means the harness already scheduled a continuation** (auto-retry / empty-stop retry) — a subscriber that narrates or notifies there speaks an answer the model is about to redo. Gate on it.
|
||||
- Read the real event/payload shape from `~/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/types/…`; grep `dist/cli.js` for the literal event names to confirm they exist.
|
||||
- A driver test that *calls* the handler by hand proves nothing about emission — that is exactly the gap this live check closes.
|
||||
|
||||
## 6. Audio output goes silent while every command reports success
|
||||
|
||||
`pw-play` exits 0 and prints `stream drained` even when nobody hears anything. Triage order:
|
||||
|
||||
```bash
|
||||
wpctl status # Sinks, the `*` default, and live Streams
|
||||
wpctl get-volume @DEFAULT_AUDIO_SINK@
|
||||
```
|
||||
- A **`Default Configured Devices` entry naming a `bluez_output.*` node that is not in the Sinks list** = remembered-but-offline Bluetooth: the classic silent-route cause.
|
||||
- Confirm the stream really links: a short clip can vanish before inspection, so use a multi-second clip and read `wpctl status` mid-play, or target explicitly: `pw-play --target <node-id> <file>` (node ids from `wpctl status`; `wpctl inspect <id>` gives `node.name`).
|
||||
- Prove the synth path independently of the player: `curl -o x.wav …` then `file x.wav` (expect `RIFF … WAVE … PCM`).
|
||||
|
||||
## 7. Land it
|
||||
|
||||
Live result belongs in the commit message, the reference page, and the work log — and the guard the live check uncovered gets its own behavior test (assert observable output only: fired count, payload content, silence).
|
||||
|
||||
## Cleanup
|
||||
|
||||
`/exit` or `proc://<id>/kill` the service; move stub scripts, logs, and clips to `~/.AGENTTRASH/<date>-<topic>/` — never `rm`. Kill background jobs that keep playing to the operator's speakers.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
name: omp-extension-stale-module-audit
|
||||
description: "Diagnose omp extension misbehavior (handler timeouts/silence, cross-instance arbitration failures like simultaneous speech) when the on-disk fix is already deployed: running sessions hold pre-upgrade in-memory bundles; compare ps lstart against artifact mtimes and merge dates."
|
||||
---
|
||||
|
||||
# Stale in-memory omp extension bundles
|
||||
|
||||
Symptom class: an extension behaves as if unfixed even though the disk artifact
|
||||
contains the fix — recurring handler timeouts, silence, or (multi-instance
|
||||
coordination features) arbitration failures that look like code bugs. Root
|
||||
cause: **omp loads the bundle at session start; `omp plugin upgrade` on disk
|
||||
does not touch the code already running in live sessions.**
|
||||
|
||||
## Recipe 1: date the sessions against the artifact
|
||||
|
||||
```bash
|
||||
date
|
||||
stat -c '%y %n' ~/.omp/plugins/node_modules/<pkg>/dist/index.js # last upgrade
|
||||
ps -eo pid,lstart,etime,args | grep 'omp' | grep -v grep # each session's boot
|
||||
```
|
||||
|
||||
Any session whose START predates the artifact mtime holds an older bundle —
|
||||
for it, the "fix" never shipped. When the feature landed in git, compare
|
||||
against the merge date, not the last upgrade (`git log --format='%h %cI %s'
|
||||
main | grep -i <feature>`): a session can predate the feature *existing*.
|
||||
|
||||
Fix is always: restart (`/quit`, then `omp --resume <id>`) — resume is
|
||||
lossless. It is the user's live session; never kill it yourself.
|
||||
|
||||
## Recipe 2: cross-process arbitration — a bypasser explains "simultaneous X"
|
||||
|
||||
When a feature serializes across sessions via files (e.g. dankvoice stage:
|
||||
`~/.cache/dankvoice/stage/` with `lease.json` = `{holder,pid,expires}`, one
|
||||
`<ISO>-<session>.json` entry per queued item, FIFO = filename order, lease TTL
|
||||
6 s), overlap between two actors proves **at least one actor never consults
|
||||
the directory** — the mechanism can't serialize against a bypasser. Census the
|
||||
queue live (`watch -g -d 'ls <dir>; cat <dir>/lease.json'`): a live, renewing
|
||||
lease proves the new sessions coordinate; then Recipe 1 finds the bypasser.
|
||||
|
||||
Known residual hazard to note, not silently fix: a lease heartbeat is a timer
|
||||
on the holder's own event loop; a stall longer than the TTL lets a peer steal
|
||||
the lease mid-action. Mitigation = pid-liveness check in the steal path.
|
||||
|
||||
## Verification
|
||||
|
||||
After restarts, the symptom must vanish while the queue shows traffic. If it
|
||||
persists, look for actors on OTHER hosts (file arbitration is per-box), or
|
||||
the TTL-steal hazard above.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: omp-lifecycle-event-selection
|
||||
description: "Pick the right omp harness lifecycle event for narration/telemetry: agent_end is a settle, turn_end waits for tool results, message_end is earliest with content, after_provider_response carries no content."
|
||||
---
|
||||
|
||||
# Choosing an omp lifecycle event (narration, telemetry, hooks)
|
||||
|
||||
Re-derived twice from the installed typings/sources (`@oh-my-pi/pi-coding-agent`, `@oh-my-pi/pi-agent-core`) while fixing dankvoice. Read this before wiring a handler; verify against the installed version's `src/extensibility/extensions/types.ts` and `pi-agent-core/src/agent-loop.ts` when precision matters.
|
||||
|
||||
| event | payload you can use | when it fires | traps |
|
||||
|---|---|---|---|
|
||||
| `after_provider_response` | `status`, `headers`, `requestId`, `metadata` **only** | response headers arrive, **before the stream body is consumed** | No message content at all: useless for anything that speaks or summarizes text. Right for latency telemetry, non-200/abort detection, `requestId` correlation. |
|
||||
| `message_end` | full `message` (assistant, user, tool-result, gate-stop, error) | the moment that message is built (`agent-loop.ts`, assistant case) | Earliest event carrying assistant text. Fires for non-assistant messages too → gate `message.role === "assistant"`. Payload is cloned for extensions. |
|
||||
| `turn_end` | `turnIndex`, `message`, `toolResults` | after that turn's **tool results** come back | Stale by the tool duration: a 14 s tool delays a `turn_end` trigger 14 s. Extension event has **no `willContinue`** (the `turnIndex` is numbered by the lifecycle mirror), so you cannot tell the terminal turn from here. |
|
||||
| `agent_end` | `messages`, `willContinue`, `awaitingAsyncWork` | once per user prompt, when the loop settles | `willContinue: true` = the harness scheduled a continuation (auto-retry / empty-stop), not a user-visible settle — narrating it speaks an answer the model is about to redo. Silence-prone on long-running sessions. |
|
||||
| `agent_start` | — | run begin | Use to reset per-run dedupe state, so a later answer with identical text is not swallowed. |
|
||||
|
||||
## Narration cadence pattern (what dankvoice ships)
|
||||
|
||||
- `turn_end` (or `message_end` if the tool-time lag matters) for mid-run commentary; `agent_end` only for the settled answer.
|
||||
- Drop a mid-run trigger while audio/IO is busy — never queue, never interrupt.
|
||||
- Dedupe the terminal text per run (`agent_start` resets), because the last turn and the settle carry the same string.
|
||||
- Skip `ctx.mode === "print" | "json"` so benches stay silent.
|
||||
|
||||
## Prove delivery, don't infer it
|
||||
|
||||
Audio/UI timing cannot show which events arrive. Link a throwaway extension that appends timestamps per event and run one real session: `omp plugin link /tmp/probe` (package.json with `"pi": { "extensions": ["./index.js"] }`), prompt a task with a slow tool, read the log, then `omp plugin uninstall probe`. Confirmed live 2026-10-19: extensions receive `agent_start`, `turn_end`, `agent_end`; `turn_end` landed seconds before `agent_end`.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
name: omp-local-artifact-path
|
||||
description: Resolve a local:// artifact to a real filesystem path (for scripts/curl) without guessing the session-slug directory.
|
||||
---
|
||||
|
||||
`local://<name>.md` artifacts live in a per-cwd, per-session directory whose name you cannot guess. Three invented paths failed before this worked (2026-10-19).
|
||||
|
||||
Actual layout:
|
||||
|
||||
```
|
||||
~/.omp/agent/sessions/<cwd-slug>/<timestamp>Z_<session-id>/local/<name>.md
|
||||
```
|
||||
|
||||
`<cwd-slug>` is the working directory with `/` replaced by `-`, **keeping the leading dash** — `/home/blank/.omp/wt/wt-20261019-125509-7c8a99d` under `/home/blank` becomes `-.omp-wt-wt-20261019-125509-7c8a99d`. Dropping the leading `-.` is the mistake that produces `FileNotFoundError`.
|
||||
|
||||
## Get the real filesystem path
|
||||
|
||||
Ask the tool, don't reconstruct it — a ranged read of the artifact reports the resolved path in its header:
|
||||
|
||||
```
|
||||
read local://pr-body-msgend.md:1-1 -> [~/.omp/agent/sessions/-.../local/pr-body-msgend.md#TAG]
|
||||
```
|
||||
|
||||
Or locate it by basename (needs enough depth; `maxdepth 5` misses it — the file sits 6–7 levels down):
|
||||
|
||||
```bash
|
||||
f=$(find ~/.omp/agent/sessions -maxdepth 7 -name 'pr-body-msgend.md' | head -1)
|
||||
cp "$f" /tmp/prbody.md # stable path for the python/curl step
|
||||
```
|
||||
|
||||
Copy to `/tmp` before handing the content to a script: one cheap command, and every later tool call stops depending on session-scoped paths.
|
||||
|
||||
## Related lookups in the same directory tree
|
||||
|
||||
- Session id for the current cwd — `~/.omp/agent/history.db` (prompts table has `session_id`, `cwd`; the column is `prompt`, **not** `text`, and raw `?q=` queries cannot be combined with a table selector or pagination):
|
||||
```
|
||||
read history.db?q=SELECT session_id, cwd, substr(prompt,1,50) AS p FROM history WHERE cwd LIKE '%wt-20261019%' ORDER BY rowid DESC LIMIT 3
|
||||
```
|
||||
Use the real id in issue/PR attribution footers; never reuse an id remembered from a compacted transcript.
|
||||
- Transcript dir for the same session sits beside `local/`.
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
name: omp-models-yml-runner-api-placement
|
||||
description: "Add or edit a non-chat (tts/image/embedding) model row in ~/.omp/agent/models.yml, wire the speech role, and diagnose a failing speech device without blaming a healthy endpoint"
|
||||
---
|
||||
|
||||
# Scope
|
||||
|
||||
`~/.omp/agent/models.yml` (or `models.yaml`) on a workstation: adding a **runner**
|
||||
model row (TTS / image / embedding / rerank / video) alongside chat providers,
|
||||
flipping `config.yml` `modelRoles` at it, and diagnosing a speech device that
|
||||
refuses to speak.
|
||||
|
||||
# The landmine
|
||||
|
||||
Provider-level `api` accepts **only chat/judge transports**:
|
||||
`openai-completions`, `openai-responses`, `openai-codex-responses`,
|
||||
`azure-openai-responses`, `anthropic-messages`, `bedrock-converse-stream`,
|
||||
`google-generative-ai`, `google-gemini-cli`, `google-vertex`,
|
||||
`openrouter-decisions`, `typesafe`.
|
||||
|
||||
Naming a runner API (`openai-speech`, `openai-images`, `openai-embeddings`, …)
|
||||
at **provider** level fails whole-file validation, and the failure mode is not
|
||||
scoped to your provider:
|
||||
|
||||
```
|
||||
Warning: models.yml validation failed — custom providers disabled
|
||||
Failed to load config file models, Schema error: providers.<id>.api: must be "openai-completions", …
|
||||
```
|
||||
|
||||
Every custom provider disappears from the registry (fleet chat endpoints stop
|
||||
resolving) and `omp models` lists only built-in locals. Measured 2026-10-18.
|
||||
|
||||
# Correct shape
|
||||
|
||||
Runner api goes on the **model entry only**; `auth: none` satisfies the
|
||||
credential requirement; omit `contextWindow`/`maxTokens` (the positivity check
|
||||
fires only when they are provided — `omp models` still prints filled defaults,
|
||||
which the runner transport never reads).
|
||||
|
||||
```yaml
|
||||
"frank-tts (dankclaw)":
|
||||
baseUrl: http://192.168.0.143:7772/v1 # no `api:` here
|
||||
auth: none
|
||||
models:
|
||||
- id: frank-tts
|
||||
name: "frank-tts (dankclaw) — cloned voices"
|
||||
api: openai-speech # runner api on the model
|
||||
input: [text]
|
||||
```
|
||||
|
||||
Roles live in `config.yml` (`modelRoles`), not in `models.yml`. Kind roles
|
||||
(`image`, `web`, `speech`, `dictation`, `judge`) take no `:thinking` suffix.
|
||||
|
||||
# Verify after every edit (never assume the file loaded)
|
||||
|
||||
```bash
|
||||
omp models --kind tts --no-extensions # the new row must appear
|
||||
omp models --no-extensions | grep -E '<each fleet chat model>' # and still be listed
|
||||
omp models | grep -c 'validation failed' # must be 0
|
||||
```
|
||||
|
||||
A run that lists only built-ins means your row broke the file — revert before
|
||||
yielding.
|
||||
|
||||
# Speech transport contract (`openai-speech`)
|
||||
|
||||
Body is `{model, input, response_format, voice?, speed?}`; `voice` is forwarded
|
||||
**only** when the caller passed `voice_id`; `sample_rate`/`bit_rate` **throw**
|
||||
(`openai-speech does not support sampleRate`). Codec comes from the
|
||||
`output_path` suffix; 15 000-character cap. An unknown `voice` name is a server
|
||||
error, not a fallback — and `auth: none` makes a dead endpoint read as
|
||||
*available*, so point `modelRoles.speech` at such a provider only after it has
|
||||
answered a real synthesis.
|
||||
|
||||
# Three speech-role laws (bundle-read + measured 2026-10-18)
|
||||
|
||||
1. **No setting carries a voice for `openai-speech`.** The transport builds the
|
||||
body with `voice` from the call's `voice_id` and nothing else.
|
||||
`tts.localVoice` is the *local Kokoro backend's* key; `speech.voice` is the
|
||||
TUI's **speak-aloud** voice, an enum of Kokoro names that never reaches an
|
||||
HTTP request (setting it to a pack name breaks the vocalizer and does nothing
|
||||
for synthesis — read `cfgSpeechVoice` in `dist/cli.js` before believing
|
||||
otherwise). So a voice-less call is the harness's **normal** shape: a
|
||||
voice-required endpoint needs a server-side default (fleet answer:
|
||||
`DEFAULT_VOICE` in `scripts/frank-tts.env`, journal marks the line
|
||||
`(default)`, unknown names still 400 with the list).
|
||||
2. **An explicit role has no fallback candidate.** `modelRoles.speech` set to one
|
||||
model yields exactly one candidate, so a dead endpoint or an unresolvable
|
||||
selector fails with *"No available speech model matches the speech role"*
|
||||
instead of degrading to Kokoro. Loud-by-design; do not "fix" it by adding a
|
||||
fallback in the client.
|
||||
3. **`models.yml` loads at process start; `config.yml` is read live.** A session
|
||||
older than the new row fails exactly like a dead endpoint while a fresh
|
||||
process succeeds in the same minute.
|
||||
|
||||
# Diagnosing "speech device won't speak" — in this order
|
||||
|
||||
1. `curl` the endpoint directly with a real voice name. 200 + audio ⇒ the service
|
||||
is fine and the fault is harness-side; stop touching the server.
|
||||
2. Control test in the *same* session: point `modelRoles.speech` at
|
||||
`local/kokoro` and call the device. It works ⇒ `config.yml` is live-read and
|
||||
the runner selector or its voice argument is the problem, not the endpoint.
|
||||
Revert the role immediately after.
|
||||
3. Distinguish stale registry from dead endpoint with a fresh process:
|
||||
`omp -p --no-session --no-extensions --no-skills --no-rules --tools tts --cwd /tmp
|
||||
'Call the tts tool exactly once with text="…" output_path=/tmp/x.wav
|
||||
voice_id=<name> — then reply with the tool result line.'`
|
||||
Succeeds ⇒ the long-lived session predates the `models.yml` row; restart it.
|
||||
4. Cross-check server-side, never trust the client's echo: the façade/access log
|
||||
line and the engine's own ledger must carry the same byte count the tool
|
||||
printed. (`model=…` in a tool result is what the client *requested*; a server
|
||||
line is the fact.)
|
||||
5. For harness-behaviour claims, read the bundle rather than reasoning from the
|
||||
tool description: `/home/blank/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`,
|
||||
e.g. `grep -o -E '.{300}response_format:[a-zA-Z$]+.{250}'` for the speech body
|
||||
builder, `grep -o -E 'cfgSpeechVoice:.{0,400}'` for what a settings key feeds.
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
name: omp-search-provider-fallback-debug
|
||||
description: "Diagnose why the omp harness web_search silently uses a fallback provider (e.g. Kagi) instead of the configured primary (SearXNG): the fallbackChains masking pattern, the searxng.endpoint base-URL scheme/guard-chain trap, and the cli.js + content-type probes that prove it."
|
||||
---
|
||||
|
||||
# omp web_search silently using the wrong provider
|
||||
|
||||
Symptom: agent searches return results from a fallback provider (e.g. **Kagi**) even though
|
||||
`modelRoles.web` is set to the intended primary (e.g. `web/searxng`). The primary errors
|
||||
every time and the failure is invisible because a fallback answers successfully.
|
||||
|
||||
## The masking mechanism (always check this first)
|
||||
|
||||
`~/.omp/agent/config.yml`:
|
||||
- `modelRoles.web:` is the primary (e.g. `web/searxng`).
|
||||
- `retry.fallbackChains.web:` is an ordered list tried when the primary throws. The FIRST
|
||||
entry silently serves whenever the primary errors. In this fleet it is `web/kagi`, so any
|
||||
SearXNG error == Kagi answers and you never see an error.
|
||||
|
||||
So "searches go through Kagi" almost always means "SearXNG is failing", not a routing mistake.
|
||||
|
||||
## The SearXNG endpoint trap (the usual root cause here)
|
||||
|
||||
`searxng.endpoint` in config.yml must be a **full base URL pointing at the direct JSON port**,
|
||||
used **verbatim** by omp (no scheme auto-added, no normalize — confirmed in `cli.js`). Two ways
|
||||
it breaks:
|
||||
|
||||
1. **Bare host** (e.g. `search.franksdank.cloud`, no scheme): omp does `new URL(\`${endpoint}/search\`)`
|
||||
→ **throws** immediately (URL needs a scheme). SearXNG never contacted.
|
||||
2. **The guarded vhost** (`https://search.franksdank.cloud`): returns Anubis' **"Making sure
|
||||
you're not a bot!"** challenge page at HTTP **200 `text/html`**. omp calls `o.json()`
|
||||
unconditionally regardless of content-type → throws on the leading `<`. There is
|
||||
deliberately NO API path through the guard chain (JS challenge + UA denylist).
|
||||
|
||||
**Correct value** (direct published port, http scheme preserved — reachable from the LAN only):
|
||||
|
||||
```yaml
|
||||
searxng:
|
||||
endpoint: http://192.168.0.143:8081 # Dankclaw; NOT the vhost, NOT schemeless
|
||||
```
|
||||
|
||||
The published port source (192.168.x/loopback/LAN) is in SearXNG's `pass_ip`, so it bypasses
|
||||
the JSON rate limiter (hardcoded `API_MAX=4`/IP/hr, not settable except via `pass_ip`).
|
||||
|
||||
## Prove it (don't guess)
|
||||
|
||||
1. Content-type probe, both paths — the decisive check:
|
||||
```
|
||||
curl -sk -o /tmp/vh -w "vhost ct=%{content_type}\n" "https://search.franksdank.cloud/search?q=test&format=json" # expect text/html (Anubis)
|
||||
curl -s -o /tmp/dp -w "direct ct=%{content_type}\n" "http://192.168.0.143:8081/search?q=test&format=json" # expect application/json
|
||||
```
|
||||
vhost = `text/html` + direct = `application/json` confirms the trap.
|
||||
2. Code path, in the installed omp dist (NOT a worktree):
|
||||
`/home/blank/.bun/install/global/node_modules/@oh-my-pi/pi-coding-agent/dist/cli.js`
|
||||
(find install: `dirname $(readlink -f $(which omp))`). Grep the SearXNG provider:
|
||||
- `searxng.endpoint` config def → description "Base URL of a self-hosted SearXNG instance".
|
||||
- the request builder `...=e.replace(/\/+$/,""),new URL(\`${r}/search\`)...` → endpoint is
|
||||
verbatim, scheme NOT added.
|
||||
- `X2c`/`J2c` → `if(!o.ok)throw` then `return await o.json()` → HTML 200 passes `.ok` and
|
||||
blows up in `.json()`.
|
||||
|
||||
## Apply + verify
|
||||
|
||||
- Fix is a harness config edit (`cfg://searxng/endpoint` prompts the user, or edit config.yml).
|
||||
Confirm with the user first — it is their live harness config.
|
||||
- Verification after applying: fire a real `web_search` and confirm it returns
|
||||
`provider:"searxng"` (not `"searxng"`-then-fallback / not `"kagi"`).
|
||||
|
||||
## Related gotcha (same masking, different cause)
|
||||
|
||||
If SearXNG answers JSON but returns empty `results` with `unresponsive_engines` (VPN-exit-IP
|
||||
CAPTCHA/429 on google/brave), omp's `vcs` throws a 503 → Kagi again. That's an engine-health
|
||||
problem, fix in `[[SearXNG]]`/`[[SearXNG Providers]]`, not the endpoint.
|
||||
|
||||
## Hardening option
|
||||
|
||||
Drop or reorder `web/kagi` / `web/perplexity` in `retry.fallbackChains.web` so primary failures
|
||||
surface instead of silently spending a paid Kagi credit. Flag to user before editing.
|
||||
@@ -1,49 +1,22 @@
|
||||
---
|
||||
name: omp-worktree-path-gotchas
|
||||
description: "Use in ~/.omp (or any multi-worktree repo) when an edit lands in the wrong checkout, git checkout is policy-refused, or an omp session id is needed — relative-path tool resolution, shared-worktree branch law, session-id lookup, edit auto-prefix trap."
|
||||
description: "Multi-worktree edits/tests land in the wrong checkout: bash persistent-cwd drift, phantom file reverts, which tree a tool really touched"
|
||||
---
|
||||
|
||||
# Multi-worktree gotchas in ~/.omp-style repos
|
||||
## Purpose
|
||||
Answer "why did my file/tests revert or behave differently than what I just wrote?" in repos with several worktrees (e.g. `~/.omp`, `~/Projects/wt-*`, `~/.omp/wt/*`).
|
||||
|
||||
Measured 2026-10-18 building PR #12/#14 in `dank/command-center` worktrees.
|
||||
## Failure modes and fixes
|
||||
|
||||
## 1. Relative paths resolve to the SESSION worktree — always edit by absolute path
|
||||
`edit`/`read`/`write` with a relative `path` resolve against the session CWD
|
||||
(the harness worktree you were spawned in), NEVER the worktree you `cd`'d into
|
||||
via bash. Identical file content in sibling worktrees yields identical
|
||||
`[path#tag]` hashes, so an edit can "succeed" against the WRONG checkout with
|
||||
no warning; only a rejected stale-hash edit reveals it.
|
||||
1. **Bash tool persistent-cwd drift** (the expensive one): the persistent shell keeps its cwd across calls; a call *without* an explicit `cwd` runs in whatever the *previous* call left, which after interleaved work may be a DIFFERENT worktree. Result: reads/greps/appends hit tree A while `git`/`bats` hit tree B — symptoms are "file keeps reverting to HEAD", duplicate test names at shifting line numbers, scenario `CALLS 0` for scenarios that exist, mismatched `grep -n` vs tool-reported line numbers.
|
||||
- **Diagnose:** `pwd` in the same call as the failing command; `wc -l <file>` in BOTH candidate worktrees; compare md5s. The file is not flapping — two files exist and commands alternate between them.
|
||||
- **Fix/prevent:** pass an explicit `cwd` on EVERY bash call once two or more checkouts are in play; never trust a read taken without one as evidence about "the" file.
|
||||
|
||||
- When working in a second worktree, ALL file-tool paths: absolute.
|
||||
- Symptom seen: edit rejected "hash #X is not from this session" while the
|
||||
bash-side `git status` showed the file clean — two different files, two
|
||||
worktrees. Recovery: revert the stray edit (CUT the inserted lines) in the
|
||||
wrong tree, re-read the right tree, redo with absolute header.
|
||||
2. **Symlinked or stale `git` state**: `git status` in a worktree shows a clean tree while the disk file differs from what you intended — the corruption is already committed. Audit committed blobs (`git log --oneline -- <file>`; per-commit `git show <c>:<path> | grep -c <sentinel>`) instead of re-reading the disk copy repeatedly.
|
||||
|
||||
## 2. Branching: shared worktrees refuse git checkout
|
||||
Workspace policy blocks `git checkout` (branch switches AND `--` discards) in
|
||||
harness-managed worktrees — concurrent agents depend on them. To start a
|
||||
branch off fresh trunk:
|
||||
3. **Same-name files across worktrees** (`tests/behavior.bats` exists in each): a `find`/`grep` rooted above the worktrees mixes copies; anchor the search path to one worktree root.
|
||||
|
||||
```bash
|
||||
git -C ~/.omp worktree add -q -b <type>/<issue#>-<slug> ~/.omp/wt/<name> origin/main
|
||||
```
|
||||
|
||||
Never force past the refusal; the new-worktree route is strictly safer.
|
||||
Custom worktrees not matching `wt/wt-*` are invisible to the live-sync reaper —
|
||||
prune them by hand after merge (`git worktree remove`, `git branch -d`).
|
||||
|
||||
## 3. omp session id for attribution footers
|
||||
Fleet law requires `session:` in issue/PR bodies. Authoritative source:
|
||||
`~/.omp/agent/sessions/<cwd-slug>/<timestamp>_<UUID>.jsonl` — the `<UUID>` is
|
||||
the session id. Pick the newest file whose first `{"type":"title"}` line names
|
||||
this conversation to prove the id is yours. The `wt/<ts>-<hash>` dir suffix is
|
||||
NOT the session id (it persists across sessions per cwd).
|
||||
|
||||
## 4. edit-tool auto-prefix trap (recurring)
|
||||
A body row written bare (intending it as the "original line" anchor or as
|
||||
context) is AUTO-PREFIXED with `+` and INSERTED — leaving the old line plus a
|
||||
duplicate. `PUT n.=n:` therefore behaves like insert, not replace, whenever
|
||||
the anchor row lacks an explicit marker. After any `PUT` where the response
|
||||
warns "Auto-prefixed bare body row(s)", immediately `CUT` the stale line number
|
||||
shown in the response (descending order for multiple CUTs).
|
||||
## Repair recipe for a corrupted-but-committed test file
|
||||
- Inventory both versions: `grep -o '@test "[^"]*"' | sort` and `grep -o 'case "[a-z]*":' | sort -u`; `comm` the sets to find lost scenarios and duplicate tests.
|
||||
- Rebuild from the committed copy (`git show HEAD:<path> > <path>`, no checkout), then re-apply only the true deltas: restore missing scenario blocks from the last clean base, dedupe imports, rename scratch-dir mkdtemp prefixes that collide with a test's own orphan-census glob (e.g. `dankvoice-stage-` matching `dankvoice-*`).
|
||||
- Re-run the targeted tests, then the full suite single-process.
|
||||
|
||||
@@ -1,27 +0,0 @@
|
||||
---
|
||||
name: one-decision-per-ask-question
|
||||
description: "Use when putting decisions to Frank via the ask tool or a grilling round — exactly one work item per question, rounds may group many single-item questions."
|
||||
---
|
||||
|
||||
# One Decision, One Work Item, One Question
|
||||
|
||||
Friction-earned rule (corrected twice on 2026-10-18, during the voice/TTS planning).
|
||||
|
||||
## Rules
|
||||
|
||||
- Each question addresses **exactly one work item**. A round may contain many questions covering different work items; a question may not.
|
||||
- Never merge two artefacts into one option because they share an implementation dependency. Sharing an endpoint, a service, or a build step is NOT grounds for one question. (`frank-tts` serving both a podcast and an audiobook generator produced exactly this bug.)
|
||||
- Never offer something Frank did not ask for inside another question's option text. If an idea is mine, it gets its own question or stays out of the plan; it never rides along as "…and also X" in an option label or preview.
|
||||
- Do not derive option wording from my own scaffolding (todo text, my table row labels, my runbook headings). Derive it from his stated goal.
|
||||
- Mark the recommended option; a bare "yes" must accept it.
|
||||
- Facts are mine to fetch (subagent/scout), never a question for him. Only decisions go to him.
|
||||
- A question whose answer depends on an unsettled fact or another open question belongs to a **later round**, not this one.
|
||||
|
||||
## Anti-patterns that triggered this
|
||||
|
||||
- One option: "Audiobook/podcast factory — open-notebook + EPUB→narration builder." Two products, one shared dependency, zero requests for the second one. He answered "I never asked for audiobook generation."
|
||||
- One question: "Is the deliverable the podcast, or the platform (harness voice + OpenWebUI + alerts)?" Four work items in one question. Rejected: "You're lumping multiple asks into one question. Ask me per item."
|
||||
|
||||
## Check before sending
|
||||
|
||||
For each question: can I name the single artefact it decides? If two names come to mind, split it.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
name: open-notebook-provision-model-rest
|
||||
description: "Provision or repair an open-notebook model (LLM/TTS/embedding) over its REST API on Dankclaw: credential→model→defaults order, the defaults-store-the-ID trap, the /api/embed 500-that-succeeds, and the two-note ranking proof for embeddings."
|
||||
---
|
||||
|
||||
# open-notebook model provisioning over REST
|
||||
|
||||
Use when adding/repairing a model row (embedding, LLM, TTS) in open-notebook on Dankclaw, when semantic
|
||||
search is dead, or when an embed error appears in the worker log. Config lives in the **database**
|
||||
(encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`) — no repo file mirrors it, REST or the UI are the only doors.
|
||||
|
||||
## Access (never put the bearer in argv)
|
||||
|
||||
The API bearer IS `OPEN_NOTEBOOK_PASSWORD`; the API publishes on loopback `127.0.0.1:5056` only, so run
|
||||
from Dankclaw over ssh. `ssh dankclaw 'curl -H "Authorization: Bearer …"'` leaks the secret into
|
||||
`/proc/<pid>/cmdline` — use a mode-600 curl config:
|
||||
|
||||
```bash
|
||||
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
|
||||
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
|
||||
B=http://127.0.0.1:5056 # not 5055: seerr holds the host's 5055
|
||||
```
|
||||
|
||||
## Order, and the address rule
|
||||
|
||||
`POST /api/credentials` → `POST /api/models` (`credential: <credential id>`) → `PUT /api/models/defaults`
|
||||
→ any profile that names a model (speaker `voice_model`, episode `speaker_config`/`outline_llm`).
|
||||
|
||||
- Credential provider for any in-house OpenAI-compatible server: **`openai_compatible`** (not `openai`),
|
||||
with a dummy `api_key` when the upstream does not authenticate.
|
||||
- The app's URL guard refuses base URLs resolving in `169.254.0.0/16`, so `host.containers.internal`
|
||||
cannot be stored; a **container name on `dankclaw_default`** (e.g. `http://frank-dragoon-halogen:8731/v1`)
|
||||
or another machine's RFC1918 IP both pass.
|
||||
|
||||
## The two traps that eat the most time
|
||||
|
||||
1. **`PUT /api/models/defaults` takes the model ID, never the name.** `"default_embedding_model":
|
||||
"qwen3-embedding-0.6b"` is accepted and echoed back, then every job dies in the worker with
|
||||
`Model with ID qwen3-embedding-0.6b not found`. Use the row's `model:…` id (GET defaults first, change
|
||||
one key, PUT the whole object back — the other keys in that response are ids and must be preserved).
|
||||
2. **`POST /api/embed` answers 500 *after* the work succeeds.** Upstream `EmbedResponse.command_id`
|
||||
declares `str` and the worker returns a `RecordID`, so FastAPI raises in serialization only. The truth is
|
||||
`podman logs open-notebook | grep "Successfully embedded"`, never the HTTP code. Note saves
|
||||
(`POST /api/notes`) submit `embed_note` themselves; `/api/embed` is only for backfills.
|
||||
|
||||
## Proof it actually works
|
||||
|
||||
- Per row: `POST /api/models/<id>/test` → `Embedding dimensions: 1024` (embeddings) or a real completion.
|
||||
- Embeddings, discriminating test: create **two** unrelated notes, then
|
||||
`POST /api/search` `{"query": "...", "type": "vector", "search_notes": true, "limit": 5}` — the enum is
|
||||
`text|vector` (not `semantic`) and the score field is `similarity`. Each query must rank its own note
|
||||
high and the other low (measured 0.659/0.292 vs 0.906/0.318). One note proves nothing: `total_count` is 1
|
||||
either way. Delete the probe notes afterwards (`DELETE /api/notes/<id>`).
|
||||
- Confirm which engine served it: `podman logs <engine container> | grep "POST /v1/embeddings"`.
|
||||
|
||||
Owner pages: `serverAppConfigs-docs/Dankclaw/Open Notebook.md` (app-side shapes), and the endpoint's own
|
||||
page for whatever serves the model.
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: openwebui-tts-chain-audit
|
||||
description: Audit/repair OpenWebUI TTS on Dankclaw when cloned voices get rejected by a 29-name Kokoro enum or synthesis is silent; rootless DB + endpoint probes
|
||||
---
|
||||
|
||||
Audit OpenWebUI's TTS/STT wiring on Dankclaw when a voice is rejected, synthesis is silent, or someone asks "is the cloned voice wired correctly". Not for model-picker/ACL issues (openwebui-picker-acl-repair owns those).
|
||||
|
||||
## Shape of the stack
|
||||
- Container `open-webui` runs **rootless as `dank`** on Dankclaw: `ssh dankclaw 'podman exec open-webui ...'` (NO sudo: `sudo podman` sees an empty world). Host port 3005.
|
||||
- Server-side config = `webui.db` `config` table (admin), per-user overrides = `user.settings` JSON column. Settings changed via UI persist per-user; admin page persists to `config`.
|
||||
- Cloned voices live behind the frank-tts façade `:7772` (`/v1/audio/speech`, `/v1/audio/voices`); voices resolve to `/home/dank/voices/<name>/reference.wav`.
|
||||
|
||||
## The trap (2026-10-18 law)
|
||||
`Error: Voice "narrator1" not found. Should be one of: af_heart, ...` (29 names) is **OpenWebUI's bundled in-browser Kokoro** (`browser-kokoro`) rejecting client-side — frank-tts never sees the request. Signature: container logs show zero `audio/speech` hits. Fix = the personal dropdown Settings -> Audio -> TTS Engine -> Default (empty string), not any deploy.
|
||||
|
||||
## Probe sequence (read-only, safe)
|
||||
1. Admin truth:
|
||||
`podman exec open-webui python -c "import sqlite3; c=sqlite3.connect('/app/backend/data/webui.db'); [print(k,'=',v) for k,v in sorted(dict(c.execute('SELECT key,value FROM config')).items()) if 'audio' in k or 'tts' in k.lower()]"`
|
||||
Expect: `audio.tts.engine=openai`, `audio.tts.openai.api_base_url=http://host.containers.internal:7772/v1`, `audio.tts.model=frank-tts`, `audio.tts.voice=<cloned name>`.
|
||||
2. User overrides — a user pinned to `browser-kokoro` beats admin config. CAUTION: `user.settings` can be SQL NULL or the JSON string `"null"`:
|
||||
`... for uid,s in c.execute('SELECT id,settings FROM user'): d=json.loads(s) if s else None; d=d if isinstance(d,dict) else {}; hits={k:v for k,v in d.items() if 'audio' in k or 'tts' in k.lower()} ...`
|
||||
3. Endpoint truth: `curl -s http://127.0.0.1:7772/v1/audio/voices` (roster), then POST the exact OpenWebUI body `{"model":"frank-tts","input":"probe","voice":"narrator1","response_format":"mp3","speed":1.0}` to `/v1/audio/speech` — expect `200 audio/mpeg`.
|
||||
4. Contrast logic: server-path failure leaves a container log line AND a façade error; browser-kokoro failure leaves neither.
|
||||
|
||||
## Law
|
||||
- Full contract + diagnosis live in serverAppConfigs docs (`Dankclaw/OpenWebUI.md` § Text-to-speech, `Dankclaw/Frank TTS.md` troubleshooting). Docs are the source; this skill is just the probe route.
|
||||
- Rootless gotcha generalizes: any Dankclaw service probe through podman goes WITHOUT sudo unless the unit is a system quadlet.
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
name: parallel-suite-shared-db-flake
|
||||
description: "Diagnose test suites that pass isolated but fail only under concurrent/uncached runs against a shared database: cross-file teardown wipes, FK-cascaded sessions, wildcard LIKE deletes."
|
||||
---
|
||||
|
||||
# Flaky only under concurrency on a shared DB
|
||||
|
||||
Symptom: a suite passes `vitest run <file>` alone, fails under `turbo run test --force`
|
||||
(or any concurrent uncached runner) with impossible errors — 401 where 404 was
|
||||
expected, rows vanishing mid-test, sessions dying between two awaited requests.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Reproduce the concurrency, not the file.** Isolated runs prove nothing.
|
||||
`bun x turbo run test --force --output-logs=errors-only` (or run the two suspect
|
||||
files simultaneously). Never conclude "flake" from one green isolated run.
|
||||
2. **Map the impossible state to a vanished row.** 401/auth failures where the
|
||||
session was created milliseconds earlier = the session/user row was DELETED
|
||||
under the test, not expired. Look for writers, not readers.
|
||||
3. **Grep every suite's teardown for shared-namespace deletes:**
|
||||
`grep -rn "TRUNCATE\|DELETE FROM\|LIKE" <pkg>/src/*.test.*`
|
||||
The classic killer: two test files sign users up under the same domain
|
||||
(`%@studio.test`) and each `afterAll` runs a wildcard `DELETE ... LIKE '%@...'`.
|
||||
With parallel files on one DB, whichever file finishes first cascades away the
|
||||
other's live rows (user FK cascades kill sessions mid-flight).
|
||||
4. **Fix = scope, never coordinate.** Each suite deletes only rows it created:
|
||||
per-file email/id prefixes (`auth*` vs `rel*`), or track created keys and
|
||||
`WHERE email = ANY(...)`. Timing tricks (sleeps, retry) hide the collision.
|
||||
5. **Prove stability:** 3 consecutive `--force` uncached runs green, then the
|
||||
full gate. Cache makes everything look green; only uncached concurrent runs
|
||||
re-exercise the collision window.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Error signatures lie: `expected 401 to be 404` reads as an auth bug but is a
|
||||
deleted-row cascade.
|
||||
- Supertest `cookieOf`-style helpers that THROW on missing cookies prove signup
|
||||
succeeded — so the session died after creation; keep hunting for the deleter.
|
||||
- Shared dev DBs (one Postgres for all workspaces/worktrees) make every wildcard
|
||||
teardown a cross-suite weapon. Suite teardowns on a shared box MUST scope to
|
||||
rows the suite created.
|
||||
@@ -1,68 +1,34 @@
|
||||
---
|
||||
name: parallel-worktree-path-hazard
|
||||
description: "\"Use when several agents edit the same repo in different git worktrees at once, or when an edit/read may have landed in the wrong checkout: which path forms bind where, how to detect with git -C status, and the repair protocol (cp + git show HEAD blob write-back) because git checkout/restore is policy-blocked in a shared workspace.\""
|
||||
description: "Use when several agents edit the same repo in different git worktrees at once, or when an edit lands in the wrong checkout: relative tool paths resolve to the session cwd, not the shell cwd."
|
||||
---
|
||||
|
||||
# Concurrent-worktree path hazard (shared repo, many agents)
|
||||
Use when several agents (or you across two checkouts) edit the same repo in different git worktrees at once, or when an edit/query lands in the wrong checkout.
|
||||
|
||||
Measured 2026-10-18 on `serverAppConfigs` with two agents live at once. The
|
||||
failure is **silent**: the tool reports success, the diff looks right, and the
|
||||
change lives in a *different* worktree.
|
||||
## The silent failure: relative tool paths resolve to the SESSION cwd
|
||||
|
||||
## Which path form binds where
|
||||
- **Parent/top-level session**: a **relative** `path` resolves against the
|
||||
session CWD — its own worktree — never against the worktree named in the
|
||||
prompt. This is how a change meant for `wt-tts-trial-docs` landed in
|
||||
`wt-20261017-…`.
|
||||
- **Subagent**: an **absolute** path to its own assigned worktree can resolve
|
||||
into the parent's worktree when the same *relative* path exists there
|
||||
(`edit` twice onto `dankclaw/.env.example`). Its `write` with the same
|
||||
absolute path was fine — the bug is `edit`/`read` path resolution, not `write`.
|
||||
`read` / `edit` / `write` / `grep` resolve a **relative** `path` against the **session working directory** — never against the `cd` inside a neighboring `bash` call, and not against a `../other-worktree/...` prefix either (that gets normalized back into the session tree). So after `cd ../wt-B && grep …` you can `read path="docs/page.md"` and get **wt-A's** copy.
|
||||
|
||||
Rules that hold:
|
||||
1. A subagent passes paths as `../<its-worktree>/<rel>` (relative to the parent
|
||||
worktree) for `read` **and** `edit`; the parent passes **absolute** paths to
|
||||
any worktree that is not its own CWD.
|
||||
2. After every edit batch, prove the landing site, do not trust the tool's echo:
|
||||
`git -C <intended-worktree> status --short` must list the file. A file you
|
||||
edited that is missing from that list went somewhere else.
|
||||
3. Keep each work item on its own branch/worktree, and keep ONE integration
|
||||
owner (the parent) for shared files — `scripts/deploy-remote.sh`,
|
||||
`scripts/deploy-plan.py`, `tests/`, `serverAppConfigs-docs/index.md`,
|
||||
`AGENTS.md` are the collision hot spots in this repo. Never rebase a
|
||||
sibling's branch.
|
||||
Why it survives "I just read it to check": the returned path is printed workspace-relative and the file name is identical in every checkout, so the output looks like a correct read of the tree you meant. Only content that differs between the checkouts betrays it.
|
||||
|
||||
## Repair protocol (no `git checkout`, no `git restore`)
|
||||
`git checkout -- <path>` and `git restore <path>` are **policy-blocked** in a
|
||||
shared workspace ("overwriting working-tree files … discards changes other
|
||||
concurrent agents rely on"). Unstaging (`git reset [<path>]`,
|
||||
`git restore --staged <path>`) is allowed but does not undo content. So:
|
||||
Rules:
|
||||
- **Absolute paths for every tool call that targets a worktree other than the session's.** No exceptions for quick reads.
|
||||
- Confirm the checkout from content you know differs (a heading, a status line), not from the echoed path.
|
||||
- A `read` error like "Did you mean .AGENTTRASH/<name>" is a signal the relative path resolved into the session tree — re-issue absolute.
|
||||
- Same trap for `bash`: it starts each call in the session cwd, so `cd` twice per call or use absolute paths there too.
|
||||
|
||||
```bash
|
||||
F="path/with spaces if any"
|
||||
# 1. move the change to the worktree that should own it (both at the same base commit)
|
||||
cp "<wrong-wt>/$F" "<right-wt>/$F"
|
||||
# 2. undo in the wrong worktree by writing the committed blob back
|
||||
git -C "<wrong-wt>" show "HEAD:$F" | tee "<wrong-wt>/$F" > /dev/null
|
||||
# 3. prove both sides
|
||||
md5sum <(git -C "<wrong-wt>" show "HEAD:$F") "<wrong-wt>/$F" "<right-wt>/$F"
|
||||
git -C "<wrong-wt>" status --short # must be empty for that file
|
||||
git -C "<right-wt>" diff --stat # must show the intended delta
|
||||
```
|
||||
## Landing a change when the session worktree is stale
|
||||
|
||||
Never fix it blind: if the wrong worktree had its own uncommitted edits to that
|
||||
file, `cp`/blob-write-back would destroy them — check
|
||||
`git -C <wrong-wt> status --short` FIRST and reconcile content, don't overwrite.
|
||||
`git diff --name-only HEAD origin/main` first. If main moved a file you are editing, your edit's base is wrong even though the file "looks fine" — do not commit on the stale base.
|
||||
|
||||
## Tidy after a cancelled sibling
|
||||
A killed subagent can leave a worktree plus an empty branch. Removing them is
|
||||
safe only when verified empty: `git -C <main> log --oneline -3 <branch>` (no
|
||||
unique commits) and `git -C <wt> status --short` (clean), then
|
||||
`git worktree remove <wt>` + `git branch -D <branch>`. An orphan worktree at a
|
||||
stale commit is the same class of trap as a leftover bench drop-in: the next
|
||||
agent reads it as unfinished work.
|
||||
1. `git worktree add ../wt-clean -b <type>/<slug> origin/main`; copy the new files in.
|
||||
2. Re-apply the edit against the clean tree's version **with absolute paths**, then read it back absolutely to prove which tree changed.
|
||||
3. Run the suite inside the clean worktree; commit there.
|
||||
4. Remove your own duplicate edit from the session worktree by editing it away (`CUT`/`PUT`). `git checkout -- <path>` and branch switches are policy-blocked in shared workspaces and should stay blocked — they discard edits other agents rely on.
|
||||
5. Never carry the same uncommitted edit in two worktrees at once; if you must keep both trees, keep a task that reconciles them.
|
||||
|
||||
## Report it
|
||||
File the misresolution to `xd://report_issue` (false positives are fine) and
|
||||
record the incident in the repo worklog — a repair nobody wrote down looks like
|
||||
a hijack later.
|
||||
## Multi-agent hazards in one shared worktree
|
||||
|
||||
- Another agent can stage, commit, or check out files under you; re-read a file immediately before editing it if any tool result hinted at intervening change.
|
||||
- Untracked scratch you drop in a shared tree (reports, notes, `.AGENTTRASH/` entries) is visible to everyone and can be committed by them. Put throwaway artifacts outside the checkout.
|
||||
- Never `git clean -f`, `rm -rf`, or branch switching to "get a clean state" in a tree another agent is using.
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: pdf-lib-determinism-moddate-leak
|
||||
description: "Diagnose pdf-lib output that isn't byte-identical across fetches: PDFDocument.load restamps /ModDate; fake-clock determinism test; inflate object streams to diff."
|
||||
---
|
||||
|
||||
# pdf-lib byte-identical output: the /ModDate clock leak
|
||||
|
||||
## Symptom
|
||||
A "reconstruct-on-read, byte-identical forever" PDF endpoint passes its back-to-back refetch test but returns **different bytes when fetches cross a wall-clock second**. Same-second fetches match; a fetch 1s later differs (often a different stream `/Length`, e.g. object stream 411→420 bytes).
|
||||
|
||||
## Root cause
|
||||
`PDFDocument.load()` has its own `updateMetadata` option (**defaults true**) and stamps `/Info /ModDate` from the wall clock **at load time**, not just at save. `doc.save({updateMetadata:false})` on its own is not enough (and in pdf-lib 1.17 `updateMetadata` isn't even in the `SaveOptions` type — only in `PDFOptions` on `load`); passing it to `save` also fails typecheck.
|
||||
|
||||
## Fix
|
||||
```ts
|
||||
const doc = await PDFDocument.load(templateBytes, { updateMetadata: false });
|
||||
// ... fill, stamp print layer ...
|
||||
return doc.save(); // save never stamps when load didn't
|
||||
```
|
||||
Output becomes a pure function of (template bytes, values).
|
||||
|
||||
## Determinism test law
|
||||
A determinism test MUST straddle a time boundary. Back-to-back fetches/fills are structurally blind to per-second stamping.
|
||||
- Unit: fake the clock (`@sinonjs/fake-timers` or `vi.useFakeTimers`) → fill → jump +2 months → re-fill → `expect(Buffer.from(second)).toEqual(first)`. No sleeps, no flake window. Faking `Date` outside a test framework requires patching `globalThis.Date` BEFORE the module import that uses it.
|
||||
- Prove it red first against the old code (expect the byte/length drift).
|
||||
|
||||
## Byte-diff forensics (when lengths differ)
|
||||
1. Dump the differing region raw: `Buffer.subarray(idx-60, idx+60)` rendered latin1 with non-printables stripped — locate the exact field (`D:YYYYMMDDHHmmss` = clock leak).
|
||||
2. If the diff is inside a compressed object stream, inflate and compare streams:
|
||||
```ts
|
||||
import { inflateSync } from "node:zlib";
|
||||
// find /Type /ObjStm ... stream chunks, inflateSync each, string-compare
|
||||
```
|
||||
3. pdf-lib escapes text-show operands — inflate + unescape `\\(\\[()\\\\]\\)` before substring-asserting printed values.
|
||||
|
||||
## Live proof recipe
|
||||
Seed a submission, then fetch both retrieval doors at t, t+1.5s, t+3s; `sha256sum` all three → one hash. Same-second comparisons prove nothing. And kill any long-running dev service holding the app port first — a live :3000-style stack can hang or false-fail an e2e gate.
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
name: pdf-widget-layer-census
|
||||
description: "Census a real-world \"flat\" PDF before assuming it needs field injection: Quartz/Preview exports often already carry an AcroForm widget layer under underscore rules; also audits pdf-lib flatten residue."
|
||||
---
|
||||
|
||||
# Real-PDF census (widget-first, then judge)
|
||||
|
||||
A PDF that *looks* flat (`pdftotext` shows `____` fill rules) may already contain a
|
||||
full AcroForm widget layer — macOS Preview/Quartz exports write widgets with
|
||||
transparent rects under drawn underscores. Injecting fields into such a file
|
||||
duplicates/shadows widgets. Always census before touching the pipeline.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Surface facts** (host):
|
||||
```bash
|
||||
pdfinfo file.pdf # "Form: AcroForm" hints at a widget layer
|
||||
pdftotext -layout file.pdf - # label lines + underscore rules = intended fields
|
||||
```
|
||||
2. **Widget census** — authoritative check. With the franksign sidecar image,
|
||||
the image ENTRYPOINT *is* `inject_fields.py`; override it:
|
||||
```bash
|
||||
podman run --rm --entrypoint python -v ~/Downloads:/in:z \
|
||||
localhost/franksign-pdf-field-sidecar:local -c "
|
||||
import pymupdf
|
||||
d = pymupdf.open('/in/file.pdf')
|
||||
for p in d.pages():
|
||||
for w in p.widgets():
|
||||
print(w.field_name, w.field_type_string, [round(v) for v in w.rect])
|
||||
print('widgets:', sum(1 for p in d.pages() for _ in p.widgets()),
|
||||
'annots:', sum(1 for p in d.pages() for _ in p.annots()))
|
||||
"
|
||||
```
|
||||
3. **Fill test**: if widgets exist, `fillTemplate()` (pdf-lib) consumes the file
|
||||
AS-IS — spec.json + sidecar are unnecessary for it. Flatten-verify below.
|
||||
|
||||
## Flatten audit gotchas
|
||||
|
||||
- `pdfinfo` prints `Form: AcroForm` even on correctly flattened pdf-lib output —
|
||||
pdf-lib `flatten()` removes every *widget/annotation* but leaves the field
|
||||
*dictionary*. Truth = MuPDF widget enumeration (0), not the pdfinfo line.
|
||||
- `describeForm(bytes, specJson)` in `packages/pdf` parses the SPEC, not the
|
||||
PDF, for the field list — pass a spec or skip it; calling it spec-less throws
|
||||
`JSON Parse error`.
|
||||
|
||||
## Decision rule
|
||||
|
||||
- widgets > 0 → the widget layer IS the spec (names/types/rects, flip y to
|
||||
bottom-left origin); NEVER run `inject_fields.py` over it.
|
||||
- widgets = 0 → genuine flat form: sidecar injection with a spec (position
|
||||
detection = issue #14 territory: underscore-run rects, label-derived names).
|
||||
- Missing signature widget is normal ("please sign" is prose); franksign
|
||||
decision 19: the typed full name is the signature.
|
||||
|
||||
## Privacy
|
||||
|
||||
Real documents are personal: scratch copies in /tmp, placeholder fill values,
|
||||
destroy scratch (`rm /tmp/*` is fine OUTSIDE the repo; repo deletions still go
|
||||
to `.AGENTTRASH/`). Never commit the document, a spike script touching it, or
|
||||
its extracted text into the vault.
|
||||
@@ -0,0 +1,61 @@
|
||||
---
|
||||
name: plugin-git-install-committed-artifact
|
||||
description: "Use when shipping or landing an omp/pi plugin (omp plugin install/upgrade git): install validation fails on a missing build artifact, a stale bun git-mirror cache is reused, the live bundle needs proof it is the merged build, or upgrade says \"not installed\"."
|
||||
---
|
||||
|
||||
# Plugin install from git: committed artifact + fresh mirror
|
||||
|
||||
`omp plugin install git:<url>` (and `pi install git:`) **clone the repo and validate the manifest's declared extension entry WITHOUT running a build step**. Two failure classes seen live (dank/dankvoice, 2026-10-19):
|
||||
|
||||
## 1. Declared entry missing from the clone
|
||||
|
||||
Symptom: `Plugin <name> extension validation failed: ./dist/index.js: declared extension entry not found on disk`.
|
||||
|
||||
Cause: build output (`dist/`) is gitignored; the manifest points at it; the clone has only `src/`.
|
||||
|
||||
Fix — ship the artifact:
|
||||
- Remove `dist/` from `.gitignore`, `bun run build`, commit `dist/`.
|
||||
- Prove build determinism first: build twice, `cmp` the outputs (bun esm builds are byte-reproducible without sourcemaps).
|
||||
- Pin the contract in a small bats suite so it can't rot:
|
||||
1. the manifest's declared entry is tracked in git (`git ls-files -- "${entry#./}"` non-empty — this is the installer's exact contract, not a config assertion);
|
||||
2. committed bytes equal a fresh build (`bun run build` then `git diff --name-only -- dist` is empty → red on drift).
|
||||
|
||||
## 2. Stale bun git-mirror cache
|
||||
|
||||
Symptom: fix pushed, retry **still** fails identically, and the retry is suspiciously fast (seconds).
|
||||
|
||||
Cause: bun keeps bare mirrors in `~/.bun/install/cache/<hash>.git` and may reuse them at an old commit — even after the fix is pushed and the default branch is correct.
|
||||
|
||||
Diagnose + fix:
|
||||
```bash
|
||||
find ~/.bun/install/cache -maxdepth 2 -iname "*<repo>*"
|
||||
git --git-dir ~/.bun/install/cache/<hash>.git log --oneline -3 --all # stale HEAD?
|
||||
rm -rf ~/.bun/install/cache/<hash>.git # it is a cache; safe
|
||||
```
|
||||
Prove the fetch path independently first (also validates credentials for private Gitea repos):
|
||||
```bash
|
||||
GIT_TERMINAL_PROMPT=0 git clone -q <url> /tmp/x && git -C /tmp/x log --oneline -1 && ls /tmp/x/dist
|
||||
```
|
||||
|
||||
## Order of investigation (don't skip)
|
||||
1. Gitea default branch correct? (`GET /api/v1/repos/<o>/<r>` with token — private repos 404/None unauthenticated.)
|
||||
2. Plain unattended clone contains the entry? If yes, the installer's view is cached → purge mirror.
|
||||
3. Only then suspect the installer itself (validation code lives in the minified bundle; `grep -oE '.{600}<error string>' "$bunfile"` on `readlink -f $(which omp)` shows the exact check).
|
||||
|
||||
## Landing a merged PR on the live box
|
||||
|
||||
After the merge and the `--ff-only` on the dev checkout (`ls -ld node_modules` after landing anything that could have touched it):
|
||||
|
||||
- **`omp plugin upgrade <name>` takes the PACKAGE name, not the install alias.** `omp plugin upgrade dankvoice` fails `dankvoice is not installed` while the live copy sits in `~/.omp/plugins/node_modules/omp-dankvoice/`. Read the directory name (or `package.json` `"name"`) and upgrade that.
|
||||
- Then close the chain of custody with one command — the deployed bytes are the merged bytes:
|
||||
```bash
|
||||
cmp ~/Projects/<repo>/dist/index.js ~/.omp/plugins/node_modules/<pkg>/dist/index.js
|
||||
```
|
||||
If only docs/tests changed, `dist/` is untouched and this should already match; if `src/` changed, the plugin must be upgraded first or this fails with `char <n>` — which is a real finding (stale live bundle), not a test bug.
|
||||
- The session that did the merge keeps the OLD bundle in memory: it is still running pre-merge code. Never conclude "the change does not work" from the merging session.
|
||||
|
||||
## After install — proof without a TUI
|
||||
- `omp plugin doctor` → per-plugin ✔ line.
|
||||
- Audible/behavioral chain when TUI input is unusable (update-banner modal freezes pty submits): run `omp --mode=rpc --no-session` with the feature env armed (dankvoice: `DANKVOICE_AUTO=1`, gate is print/json-only) and watch for objective side effects, e.g. PipeWire: `pw-dump | grep -q "say-<ms>.wav"` proves synth+playback end-to-end.
|
||||
- When pty submits DO work (no modal), the strongest instrument is a stub player plus a clip count, not ears or pixels: `DANKVOICE_PLAYER=/tmp/stub.sh` where the stub appends `$1` to a log and exits 0. Launch the DEPLOYED bundle by path (`omp --no-extensions --no-session -e ~/.omp/plugins/node_modules/<pkg>/dist/index.js`) in a neutral cwd with `DANKVOICE_CONFIG` pointed at a scratch file so the operator's settings are untouched.
|
||||
- **Count clips per narration EVENT, not per turn.** One turn can fire several narrations (dankvoice: `message_end` carries prose+thinking, `agent_end` the settled answer; the dedupe only skips byte-identical text, so thinking-on means both speak). A "too many clips" result is usually correct behaviour — check the harness transcript for the turn count before believing a bug. The player deletes each temp wav, so the log line count is the only surviving evidence: park it in `~/.AGENTTRASH/<date>-<topic>/` if it becomes evidence for an issue.
|
||||
@@ -0,0 +1,51 @@
|
||||
---
|
||||
name: radiance-decode-prefill-interference-census
|
||||
description: "Diagnose decode collapsing while another session prefills on a radiance endpoint: phase-ratio step counters, lag test, metric-prefix and scrape traps, and which knob acts on the measured mechanism."
|
||||
---
|
||||
|
||||
Read `core/config.cpp` (help table + validator) in the PINNED image source before writing any radiance flag, enum member, or metric name into a runbook: `config_parse` refuses unknown enum values at boot, so a remembered name can make the recorded arm undeployable. Enumerate members from the validator's error string, never from memory.
|
||||
|
||||
**Run the census on the pinned build's own metric set.** An older pin ranks the wrong mechanism: on radiance 1.0.13 the bucket table put a "collapse" on a shape whose counters did not exist yet, because every counter resets at a restart and the metric set changes with the build. Prove the window is one shape (`up`, or the first sample of a counter only the current build exports) before ranking buckets.
|
||||
|
||||
## Step ledger first, knobs second
|
||||
|
||||
Three counters decide which mechanism you are looking at, and none of them is a rate:
|
||||
|
||||
- `radiance:engine_step_seconds_decode_total / engine_steps_decode_total` = mean **decode-only** step.
|
||||
- `(engine_step_seconds_total - engine_step_seconds_decode_total) / (engine_steps_total - engine_steps_decode_total)` = mean **chunk step** (a chunk alone, or a chunk with decode rows beside it).
|
||||
- `engine_steps_held_total` — if it is zero, nothing is protecting decoders.
|
||||
|
||||
If decode-only steps stay ~15 ms while the rate collapses, the mechanism is **step-time sharing** (a decode row in a chunk step waits the chunk's whole time), and any eviction/residency story is worth ~10 % of one step. The residency half is not censusable: the arena / expert slab / heat engine / prefill stager have **no metric at all** — they are read from the boot ledger, `/stats`, and `/diagnostics`.
|
||||
|
||||
## Bucket, then lag-test, before believing a cause
|
||||
|
||||
Bucket `radiance:decode_tokens_per_second` by `radiance:prefill_tokens_per_second` from the *same* scrape, step ≤ 30 s (a 15 s scrape against a 120 s step samples one in eight). Per-stream rate = decode gauge ÷ `vllm:num_requests_running`; the tps gauges are ~1 s decayed batch-pooled windows.
|
||||
|
||||
Then shift the prefill series by lags and re-bucket:
|
||||
|
||||
- lag-0-only ⇒ step-time sharing — the chunk is holding the device.
|
||||
- a lagged tail (2–14 min) ⇒ something evicted (arena loan, expert slab, KV).
|
||||
- bursts in `vllm:num_preemptions_total` ⇒ a preempted sequence stopped and is re-prefilling from token 0: a stall, not sludge.
|
||||
|
||||
Also bucket TTFT (`histogram_quantile` over `vllm:time_to_first_token_seconds_bucket`) by the same series: the decode gauge only sees sequences already running, so an arriving request pays the same steps and shows in no decode number.
|
||||
|
||||
## Metric traps
|
||||
|
||||
- **Latency is `vllm:`, cadence is `radiance:`.** TTFT / inter-token / per-output-token / e2e / decode-time histograms and the running/waiting gauges exist only under the vLLM compatibility prefix. Preemption is `vllm:num_preemptions_total` (counter); `radiance:num_requests_preempted` is the *instant* depth. A `radiance:`-only query finds no latency and no preemption and reads as "the endpoint never stalls anyone".
|
||||
- **A 2-minute mean step time dilutes the chunk** — if ~5 % of steps carry a third of the step time, `step_seconds/steps` cannot move. Use the phase ratio.
|
||||
- **`prefix_hit_cut_tokens_total` is prefill load you created yourself**: on a hybrid (linear + paged) model a hit reuses only to the last linear-state checkpoint, so cached tokens replay as prefill. A high prefill rate is not always a client sending new tokens.
|
||||
- Radiance's own step-detail histograms (`prefill_steps_duration`, `prefill_step_tokens`, `token_queue_wait`) may be exported but unscraped; per-request forensics then mean `curl localhost:7770/stats` on the endpoint's host, and `/diagnostics` answers "is the *box* the limit" in one call (configuration drift, VRAM split, per-card link chain and clocks, findings with fixes).
|
||||
- Rootless container logs: read as the deploy user (`podman logs <svc>`), never `sudo podman logs`; check where the serve script redirects before trusting a log file.
|
||||
- In a temporary/remote shell, zsh aborts on a non-matching glob — quote or guard `/sys/...` wildcards.
|
||||
|
||||
## Knob semantics (radiance 1.8.x, GUIDE §4.5; pinned source is referee)
|
||||
|
||||
- `--prefill-interleave N` — the **decoder dial**: N decode-only steps follow any chunk step taken beside decoders; held steps admit nothing. Costs the prefilling prompt's TTFT and a queued request its first step. Counts as `engine_steps_held_total`.
|
||||
- `--busy-prefill-chunk` — a **prefill-speed dial**, not decoder protection: `auto` reserves 1+spec rows per decoding sequence and 1/8 of the budget per waiting request, and the source refuses to halve for decode rows alone. Never below the prefill stager's smallest pass (named on the boot line) — below it every routed layer reads its pooled experts synchronously.
|
||||
- `--prefill-span N` — needs `--max-num-batched-tokens` = N intervals; re-funds a *lone* prefill only, so it cannot buy back held steps while decoders run, and its extra intervals land on the arena that the expert slab loses.
|
||||
- `--checkpoint-policy` — retention only, three names (`geometric-backoff` | `tip-only` | `keep-all`); snapshot writes happen at every interval whatever the policy says (only `--prefill-span` skips them), so `keep-all` buys a bounded replay for pinned host RAM and no extra PCIe. Shortening `--checkpoint-interval` does NOT fix deep divergence: the geometric ladder's coarsest scale is the largest `interval × 2^k` under the depth, so the deepest gap stays ≈ half the prompt while every cut multiplies the writes.
|
||||
- `--checkpoint-slots` is a **total**, `auto` = policy `held()` + 1, per `--max-num-seqs`, within an eighth of host RAM; `--max-num-batched-tokens` is the blunt rung — it raises the arena the slab loses and its resonance on an expert-tiered payload can be violent.
|
||||
|
||||
## Verify an arm, don't assume it
|
||||
|
||||
Flags may be passed conditionally by the serve script, so prove the flag reached argv (`tr '\0' ' ' < /proc/$(pidof <engine>)/cmdline`), then the counter the knob moves (`engine_steps_held_total` must leave zero), then the bucket pair. A deploy is not evidence a knob engaged. Rollback for each of these is its env line, not a rebuild.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user